Nonlinear system identification based on recurrent neural networks with shared and specialized memories

Recurrent neural network (RNN) is an important method for nonlinear adaptive filtering in an uncertain environment. In this work, we propose an RNN with shared and specialized memories (SSMs) to improve the accuracy and computational complexity in online tasks. The specialized memory consists of the weights of the linear output layer, and the other nonlinear weights are shared memory. The shared weights are trained in an offline manner by data generated with possible environment parameters. Then they are frozen in online tasks while the specialized memory is adopted sequentially by utilizing the online training data. To improve the generalization capacity of our model, a dropout based recursive least squares algorithm is proposed. We apply our model for nonlinear system identification in both stationary and non-stationary condition. The experiment results show that the performance of our method is superior to the state-of-the-art methods.