Learning nonlinear generative models of time series with a Kalman filter in RKHS

This paper presents a novel generative model for time series based on the Kalman filter algorithm in a reproducing kernel Hilbert space (RKHS) using the conditional embedding operator. The end result is a nonlinear model that quantifies the hidden state uncertainty and propagates its probability distribution forward as in the Kalman algorithm. The embedded dynamics can be described by the estimated conditional embedding operator constructed directly from the training measurement data. Using this operator as the counterpart of the state transition matrix, we reformulate the Kalman filter algorithm in RKHS. For the state model, the hidden states are the estimated embeddings of the measurement distribution, while the measurement model serves to connect the estimated measurement embeddings with the current mapped measurements in the RKHS. This novel algorithm is applied to noisy time-series estimation and prediction, and simulation results show that it outperforms other existing algorithms. In addition, improvements are proposed to reduce the size of the operator and reduce the computation complexity.