Determination of number of signals based on minimum MSE

A criterion based on the minimum MSE (mean square error) is given to determine the number of signals mixed with white Gaussian noise in order to estimate their frequencies based on the modified minimum square error method using a higher-order linear prediction model (CLS). Being different from the usual autoregression (AR) model of linear prediction, the method has the advantage of iterative reduction of noise by subtraction of noise variances from diagonal elements of the correlation matrix of observed signal disturbed by noise at the instance of taking an inversion. But it needs precise estimation of both the rank of the correlation matrix and variances of noise by using only observed signals. In this paper, the following three points are made: (1) the analytic method to determine the order of eigenvalues which minimizes MSE criterion derived from eigenvalue analysis; (2) the formula which must be satisfied by noise variances; and (3) the iterative algorithm which gives simultaneous solution for rank order determination of the correlation matrix and estimation of noise variances. Finally, numerical examples are given to show the usefulness and correctness of the proposed MSE criterion, and some comparisons with other methods are made.