Dynamic error spectrum for IMM performance evaluation

The commonly used root-mean-quare error (RMSE) for estimation performance evaluation is easily dominated by large error terms. Then many new alternative absolute metrics are proposed in [1]. But each of these metrics is incomprehensive and only reflects one narrow aspect of estimation performance respectively. A comprehensive measure, error spectrum, was presented in [2] aggregating all these incomprehensive measures. It gives a curve at every time instant and disclose more aspects of estimation performance at the same time. However, when applying this measure to dynamic systems, it will plot a 3D figure over the total time span, which is not intuitive and difficult to be analyzed. To overcome its drawback, this paper provides a metric, called dynamic error spectrum (DES), to summarize the ES curve in three different forms, one of which is balanced considering both good and bad behavior and so can obtain more impartial evaluation results. DES can be applied to a variety of dynamic systems directly. Then we choose interacting multiple model algorithm (IMM) as the testing case to illustrate the superiority of our DES method comparing with RMSE. After analyzing different factors that affect the performance of the IMM algorithm, the simulation results verify the utility and effectiveness of this measure.