Track-oriented evaluation of multi-target tracking without knowing ground truth

Evaluating the performance of multi-target tracking with respect to tracks rather than unlabeled estimated points is important and challenging. Existing approaches assume exact knowledge of the ground truth. However, this is far from the reality. This paper proposes a method to deal with the case of unknown ground truth by measuring the difference between mock tracks and the assumed targets in the measurement space. The mock tracks are generated using the tracking results (tracks) of the algorithm. The assumed (true trajectories of) targets are extracted from the observations using the prior knowledge of the target motion. The method assigns the mock tracks to the assumed targets and then calculates the metrics. To solve the important and complex assignment problem, we propose a voting method, in which the assumed targets vote for the mock tracks. The voting rule is designed based on the prior knowledge. Incorporating the prior information and the online measurements, the proposed evaluation method makes good use of the mock data method and a voting strategy. Analysis and simulation demonstrate its effectiveness.