A lightweight discriminative tracker based on classification and similarity

Convolutional neural network (CNN) based trackers have achieved significant performances in tracking recently. Most existing CNN-based trackers regard tracking as a classification or similarity searching problem. The two methods have their respective superiorities and limitations because of different supervised objectives. In this paper, we propose a multi-task CNN for visual tracking, not only fully leveraging the training data, but also benefiting from a regularization effect that results in more general and discriminative representations that extend to tasks in new domains. Our multi-task CNN approach combines tasks of classification and similarity searching. Specifically, given a pair of examplar and search images, the network predicts the categories of the two images and search for the most similar regions to the examplar image in the search image. And then we use only the similarity module to conduct tracking, which makes our tracker operate at frame-rates beyond real-time. Extensive evaluation on the challenging benchmark sequences demonstrates that the proposed tracker performs favourably against the state-of-the-arts.