Visual Tracking Using Online Deep Reinforcement Learning with Heatmap

Visual tracking can be formulated as a Markov decision process over a parameterized family of policies. In this case, the problem of visual tracking is to make decisions about whether and how to adjust the agent. This paper introduce an end-to-end methodology which contains a framework from motion prediction to online update for single object tracking. For the prediction phase, we incorporate heatmap with appearance feature to learn a deep metric, and we introduce Region Proposal Network to regress a reliable location. For the online update phase, we take tracking as agent decision making process via learning a policy to pick a proper action about whether and how to update the state transition using Actor-Critic Network. The proposed tracking framework can be trained as an end-to-end fashion, and we demonstrate that our tracking performance is rather competitive with other state-of-the-art visual tracking algorithms. Other than this, from the experimental results we can see that, our prediction network using heatmap learning can acheive rather robust results under usual circumstance, and using reinforcement learning to make online decisions can be helpful when dealing with more complicated cases.