Imitating Human Selective Attention Using Dual Policy Network for Scanpath Prediction

Understanding how selective attention influences human gaze behaviors is important for behavioral vision and visual psychology. However, most existing scanpath models ignore the internal sub-stages in visual search and do not imitate human-like selective attention. To bridge this gap, a novel inverse reinforcement learning model with a dual policy network (DPNet) is proposed to accurately predict how humans select and shift their attention at different stages of a task. Additionally, to establish the semantic correlation between objects for state representation modeling, this paper employs Augmented Belief Maps (ABMs). Besides, an Option-Viterbi module is introduced to infer sub-task options of the real human scanpaths, learning the sub-task switching during visual search. Experimental results on widely used datasets for visual search justify the effectiveness of our method in terms of scanpath similarity and sub-task switching.