TY - GEN
T1 - SCANPATH PREDICTION VIA SEMANTIC REPRESENTATION OF THE SCENE
AU - Zhang, Kepei
AU - Lu, Meiqi
AU - Lu, Zheng
AU - Zhang, Xuetao
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Aiming at the problem that the current scanpath prediction methods have insufficient representation of object association, we propose a scanpath prediction model based on semantic representation of the scene. Our model uses a panoramic segmentation network to separate object instances and backgrounds in scenes, and uses the attention mechanism to learn the semantic correlation between objects, which effectively extracts the deep image information related to the current task. We also propose a dual-branch structure predicting the fixation position and duration simultaneously, to fully simulate the temporal and spatial distribution of the human eye's attention in visual search. Experimental results show that our model has obvious advantages over the existing scanpath prediction methods in search efficiency and scanpath similarity, and can accurately predict the fixation duration.
AB - Aiming at the problem that the current scanpath prediction methods have insufficient representation of object association, we propose a scanpath prediction model based on semantic representation of the scene. Our model uses a panoramic segmentation network to separate object instances and backgrounds in scenes, and uses the attention mechanism to learn the semantic correlation between objects, which effectively extracts the deep image information related to the current task. We also propose a dual-branch structure predicting the fixation position and duration simultaneously, to fully simulate the temporal and spatial distribution of the human eye's attention in visual search. Experimental results show that our model has obvious advantages over the existing scanpath prediction methods in search efficiency and scanpath similarity, and can accurately predict the fixation duration.
KW - Human Scanpath
KW - Inverse Reinforcement Learning
KW - Semantic Representation
KW - Visual Search
UR - https://www.scopus.com/pages/publications/85146732002
U2 - 10.1109/ICIP46576.2022.9897207
DO - 10.1109/ICIP46576.2022.9897207
M3 - 会议稿件
AN - SCOPUS:85146732002
T3 - Proceedings - International Conference on Image Processing, ICIP
SP - 1976
EP - 1980
BT - 2022 IEEE International Conference on Image Processing, ICIP 2022 - Proceedings
PB - IEEE Computer Society
T2 - 29th IEEE International Conference on Image Processing, ICIP 2022
Y2 - 16 October 2022 through 19 October 2022
ER -