TY - GEN
T1 - Enhancing Interactive Gaze Behavior Recognition via Co-training with Temporal Gaze Segmentation
AU - Xu, Tianchen
AU - Liu, Weimin
AU - Jin, Xi
AU - Yang, Yang
AU - Li, Hui
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2027.
PY - 2027
Y1 - 2027
N2 - Dynamic gaze recognition based on video sequences faces multiple challenges, including complex factors such as multi-person gaze tracking, gaze pattern transitions, and gaze point variations. We note that this task aligns closely with temporal gaze segmentation, which aims to identify the start and end boundaries of gaze behavior in videos. To this end, we propose a dual-task collaborative training framework with a temporal consistency loss. Here, gaze recognition and temporal segmentation are simultaneously optimized while explicitly constraining the confidence of gaze patterns according to temporal boundaries. In addition, to encode multi-factor contextual relationships, we employ an adaptive scene and gaze-heatmap interaction module. Through a cross-attention mechanism, it dynamically captures semantic correlations between gazes and scenes. Furthermore, to validate the method’s effectiveness, we establish a new gaze behavior dataset containing basic dual-person interaction scenarios, annotated with gaze pattern categories and temporal segmentation labels. Experimental results demonstrate that our method significantly improves the performance of gaze behavior recognition and exhibits superior generalization capabilities for handling multiple gaze patterns compared to other methods.
AB - Dynamic gaze recognition based on video sequences faces multiple challenges, including complex factors such as multi-person gaze tracking, gaze pattern transitions, and gaze point variations. We note that this task aligns closely with temporal gaze segmentation, which aims to identify the start and end boundaries of gaze behavior in videos. To this end, we propose a dual-task collaborative training framework with a temporal consistency loss. Here, gaze recognition and temporal segmentation are simultaneously optimized while explicitly constraining the confidence of gaze patterns according to temporal boundaries. In addition, to encode multi-factor contextual relationships, we employ an adaptive scene and gaze-heatmap interaction module. Through a cross-attention mechanism, it dynamically captures semantic correlations between gazes and scenes. Furthermore, to validate the method’s effectiveness, we establish a new gaze behavior dataset containing basic dual-person interaction scenarios, annotated with gaze pattern categories and temporal segmentation labels. Experimental results demonstrate that our method significantly improves the performance of gaze behavior recognition and exhibits superior generalization capabilities for handling multiple gaze patterns compared to other methods.
KW - Adaptive interaction
KW - Dual-task
KW - Gaze behavior recognition
KW - Temporal gaze segmentation
UR - https://www.scopus.com/pages/publications/105047067890
U2 - 10.1007/978-3-032-31452-9_47
DO - 10.1007/978-3-032-31452-9_47
M3 - 会议稿件
AN - SCOPUS:105047067890
SN - 9783032314512
T3 - Lecture Notes in Computer Science
SP - 711
EP - 726
BT - Pattern Recognition - 28th International Conference, ICPR 2026, Proceedings
A2 - De Marsico, Maria
A2 - Ho, Tin Kam
A2 - Jurie, Frederic
A2 - Liu, Cheng-Lin
A2 - Lopresti, Daniel
A2 - Nyström, Ingela
A2 - Ogier, Jean-Marc
A2 - Ross, Arun
A2 - Wang, Liang
PB - Springer Science and Business Media Deutschland GmbH
T2 - 28th International Conference on Pattern Recognition, ICPR 2026
Y2 - 17 August 2026 through 22 August 2026
ER -