MCV-Intention: A Multimodalities and Cross-View Dataset for Human Assembly Intention Recognition

In recent years, Industry 5.0 has emphasized human-centric intelligent manufacturing, positioning human-robot collaboration as a cornerstone for achieving mass customization. Consequently, enabling robots to perceive human state has become critical for efficient and safe human-robot collaborative assembly. However, current vision-based methods for assembly intention recognition face challenges such as limited dataset modalities, difficulties in reflecting real-world assembly processes, and inconsistent annotation workflows. To address these issues, this paper introduces the MCV-Intention dataset—a multimodalities, cross-view dataset designed for assembly scene understanding. Collected from 15 subjects, each assembly sequence encompasses six modalities and two views, capturing data from operators both before and after training. In this paper, we first outline the dataset collection process, detailing the hardware and software systems as well as the assembly objects involved. Subsequently, we present a comprehensive annotation protocol for assembly intention recognition and analyze the dataset from the viewpoints of structure and distribution. Finally, we conducted a series of benchmark experiments using state-of-the-art algorithms to establish baselines for future researches.