Neuromorphic computing-enabled multimodal data fusion for intelligent machine fault diagnosis

The rapid development of data-driven methods in the past years has significantly improved the performance of machine fault diagnosis. Currently, the mainstream intelligent fault diagnosis approaches are generally based on limited modality data. In real industrial applications, the strict limitation of the data source compromises the flexibility of such methods. In recent years, intelligent modeling with multimodal data has attracted increasing attention in different fields. Multimodal data can also benefit machine fault diagnosis with more complete health condition information. However, the different modalities are usually inconsistent in data structure, which brings significant challenges. Furthermore, current data-driven methods require substantial computational resources for practical deployment, particularly when applied to multimodal data. To address the aforementioned issues, this paper proposes a neuromorphic computing-enabled multimodal data fusion method for intelligent machine fault diagnosis. Multimodal condition monitoring data including vibration, current, etc., are first converted into a unified spiking representation space. Subsequently, dedicated feature extraction modules of each modality are designed to enhance feature extraction efficiency. A generalized multimodal contrastive learning (GMCL) framework is proposed to accurately align data from different modalities. The fault diagnosis model is developed with the neuromorphic computing framework, which not only ensures high diagnostic reliability but also significantly reduces power consumption. Compared to mainstream methods, the proposed approach achieves at least 88% optimization in response latency. The experimental results on two multimodal machine condition monitoring datasets demonstrate the effectiveness of the proposed method, which provides a promising solution for deployment in real industrial fault diagnosis applications.