Abstract
Large language models (LLMs) have been showing growing potential in the field of intelligent operation and maintenance, due to their strong capabilities in understanding and generating knowledge across data in multiple modalities. However, in operation and maintenance, time-series signals are among the most critical monitoring data, and their unique formats and high dimensionality pose significant challenges for direct application of LLMs. To address this limitation, we propose a novel large multimodal model for fault diagnosis (LMM-FD), which is a key problem in operation and maintenance. The proposed large multimodal framework effectively aligns time-series vibration signals with textual fault diagnosis knowledge, enabling interpretable and generalized fault diagnosis. The framework includes signal preprocessing, cross-modal alignment through a knowledge graph and graph neural networks, and automated generation of textual diagnostic reports. Extensive experiments on machinery condition monitoring datasets demonstrate that LMM-FD consistently outperforms existing baselines by leveraging multimodal data and constructed triplet-based knowledge graph. The proposed model obtains fairly high accuracy on multiple fault diagnosis scenarios, while achieving strong zero-shot generalization capabilities to unseen compound faults. Furthermore, by bridging numerical sensor data with textual knowledge, LMM-FD provides interpretable fault descriptions, highlighting its potential for practical industrial applications.
| Original language | English |
|---|---|
| Article number | 101061 |
| Journal | Journal of Industrial Information Integration |
| Volume | 50 |
| DOIs | |
| State | Published - Mar 2026 |
Keywords
- Contrastive language–image pretraining
- Intelligent fault diagnosis
- Knowledge graph
- Large multimodal model
- Zero-shot generalization
Fingerprint
Dive into the research topics of 'Multimodal data-enabled large model for machine fault diagnosis towards intelligent operation and maintenance'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver