Efficient temporal flow Transformer accompanied with multi-head probsparse self-attention mechanism for remaining useful life prognostics

Predictive maintenance, such as remaining useful life (RUL) prognostics, requires precise long time-series forecasting, which demands a higher predictive capability of data-driven models. Nevertheless, the typical convolution and recurrent frameworks are still inadequate in the feature extraction and temporal complexity analysis, which makes them difficult to efficiently capture the precise long-term dependency coupling. Recent research has demonstrated the potential of Transformer-based framework to improve the prediction capability by the massive success in sequence processing. Inspired by the above, this paper proposes an efficient end-to-end Temporal Flow Transformer (TFT) for RUL prognostics of rolling bearings. Its main framework is composed of multi-layer encoders, which can directly extract effective degradation features from the time-frequency representations of raw signals, with two distinctive characteristics: (1) Specially designed multi-head probsparse self-attention mechanism can effectively highlight the dominant attention, which makes the TFT have considerable performance in reducing the computational complexity of extremely long time-series; (2) The TFT trained by knowledge-induced distillation strategy can significantly improve its domain adaptability, making it possible to achieve accurate RUL prediction under cross-operating conditions. Extensive experiments on two life-cycle bearing datasets indicate that the TFT greatly outperforms the existing state-of-the-art methods and provides a new solution for RUL prognostics.