A time-frequency map generation network embedded with spectral self-attention for motor imagery classification

End-to-end networks have achieved remarkable success in Motor Imagery (MI) classification by directly extracting features from raw Electroencephalogram (EEG) signals and performing classification. The self-attention mechanism has been successfully introduced to improve the performance of these networks further. However, the features extracted from existing end-to-end networks lack interpretable and comprehensive spectral information, making it difficult to develop spectral self-attention mechanisms. To overcome this challenge, we proposed a novel end-to-end network called Time-Frequency Map Generation Network (TFGNet), which utilizes the frequency domain information of raw EEG signals to generate interpretable time–frequency maps. TFGNet has shown comparable performance to State-of-the-Art (SotA) methods on MI classification tasks. Subsequently, the first spectral attention module for end-to-end networks for discriminative analysis of frequency components is presented. Extensive experiments have shown that using this module in the TFGNet increases classification accuracy by approximately 3% for classification tasks involving multiple objects. The visualization results have also illustrated the process of generating time–frequency maps and the mechanism of spectral self-attention.