HeteFedEEG: Prototype-Transfer Heterogeneous Federated Learning for EEG-based Emotion Recognition

Electroencephalography (EEG)-based emotion recognition is a promising avenue in affective computing due to its high temporal resolution, non-invasiveness, and cost-effectiveness. However, practical deployment faces significant challenges, including data privacy constraints, environmental noise, and substantial inter-subject variability, which hinder model generalization when trained on isolated datasets. Federated Learning (FL) has emerged as a compelling solution by enabling collaborative training across institutions without sharing raw data. Nevertheless, existing Federated EEG-based Emotion recognition approaches often suffer from two critical forms of heterogeneity: (1) subject heterogeneity, where EEG signals vary significantly across individuals, and (2) model heterogeneity, where clients adopt diverse model architectures and training strategies. To address these challenges, we propose HeteFedEEG, a heterogeneous federated learning framework based on prototype transfer. HeteFedEEG leverages information bottleneck theory to decouple EEG features into emotion-relevant prototypes and individual-specific representations. Instead of sharing model parameters, clients transmit perturbed emotion prototypes, enabling effective cross-institutional collaboration while preserving data privacy. A central server aggregates these prototypes into a unified emotional feature space, facilitating communication across heterogeneous models. Furthermore, to enhance learning stability, we introduce prototype regularization and a lifelong learning loss to mitigate catastrophic forgetting during local updates. Experiments on two public datasets, SEED and DEAP, demonstrate that HeteFedEEG consistently outperforms state-of-the-art methods, achieving an average improvement of 2.7% in accuracy and 1.5% in F1 score. These results highlight the potential of prototype transfer for addressing heterogeneity in federated EEG-based emotion recognition while ensuring privacy and generalizability.