KF4FL: Kalman Filtering for Privacy-Preserving Federated Learning With Momentum

Federated learning (FL) empowers numerous industrial applications by analyzing tremendous data from distributed smart devices. To reduce communication costs and training delays, momentum-based optimizers are often adopted in FL for training acceleration. Meanwhile, considering frequent model exchange in FL, differential privacy (DP) is also utilized to provide a rigorous privacy guarantee. Therefore, it is promising to combine momentum-based FL with DP for achieving efficient and private FL. However, besides impacting the FL model utility, the added noise would also exacerbate the overshoot issue in momentum-based FL optimization. To this end, we propose KF4FL, a DP-enhanced and momentum-accelerated FL framework by exploring the application of Kalman filter (KF), which can achieve both high model utility and stable FL training without compromising the privacy guarantee. Specifically, KF4FL is a two-fold KF scheme consisting of a gradient noise filtering for reducing the impact of noise in model utility, and a parameter overshoot filtering for stability enhancement in momentum-based training. Besides, by adapting the KF variant of Kalman-consensus information filter, we further propose KF4FL+, an efficient extension of KF4FL in the decentralized FL setting. Extensive experiments demonstrate that KF4FL can reduce up to 5% accuracy loss and 20% convergence time than the state-of-the-art methods. KF4FL+ further improves consensus efficiency by 10% under sparse network topologies.