TY - JOUR
T1 - DM-DPL
T2 - Toward Discrete Matrixing Differentially Private Learning
AU - Zhou, Jinhao
AU - Su, Zhou
AU - Wang, Yuntao
AU - Wu, Jun
N1 - Publisher Copyright:
© 2005-2012 IEEE.
PY - 2025
Y1 - 2025
N2 - Differential private learning is widely used in machine learning (ML) to protect continuous and scalar-valued data. The demand for discrete and matrix-valued computations is increasing, particularly in quantized neural networks and graph learning, which require discrete-valued parameters and large-scale matrix operations for efficient data processing. However, privacy protection for discrete and matrix-valued data is less explored. Traditional differential private mechanisms fail to maintain the discrete nature of data after perturbation and often overlook data correlations, struggling to balance privacy and utility. In this paper, we propose a Discrete Matrixing Differentially Private Learning (DM-DPL) framework, which protects the privacy of discrete and matrix-valued data during ML training by adding discrete matrix-variate Gaussian noise. First, we propose a novel Discrete Matrix-Variate Gaussian (DMVG) mechanism with rigorous conditions necessary to guarantee (∈, δ) -differential privacy. Additionally, we present an eigenvalue-weighted analysis-based precision budget allocation strategy, designed to maintain the utility of significant dimensions while providing consistent privacy guarantees. Finally, the results illustrate that our approach significantly surpasses existing state-of-the-art methods when applied to quantized federated learning. To the best of our knowledge, this is the first work to specifically protect discrete and matrix-valued data during ML training.
AB - Differential private learning is widely used in machine learning (ML) to protect continuous and scalar-valued data. The demand for discrete and matrix-valued computations is increasing, particularly in quantized neural networks and graph learning, which require discrete-valued parameters and large-scale matrix operations for efficient data processing. However, privacy protection for discrete and matrix-valued data is less explored. Traditional differential private mechanisms fail to maintain the discrete nature of data after perturbation and often overlook data correlations, struggling to balance privacy and utility. In this paper, we propose a Discrete Matrixing Differentially Private Learning (DM-DPL) framework, which protects the privacy of discrete and matrix-valued data during ML training by adding discrete matrix-variate Gaussian noise. First, we propose a novel Discrete Matrix-Variate Gaussian (DMVG) mechanism with rigorous conditions necessary to guarantee (∈, δ) -differential privacy. Additionally, we present an eigenvalue-weighted analysis-based precision budget allocation strategy, designed to maintain the utility of significant dimensions while providing consistent privacy guarantees. Finally, the results illustrate that our approach significantly surpasses existing state-of-the-art methods when applied to quantized federated learning. To the best of our knowledge, this is the first work to specifically protect discrete and matrix-valued data during ML training.
KW - Machine learning
KW - differential privacy
KW - discrete Gaussian
KW - matrix-variate Gaussian
UR - https://www.scopus.com/pages/publications/105008807054
U2 - 10.1109/TIFS.2025.3580231
DO - 10.1109/TIFS.2025.3580231
M3 - 文章
AN - SCOPUS:105008807054
SN - 1556-6013
VL - 20
SP - 6149
EP - 6161
JO - IEEE Transactions on Information Forensics and Security
JF - IEEE Transactions on Information Forensics and Security
ER -