摘要
In this paper, we study the generalization performance of global minima of empirical risk minimization (ERM) on over-parameterized deep ReLU nets. Using a novel deepening scheme for deep ReLU nets, we rigorously prove that there exist perfect global minima achieving optimal generalization error rates for numerous types of data under mild conditions. Since over-parameterization of deep ReLU nets is crucial to guarantee that the global minima of ERM can be realized by the widely used stochastic gradient descent (SGD) algorithm, our results present a potential way to fill the gap between optimization and generalization of deep learning.
| 源语言 | 英语 |
|---|---|
| 页(从-至) | 1978-1993 |
| 页数 | 16 |
| 期刊 | IEEE Transactions on Information Theory |
| 卷 | 71 |
| 期 | 3 |
| DOI | |
| 出版状态 | 已出版 - 2025 |
学术指纹
探究 'Generalization Performance of Empirical Risk Minimization on Over-Parameterized Deep ReLU Nets' 的科研主题。它们共同构成独一无二的学术指纹。引用此
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver