| title | Neural Networks Provably Learn Spectral Representations for Group Composition |
|---|---|
| authors | Jianliang He, Leda Wang, Fengzhuo Zhang, Siyu Chen, Zhuoran Yang |
| date | 2026-06-02 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.02993 |
| upvotes | 6 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict g_1 star g_2 for elements of a finite group G. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation-theoretic energy functional. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross-layer Fourier coefficients achieve a rotational rank-one alignment. This framework provides a representation-theoretic account of feature learning and characterizes a novel low-rank compression phenomenon for matrix-valued group representations. Moreover, for Abelian groups, we provide a complete population-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar-uniform phases, jointly approximating the indicator via a majority-vote mechanism. We further prove that both phase alignment and representation competition emerge with exponential convergence rates. |
cite
@misc{he2026neural,
title={Neural Networks Provably Learn Spectral Representations for Group Composition},
author={Jianliang He and Leda Wang and Fengzhuo Zhang and Siyu Chen and Zhuoran Yang},
year={2026},
eprint={2606.02993},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.02993}
}