Dimension-Free Rank Lifting from Random Hyperplane Arrangements

Excerpt

arXiv:2609.39855v1 Announce Type: new Abstract: We study the width required for a randomly initialized hidden layer of a neural network to achieve rank lifting. Namely, given a dataset $X \in \mathbb{R}^{m \times d}$ of $m$, $d$-dimensional input vectors separated by an angle of at least $\theta$, we consider the random feature matrix $\sigma(XR)$, where $R$ is standard Gaussian. For positively homogeneous nonpolynomial activations, which include sign, Heaviside, ReLU, and ReLU powers among othe