Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces

Excerpt

arXiv:2610.00751v1 Announce Type: new Abstract: Recent theoretical work identified fundamental properties of representation geometry that shape inference ability of deep neural networks. These include signal-noise factorization (SNF), the ability to segregate signal from noise, and signal-signal factorization (SSF), the ability to segregate task-specific and task-irrelevant signals. Here, we built regularizers that reinforce these two properties during training. We compared networks trained with