Effective Does Not Mean Useful: Conditional Functional Substitutability for Redundancy and Scaling in Transformers

Excerpt

arXiv:2609.39259v1 Announce Type: cross Abstract: Modern neural networks scale predictably, yet the mechanisms behind these regularities remain unclear. Neural redundancy is typically characterized by component importance or representational similarity, both indirect proxies. We view redundancy as an input-conditioned, dynamic relation: intermediate computational states are functionally redundant when they induce similar downstream responses. We introduce Conditional Functional Substitutability