DGA-Muon: Decoupled Geometry-Aligned Adaptive Scaling for Muon

Excerpt

arXiv:2610.06578v1 Announce Type: new Abstract: While NorMuon has achieved strong empirical performance in large-scale pretraining by enhancing Muon with row-wise adaptive scaling, its underlying adaptive mechanism remains poorly understood. In this work, we provide the first systematic analysis of NorMuon's adaptivity, revealing that it originates primarily from orthogonalization-induced geometry rather than genuine optimization-relevant information. Under exact orthogonalization, the adaptive