From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignment

Excerpt

arXiv:2610.01864v1 Announce Type: cross Abstract: How can we understand what a music foundation model has learned \textit{internally}? Most interpretability approaches, such as probing and Sparse Autoencoders (SAEs), focus on identifying individual features with minimal structural assumptions. We argue that many concepts are better understood as \textit{structured relations} rather than isolated features. This is especially prominent in music, where tonal structures are organized in the space of