ManifoldFlow: SPD-Relaxed Stiefel Layers with Learnable Singular Spectrum

Excerpt

arXiv:2607.04535v2 Announce Type: replace Abstract: Orthogonal and Stiefel layers give neural weights exact spectral control, but they also impose a strong modeling constraint: all represented singular values are fixed at one. Many settings that benefit from an orthonormal basis still need direction-dependent attenuation or amplification. We introduce ManifoldFlow, a minimal relaxation of a fixed-spectrum Stiefel layer that keeps the basis on the Stiefel manifold while learning a bounded positiv