GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning

Excerpt

arXiv:2605.14841v2 Announce Type: replace Abstract: Low-rank adaptation (LoRA) has become a dominant paradigm for parameter-efficient fine-tuning (PEFT) of large-scale deep learning models. However, its bilinear parameterization induces a parameter-dependent geometry: the mapping from trainable parameters to weight updates is not generally distance-preserving. Related methods that project a low-dimensional vector into LoRA's parameter space, such as Uni-LoRA, improve parameter efficiency, but th