FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection

Excerpt

arXiv:2610.01620v1 Announce Type: new Abstract: Federated training of foundation models is constrained by client memory and communication costs. LoRA-based methods reduce these costs through low-rank adapters, but their fixed rank budget can limit adaptation. Gradient low-rank optimization offers greater flexibility, yet independently chosen client subspaces create a problem we term \emph{subspace fragmentation}: local projections interact with data heterogeneity to bias aggregated directions, w