FedMIX-P: Mixing Local and Global Preconditioners for Federated Vision and Language Model Training

Excerpt

arXiv:2610.01515v1 Announce Type: new Abstract: Adaptive preconditioners accelerate model training, but heterogeneous client geometries can bias federated updates even when gradients are evaluated at the same model. Round-start synchronization alone cannot prevent this mismatch from reappearing during local training. We propose \texttt{FedMIX-P}, which mixes shared and local preconditioners at every local step, retaining local adaptation while reducing mean-squared operator mismatch by a factor