Local Superior Soups: A Catalyst for Model Merging in Cross-Silo Federated Learning

Excerpt

arXiv:2410.23660v2 Announce Type: replace Abstract: Federated learning (FL) is a learning paradigm that enables collaborative training of models using decentralized data. Recently, the utilization of pre-trained weight initialization in FL has been demonstrated to effectively improve model performance. However, the evolving complexity of current pre-trained models, characterized by a substantial increase in parameters, markedly intensifies the challenges associated with communication rounds requ