Federated Mixture-of-Experts Alignment on Mobile Edge Networks under Data Heterogeneity

Excerpt

arXiv:2603.21276v2 Announce Type: replace Abstract: The growing demand for on-device large language model (LLM) services on mobile edge devices has driven the adoption of Mixture-of-Experts (MoE) architectures, which scale model capacity with limited computation. Since fine-tuning MoE-based LLMs relies on privacy-sensitive local data, federated learning (FL) offers a natural paradigm for collaborative training without exposing raw data. However, integrating MoE-based LLM fine-tuning into FL face