PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examples

Excerpt

arXiv:2403.06131v3 Announce Type: replace-cross Abstract: Instruction tuning aligns large language models (LLMs) with human intentions but requires diverse, high-quality data that are difficult to collect in privacy-sensitive domains. Federated instruction tuning (FedIT) enables collaborative training across data owners, yet existing methods typically assume sufficient local data. In realistic few-shot settings, limited samples can cause overfitting, degrade performance, and increase vulnerabili