Exploring Heterogeneous Model Merging Approach for Complex Knowledge Transfer

Excerpt

arXiv:2609.39369v1 Announce Type: cross Abstract: Specialized models encode task-oriented behavior, but transferring that behavior to a general language model usually requires training, distillation, or representation alignment. We study whether such ability can instead be transferred directly at the parameter level. We apply two existing training-free heterogeneous merging methods, previously shown to transfer knowledge between general language models, to specialist-to-general transfer, project