Mixture-Trained Merging for Unified Multi-Objective Models

Excerpt

arXiv:2610.01238v1 Announce Type: new Abstract: Unified language models are increasingly expected to combine heterogeneous capabilities, such as mathematics, code, instruction following, and controllable thinking behavior, within a single set of parameters. A common solution is sequential post-training on multiple objectives, but this entangles all objectives along one optimization trajectory and makes the final model highly sensitive to training order, data ratios, schedules, and stopping crite