The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization

Excerpt

arXiv:2609.39055v1 Announce Type: new Abstract: How can we personalize a shared expert library from a user's pairwise choices? Prior work can realize different reward trade-offs by merging reward-specialized experts, given a vector of trade-off weights. In practice, users can more naturally choose between outputs than specify numerical weights. The challenge is therefore to turn these choices into the coefficients required for merging, while accounting for ambiguity when feedback is limited. Our