EnGRICH: Enhancing Generative Reward Modeling with Critiques from Humans

Excerpt

arXiv:2610.05370v2 Announce Type: new Abstract: Generative reward models (GRMs) are important for LLM optimization. Unlike scalar reward models, GRMs generate natural-language critiques alongside preference judgments, providing finer-grained evaluation signals. Their effectiveness depends heavily on critique reliability. However, existing GRM training typically uses final preference correctness as outcome supervision. Because the preference outcome space is highly constrained, unreliable critiqu