Towards Direct Evaluation of Harness Optimizers via Priority Ranking

Excerpt

arXiv:2605.22505v2 Announce Type: replace Abstract: Harness optimization enables automated agent creation by having an optimizer agent iteratively update the harness of target agents. Despite its success, current studies evaluate optimizers solely by observing target agents' performance gains. This indirect end-improvement evaluation neglects optimizers' actions at intermediate steps, which are often erroneous and hinder agent performance in deployment. Therefore, it is unclear whether harness o