GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution

Excerpt

arXiv:2610.00948v1 Announce Type: new Abstract: The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termination are controlled. Compared with harness optimization for non-GUI agents, automatically optimizing this harness poses three coupled challenges: reconciling model intent with observed visual effects, diagnosing failures under variable execution outcomes, and identifying recurrent failure patterns ac