SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

Excerpt

arXiv:2609.29465v2 Announce Type: replace Abstract: Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks, however, usually start from a human-identified issue and evaluate whether a patch satisfies a functional signal. We present SWE-Prometheus, a benchmark for the broader task of improving repository engineering governance. Each task provides a fixed snapshot and an open-ended objective, requiring