PaperBench: Evaluating AI’s Ability to Replicate AI Research

Excerpt

We introduce PaperBench, a benchmark evaluating the ability of AI agents to replicate state-of-the-art AI research.