VibeMemBench: Evaluating Memory Systems for Coding Agents on Real Repository Coding Tasks

Excerpt

arXiv:2609.23570v1 Announce Type: cross Abstract: Coding agents operate on real repository coding tasks, and persistent memory systems promise to reuse experience across tasks. Yet existing evaluations do not show whether those systems improve executable repository work. Repository benchmarks test code changes but do not isolate memory, while memory benchmarks score recall without measuring downstream coding outcomes. We introduce VibeMemBench, a benchmark for evaluating memory systems on 111 co