Spec2Game: Can LLMs Generate Complete Playable Games from Detailed Specifications?

Excerpt

arXiv:2610.04253v1 Announce Type: new Abstract: Generating an executable program does not necessarily mean that it correctly implements the behavioral requirements specified in natural language. To evaluate large language models' ability to realize detailed specifications as complete interactive programs, we introduce Spec2Game, a benchmark that requires models to generate complete Pygame projects from detailed natural-language game specifications. Spec2Game comprises 15 game families and 150 ta