Can AI Agents Simulate A/B Test Outcomes? A Validation Framework for Agentic Experimentation

Excerpt

arXiv:2608.02345v2 Announce Type: replace-cross Abstract: A/B testing remains the standard for rolling out new features in the technology industry. Each experiment, however, consumes real traffic, engineering effort, and weeks of wall-clock time. Can AI agents---conditioned on behavioral profiles and contextual descriptions of the intervention---simulate outcomes accurately enough to vet candidate treatments before committing live traffic? We formalize this question as a \emph{Simulated Randomiz