From Tables to Quantified Statements: Evaluating LLM Inference Generation through Executable Verification

Excerpt

arXiv:2609.23966v1 Announce Type: new Abstract: LLMs can generate fluent descriptions from tables, but their outputs may remain logically unsupported by the structured data. We introduce STAT-TO-TEXT, a controlled task in which LLMs generate quantified natural language inferences from statistical tables using quantified constructions such as all, some, no, and most. To evaluate these inferences, we use an LLM generated Python checker code which when executed verifies the corresponding truth cond