Chronologic: Measuring Language Models' Ability to Represent the Past

Excerpt

arXiv:2609.23178v1 Announce Type: new Abstract: Language models are appealing tools for research on the past. But to trust the evidence a model provides, researchers need to know whether its responses fit the period represented. Validation is challenging, because this is not a task living people ordinarily perform, and because many questions have multiple correct answers. We use historical texts to develop a benchmark for a model's representation of English-language contexts 1831-1930, relying o