N-gram-like Language Models Predict Naturalistic Reading Time Best

Excerpt

arXiv:2603.09872v2 Announce Type: replace Abstract: Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting naturalistic reading time. In this paper, we propose that this can be explained by reading time being shaped by simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neur