JEV versus LLMs: Accuracy, Cost and Calibration on Seven Political Science Replications

Excerpt

arXiv:2610.06625v2 Announce Type: replace Abstract: Large language models (LLMs) annotate and scale political text or constructs by generating text tokens. A new class of models, which TypeSafe markets as "System One" models, instead returns decisions and probability distributions across a user-supplied fixed answer set. A commercial model, JEV, is advertised as having a dramatic cost and speed advantage over traditional LLMs along with better calibrated decisions. As such, it might be useful fo