The Truncation Blind Spot: How Decoding Strategies Systematically Exclude Human-Like Token Choices

Excerpt

arXiv:2603.18482v4 Announce Type: replace Abstract: Why does machine-generated text remain detectable? We investigate a mechanistic explanation at the decoding stage: standard strategies such as top-$k$ and nucleus sampling restrict generation to high-probability tokens, while human writers routinely choose contextually appropriate words from deeper in the model's probability distribution. Truncation makes a measurable share of these choices unreachable; we call this the \emph{truncation blind s