Demystifying Entropy-based Selection for Chain-of-Thought Compression in Large Reasoning Models

Excerpt

arXiv:2607.28707v3 Announce Type: replace Abstract: Entropy-based pruning has been proposed as an effective method for compressing Chain-of-Thought (CoT) reasoning with negligible accuracy loss. We test the robustness of low- and high-entropy CoT step selection methods across various models and reasoning tasks, showing that entropy offers no advantage over random pruning in any evaluated setting. Moving from sentences to tokens, we then show that retaining low-entropy tokens seems effective only