Auditing Routing Entropy as an Uncertainty Signal in Attention-Residual Transformers

Excerpt

arXiv:2610.01495v1 Announce Type: new Abstract: Dynamic architectures leave a per-example routing trace beside each prediction, and diffuse routing is easy to read as a sign that the prediction is unreliable. We audit that reading for routing entropy in Attention-Residual (AR) variants of Swin-Tiny and DeiT-Small, trained from scratch on CIFAR-10/100 with a soft-binned calibration auxiliary loss, asking whether the trace carries information about correctness beyond what the model's own confidenc