Do I Need the Cloud? Uncertainty-Aware Step-Level Handoff for Small Language Model Agents

Excerpt

arXiv:2610.07816v1 Announce Type: cross Abstract: Small language models (SLMs) are attractive as local agent controllers because they reduce remote inference, latency, and deployment footprint, yet structured tool errors can cause an agent step to fail. Existing routers typically select a model once per query. However, agents expose sequential decision points whose difficulty dynamically changes based on intermediate observations. We propose STEPGATE, an uncertainty-aware handoff framework that