PhoneCLI: From App Interfaces to Callable Commands for Mobile Agents

Excerpt

arXiv:2609.35671v2 Announce Type: replace Abstract: Mobile GUI agents operate through a perception--action loop: at each step they screenshot the device, invoke a vision--language model (VLM), and emit an action. It is slow, costly, and brittle, yet most of what it does is navigation---and everyday navigation is static, ordered, and endlessly repeated. We present PhoneCLI, which compiles an app's GUI navigation into callable commands, without any app-internal API, runtime instrumentation, or mod