CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents

Excerpt

arXiv:2607.05378v2 Announce Type: replace Abstract: Long-horizon agentic LLMs are increasingly limited by finite context windows, as extended interaction trajectories can exceed the maximum context length before a task is completed. Context compaction offers a natural solution by summarizing previous interaction states and continuing the rollout under a compressed context, but incorporating compaction into reinforcement learning remains underexplored. We propose CompactionRL, a reinforcement lea