Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities

Excerpt

arXiv:2511.02817v2 Announce Type: replace-cross Abstract: As model context lengths continue to grow, concerns about whether models effectively use the full context length have persisted. While several carefully designed long-context evaluations have recently been released, these evaluations tend to rely on retrieval from one or more sections of the context, which allows nearly all of the context tokens to be disregarded as noise. This represents only one type of task that might be performed with