Complementary Roles of Activation and Parametric Memory in Few-Shot Learning

Excerpt

arXiv:2609.28250v1 Announce Type: new Abstract: At test time, large language models (LLMs) can encode historical information in activation memory (i.e., KV caches) and parametric memory (i.e., updated parameters). While activation memory is generally considered effective for factual recall and parametric memory for learning new tasks, their interplay remains unclear. In this work, we systematically investigate the role of memory in few-shot learning through controlled experiments. We find that a