How Learning Governs Unlearning across the Memorization-Generalization Spectrum

Excerpt

arXiv:2610.08577v1 Announce Type: new Abstract: While unlearning seeks to negate undesired capabilities acquired through learning, little research has examined how the way models learn shapes their subsequent unlearning. In this paper, we investigate this connection from the perspectives of memorization and generalization, the two most representative yet competing strategies that models employ during training. We first classify memorization- and generalization-heavy models using grokking in modu