The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

Excerpt

arXiv:2608.14229v1 Announce Type: new Abstract: Popular facts are memorised more deeply during pretraining and resist removal longer than rare ones, yet existing LLM unlearning methods apply uniform gradient pressure regardless of training-data frequency. We propose the AdaPop (Adaptive Popularity) method, which combines local token confidence with a per-fact popularity-dependent exponent derived from an external proxy (e.g., Wikidata sitelinks, LLM-as-Judge), and automates the forget-retain bal