ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

Excerpt

arXiv:2610.00431v1 Announce Type: cross Abstract: Continual parameter-efficient fine-tuning for large language models (LLMs) must balance retention of previously acquired knowledge, adaptation to new tasks, and strict parameter budgets. We present \textbf{ChainLoRA}, a replay-free continual merging framework built on chain-updated task-vector geometry. From a parameter-merging perspective, we formulate a geometric view of forgetting through a measurable interaction between task updates, separati