About this paper
CurvDelta and CurvLoRA constrain LLM updates to a low-curvature preservation space, improving retention while maintaining target adaptation across reasoning, code, and unlearning tasks.
A central challenge in adapting large language models (LLMs) is capability preservation: an adaptation can achieve its target while degrading general capabilities. We introduce CurvDelta, a scalable framework that incorporates the preservation constraint into the parameterization of the model update. By analyzing the local geometry of the preservation constraint around the pre-adaptation model, CurvDelta constructs a low-curvature space in which parameter updates leave the preservation loss nearly invariant. We make its construction practical for LLMs through Kronecker-factored approximate curvature (K-FAC) and a factorized projection. We also introduce CurvLoRA, a low-rank variant within…
large language modelscapability preservationCurvDeltapreservation constraintparameterizationlocal geometrylow-curvature spacepreservation lossKronecker-factored approximate curvatureK-FAC
- Upvotes
- 11
- Authors
- 6
- Published
- Nov 18, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
CurvDelta: Low-Curvature Delta Modeling For Surgical Learning and Unlearning was first seen on Hugging Face · Daily papers on Nov 19, 2026. Since then it has appeared in 13 checks on 2 different days. Its best position was #1 of 30, reached on Nov 20, 05:29 UTC. It dropped off the list after Nov 20, 09:29 UTC.