efficient fine tune storage

Excerpt

Imagine Git for model fine-tunes that also saves you storage. DeltaTensors compresses fine-tuned model checkpoints by storing the weight difference from the base model instead of another full copy of the model. If you have a bunch of fine-tunes or checkpoints from the same base model, you keep the base once and store a much smaller .wdelta for each version. On a Qwen2.5-0.5B fine-tune, a 953 MB model compressed to a 294 MB delta. Reconstructing it changed perplexity from 19.11 to 19.22. It works