GFD-OPD: Guidance-Folded On-Policy Distillation of Diffusion Models Across Scales

Excerpt

arXiv:2609.39692v1 Announce Type: new Abstract: On-policy distillation (OPD) has demonstrated two important capabilities in language models: compressing large teachers into smaller students and merging expert models into a single model. Existing diffusion OPD, however, mostly focus on the latter, with teachers and students sharing the same backbone and scale. We investigate large-to-small diffusion opd from large teachers to a small student and find that the standard recipe fails. To find the un