Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

Excerpt

arXiv:2606.18216v3 Announce Type: replace Abstract: Knowledge distillation transfers a teacher's competence to a small student but is brittle in the small-student regime: forcing the student to imitate logits from a much larger teacher concentrates it on the teacher's sharpest modes, hurting generalization on benchmark families beyond the training corpus. Reinforcement learning (RL) avoids logit imitation by training on the student's own rollouts. However, on questions where every rollout fails,