Distillation

Training a smaller 'student' model to imitate a larger 'teacher' model, keeping much of its ability at a fraction of the cost. Many fast, cheap models are partly distilled from frontier ones. Distillation is one reason capability that once required the biggest model reaches budget tiers within months.