What Is Model Distillation? How Smaller AI Models Learn
AI-generated summary Model distillation transfers behavior from a larger teacher model to a smaller student model so teams can lower inference cost and latency. The 2015 paper by Geoffrey Hinton, Oriol Vinyals, and Jeff Dean introduced the modern formulation of knowledge distillation.