What Is Model Distillation? How Large AI Models Teach Smaller Ones
AI-generated summary Model distillation transfers behavior from a larger or more capable teacher into a smaller student model using teacher outputs, internal signals, or softened probability targets. This process differs from simple compression because it requires teaching the smaller model rather than simply removing parameters.