Reposted
15 types and algorithms of Knowledge Distillation (KD):
▪️ Response-based
▪️ Feature-based
▪️ Relation-based
▪️ Multi-teacher
▪️ Cross-modal
▪️ Graph-based
▪️ Attention-based
▪️ Data-free
▪️ Quantized
▪️ Speculative
...
Save the list and check out our article for more about KD ->
▪️ Response-based
▪️ Feature-based
▪️ Relation-based
▪️ Multi-teacher
▪️ Cross-modal
▪️ Graph-based
▪️ Attention-based
▪️ Data-free
▪️ Quantized
▪️ Speculative
...
Save the list and check out our article for more about KD ->
Everything You Need to Know about Knowledge Distillation
A Blog post by Ksenia Se on Hugging Face
huggingface.co
March 8, 2025 at 2:03 PM
15 types and algorithms of Knowledge Distillation (KD):
▪️ Response-based
▪️ Feature-based
▪️ Relation-based
▪️ Multi-teacher
▪️ Cross-modal
▪️ Graph-based
▪️ Attention-based
▪️ Data-free
▪️ Quantized
▪️ Speculative
...
Save the list and check out our article for more about KD ->
▪️ Response-based
▪️ Feature-based
▪️ Relation-based
▪️ Multi-teacher
▪️ Cross-modal
▪️ Graph-based
▪️ Attention-based
▪️ Data-free
▪️ Quantized
▪️ Speculative
...
Save the list and check out our article for more about KD ->