🔗 Read the full paper here: arxiv.org/pdf/2410.13155

Thanks to my collaborators Xianyang Zhan, Yilun Chen,
@ceshwar.bsky.social, and @kous2v.bsky.social for their help!

#NAACL2025

[7/7]

⚖ Practical implications:

- SLMs are lighter and cheaper to deploy for real-time moderation
- They can be adapted to community-specific norms
- They’re more effective at triaging potentially harmful content
- The open-source nature provides more control and transparency

[6/7]

🌐 Cross-community transfer learning also shows promise! Fine-tuned SLMs can effectively moderate content in communities they weren’t explicitly trained on.

This has major implications for new communities and cross-platform moderation techniques.

[5/7]

📊 Our error analysis reveals:

- SLMs excel at detecting rule violations in short comments
- LLMs tend to be more conservative about flagging content
- LLMs perform better with longer comments where context helps determine appropriateness

[4/7]

💡 Key insight: SLMs adopt a more aggressive moderation approach, leading to higher recall but slightly lower precision compared to LLMs.

This trade-off is actually beneficial for platforms where catching harmful content is prioritized over occasional false positives.

[3/7]

🔍 We evaluated fine-tuned SLMs against LLMs across 15 popular Reddit communities. Results show SLMs surpass zero-shot LLMs with:

- 11.5% higher accuracy
- 25.7% higher recall
- Better performance on realistic imbalanced datasets

Even ICL didn’t help LLMs catch up!

[2/7]