Donatella Genovese
@donatellag.bsky.social
PhD Student | Works on Explainable AI | https://donatellagenovese.github.io/
Really cool paper by @kayoyin.bsky.social about interpretability of In Context Learning, they found that Function Vectors (FV) heads are crucial for few-shot ICL.
www.arxiv.org/abs/2502.14010
www.arxiv.org/abs/2502.14010
February 28, 2025 at 4:52 PM
Really cool paper by @kayoyin.bsky.social about interpretability of In Context Learning, they found that Function Vectors (FV) heads are crucial for few-shot ICL.
www.arxiv.org/abs/2502.14010
www.arxiv.org/abs/2502.14010
🚀 Meta’s new LLM pretraining framework predicts concepts and integrates them into its hidden state to enhance next-token prediction. 🚀
It achieves the same performance with 21.5% fewer tokens and better generalization! 🎯
📝: arxiv.org/abs/2502.08524
It achieves the same performance with 21.5% fewer tokens and better generalization! 🎯
📝: arxiv.org/abs/2502.08524
February 14, 2025 at 1:07 PM
🚀 Meta’s new LLM pretraining framework predicts concepts and integrates them into its hidden state to enhance next-token prediction. 🚀
It achieves the same performance with 21.5% fewer tokens and better generalization! 🎯
📝: arxiv.org/abs/2502.08524
It achieves the same performance with 21.5% fewer tokens and better generalization! 🎯
📝: arxiv.org/abs/2502.08524