Anej Svete
@anejsvete.bsky.social
PhD student in NLP at ETH Zurich.
anejsvete.github.io
anejsvete.github.io
Reposted by Anej Svete
Andy Yang, Christopher Watson, Anton Xue, Satwik Bhattamishra, Jose Llarena, William Merrill, Emile Dos Santos Ferreira, Anej Svete, David Chiang: The Transformer Cookbook https://arxiv.org/abs/2510.00368 https://arxiv.org/pdf/2510.00368 https://arxiv.org/html/2510.00368
October 2, 2025 at 6:33 AM
Andy Yang, Christopher Watson, Anton Xue, Satwik Bhattamishra, Jose Llarena, William Merrill, Emile Dos Santos Ferreira, Anej Svete, David Chiang: The Transformer Cookbook https://arxiv.org/abs/2510.00368 https://arxiv.org/pdf/2510.00368 https://arxiv.org/html/2510.00368
Reposted by Anej Svete
We present The Transformer Cookbook: a collection of recipes for programming algorithms directly into transformers!
Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!
Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!
The Transformer Cookbook
We present the transformer cookbook: a collection of techniques for directly encoding algorithms into a transformer's parameters. This work addresses the steep learning curve of such endeavors, a prob...
arxiv.org
October 3, 2025 at 4:24 PM
We present The Transformer Cookbook: a collection of recipes for programming algorithms directly into transformers!
Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!
Hungry for an induction head? Craving a Dyck language recognizer? We show you step-by-step how to cook up transformers for these algorithms and many more!
Reposted by Anej Svete
Introducing Asta—our bold initiative to accelerate science with trustworthy, capable agents, benchmarks, & developer resources that bring clarity to the landscape of scientific AI + agents. 🧵
August 26, 2025 at 1:05 PM
Introducing Asta—our bold initiative to accelerate science with trustworthy, capable agents, benchmarks, & developer resources that bring clarity to the landscape of scientific AI + agents. 🧵
Reposted by Anej Svete
As part of Asta, our initiative to accelerate science with trustworthy AI agents, we built AstaBench—the first comprehensive benchmark to compare them. ⚖️
August 26, 2025 at 3:02 PM
As part of Asta, our initiative to accelerate science with trustworthy AI agents, we built AstaBench—the first comprehensive benchmark to compare them. ⚖️
Reposted by Anej Svete
Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚
July 28, 2025 at 5:00 PM
Ai2 is excited to be at #ACL2025 in Vienna, Austria this week. Come say hello, meet the team, and chat about the future of NLP. See you there! 🤝📚
Reposted by Anej Svete
We are #1 on the @huggingface heatmap - this is what true openness looks like!🥇🎉
750+ models
230+ datasets
And counting...
Come build with us
huggingface.co/spaces/cfahl...
750+ models
230+ datasets
And counting...
Come build with us
huggingface.co/spaces/cfahl...
Model Release Heatmap - a Hugging Face Space by cfahlgren1
Search this app to see model release activity for any Hugging Face organization or user over time. Just enter the org name to view their heatmap.
huggingface.co
June 12, 2025 at 6:16 PM
We are #1 on the @huggingface heatmap - this is what true openness looks like!🥇🎉
750+ models
230+ datasets
And counting...
Come build with us
huggingface.co/spaces/cfahl...
750+ models
230+ datasets
And counting...
Come build with us
huggingface.co/spaces/cfahl...
🧵 Excited to share our paper "Unique Hard Attention: A Tale of Two Sides" with Selim, Jiaoda, and Ryan, where we show that the way transformers break ties in attention scores has profound implications on their expressivity! And it got accepted to ACL! :)
The paper: arxiv.org/abs/2503.14615
The paper: arxiv.org/abs/2503.14615
Unique Hard Attention: A Tale of Two Sides
Understanding the expressive power of transformers has recently attracted attention, as it offers insights into their abilities and limitations. Many studies analyze unique hard attention transformers...
arxiv.org
May 17, 2025 at 2:28 PM
🧵 Excited to share our paper "Unique Hard Attention: A Tale of Two Sides" with Selim, Jiaoda, and Ryan, where we show that the way transformers break ties in attention scores has profound implications on their expressivity! And it got accepted to ACL! :)
The paper: arxiv.org/abs/2503.14615
The paper: arxiv.org/abs/2503.14615
Reposted by Anej Svete
Current KL estimation practices in RLHF can generate high variance and even negative values! We propose a provably better estimator that only takes a few lines of code to implement.🧵👇
w/ @xtimv.bsky.social and Ryan Cotterell
code: arxiv.org/pdf/2504.10637
paper: github.com/rycolab/kl-rb
w/ @xtimv.bsky.social and Ryan Cotterell
code: arxiv.org/pdf/2504.10637
paper: github.com/rycolab/kl-rb
May 6, 2025 at 2:59 PM
Current KL estimation practices in RLHF can generate high variance and even negative values! We propose a provably better estimator that only takes a few lines of code to implement.🧵👇
w/ @xtimv.bsky.social and Ryan Cotterell
code: arxiv.org/pdf/2504.10637
paper: github.com/rycolab/kl-rb
w/ @xtimv.bsky.social and Ryan Cotterell
code: arxiv.org/pdf/2504.10637
paper: github.com/rycolab/kl-rb
Reposted by Anej Svete
I will be at #NeurIPS2024 in Vancouver. I am excited to meet people working on AI Safety and Security. Drop a DM if you want to meet.
I will be presenting two (spotlight!) works. Come say hi to our posters.
I will be presenting two (spotlight!) works. Come say hi to our posters.
December 9, 2024 at 5:02 PM
I will be at #NeurIPS2024 in Vancouver. I am excited to meet people working on AI Safety and Security. Drop a DM if you want to meet.
I will be presenting two (spotlight!) works. Come say hi to our posters.
I will be presenting two (spotlight!) works. Come say hi to our posters.
Reposted by Anej Svete
No joke, FLaNN is one of the most interesting servers around. Check out the website for talk information!
flann.super.site
flann.super.site
November 26, 2024 at 4:04 PM
No joke, FLaNN is one of the most interesting servers around. Check out the website for talk information!
flann.super.site
flann.super.site
Reposted by Anej Svete
Happy to share our work "Counterfactual Generation from Language Models" with @AnejSvete, @vesteinns, and Ryan Cotterell! We tackle generating true counterfactual strings from LMs after interventions and introduce a simple algorithm for it. (1/7) arxiv.org/pdf/2411.07180
November 12, 2024 at 4:00 PM
Happy to share our work "Counterfactual Generation from Language Models" with @AnejSvete, @vesteinns, and Ryan Cotterell! We tackle generating true counterfactual strings from LMs after interventions and introduce a simple algorithm for it. (1/7) arxiv.org/pdf/2411.07180