Yanai Elazar
@yanai.bsky.social
Assistant Professor at Bar-Ilan University
https://yanaiela.github.io/
https://yanaiela.github.io/
I’m excited to share that I'm joining Bar-Ilan University as an assistant professor!
August 14, 2025 at 12:45 PM
I’m excited to share that I'm joining Bar-Ilan University as an assistant professor!
I'm in Singapore for ICLR to present this paper:
Tomorrow, April 26th, 10-12:30 in Hall 3+2B #236
Come check it out!
arxiv.org/abs/2504.12459
Tomorrow, April 26th, 10-12:30 in Hall 3+2B #236
Come check it out!
arxiv.org/abs/2504.12459
April 25, 2025 at 1:55 AM
I'm in Singapore for ICLR to present this paper:
Tomorrow, April 26th, 10-12:30 in Hall 3+2B #236
Come check it out!
arxiv.org/abs/2504.12459
Tomorrow, April 26th, 10-12:30 in Hall 3+2B #236
Come check it out!
arxiv.org/abs/2504.12459
I think this is super exciting as it shows a way to use interpretability metrics as a useful feature to explain, and reveal information about the training data, even without having access to it (see some examples in the attached table).
April 25, 2025 at 1:55 AM
I think this is super exciting as it shows a way to use interpretability metrics as a useful feature to explain, and reveal information about the training data, even without having access to it (see some examples in the attached table).
Finally, not only we are able to predict the linearity of a relation based on its frequency, but we can also predict such frequencies based on the linearity metric!
April 25, 2025 at 1:55 AM
Finally, not only we are able to predict the linearity of a relation based on its frequency, but we can also predict such frequencies based on the linearity metric!
An interesting side finding is that the quality of the model does not seem to be a confounding factor to the formation of linearity, as we observe the same trends across model checkpoints.
April 25, 2025 at 1:55 AM
An interesting side finding is that the quality of the model does not seem to be a confounding factor to the formation of linearity, as we observe the same trends across model checkpoints.
We set our goal to explain this phenomenon.
And our main finding, was that linearity, is mostly explained by the occurrence statistics of the relation's entities (subject and object) in the training data!
And our main finding, was that linearity, is mostly explained by the occurrence statistics of the relation's entities (subject and object) in the training data!
April 25, 2025 at 1:55 AM
We set our goal to explain this phenomenon.
And our main finding, was that linearity, is mostly explained by the occurrence statistics of the relation's entities (subject and object) in the training data!
And our main finding, was that linearity, is mostly explained by the occurrence statistics of the relation's entities (subject and object) in the training data!
But... not all the time.
Why is that?
Why is that?
April 25, 2025 at 1:55 AM
But... not all the time.
Why is that?
Why is that?
Linear representations that form in neural networks are interesting, and have been studied before.
For instance,
Hernandez et al. showed that factoids of different relations (e.g., born-in) are often encoded linearly in Transformer's hidden representation.
For instance,
Hernandez et al. showed that factoids of different relations (e.g., born-in) are often encoded linearly in Transformer's hidden representation.
April 25, 2025 at 1:55 AM
Linear representations that form in neural networks are interesting, and have been studied before.
For instance,
Hernandez et al. showed that factoids of different relations (e.g., born-in) are often encoded linearly in Transformer's hidden representation.
For instance,
Hernandez et al. showed that factoids of different relations (e.g., born-in) are often encoded linearly in Transformer's hidden representation.
💡 New ICLR paper! 💡
"On Linear Representations and Pretraining Data Frequency in Language Models":
We provide an explanation for when & why linear representations form in large (or small) language models.
Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social
"On Linear Representations and Pretraining Data Frequency in Language Models":
We provide an explanation for when & why linear representations form in large (or small) language models.
Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social
April 25, 2025 at 1:55 AM
💡 New ICLR paper! 💡
"On Linear Representations and Pretraining Data Frequency in Language Models":
We provide an explanation for when & why linear representations form in large (or small) language models.
Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social
"On Linear Representations and Pretraining Data Frequency in Language Models":
We provide an explanation for when & why linear representations form in large (or small) language models.
Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social
How many _final_ will we end up with?
April 18, 2025 at 2:24 AM
How many _final_ will we end up with?
I mean, if you ask about my desired position...
November 25, 2024 at 9:43 PM
I mean, if you ask about my desired position...
Dear EiC,
Get some sleep,
Yours truly,
Yanai
Get some sleep,
Yours truly,
Yanai
November 19, 2024 at 10:08 PM
Dear EiC,
Get some sleep,
Yours truly,
Yanai
Get some sleep,
Yours truly,
Yanai
Training LLMs still has a way to go in terms of the number of authors (blog.google/outreach-ini...)
November 19, 2024 at 10:05 PM
Training LLMs still has a way to go in terms of the number of authors (blog.google/outreach-ini...)
And the swag, most importantly
November 12, 2024 at 2:40 PM
And the swag, most importantly
I also have more WIMBD stickers!
Ping me if you want some
Ping me if you want some
November 11, 2024 at 2:30 PM
I also have more WIMBD stickers!
Ping me if you want some
Ping me if you want some
Bhargavi Paranjape & Hao Peng & Sarah Wiegreffe & Khyathi Raghavi & Vivek Srikumar & Sameer Singh & Noah A. Smith
Finding paper - ping us to hear more, or read the paper here - arxiv.org/abs/2311.09605
Finding paper - ping us to hear more, or read the paper here - arxiv.org/abs/2311.09605
November 11, 2024 at 2:29 PM
Bhargavi Paranjape & Hao Peng & Sarah Wiegreffe & Khyathi Raghavi & Vivek Srikumar & Sameer Singh & Noah A. Smith
Finding paper - ping us to hear more, or read the paper here - arxiv.org/abs/2311.09605
Finding paper - ping us to hear more, or read the paper here - arxiv.org/abs/2311.09605
Session 11 posters (10:30-12:00), Thursday, Nov 14th
Bar Iluz & Asaf Yehudai & Gabriel Stanovsky
arxiv.org/abs/2406.00787
Bar Iluz & Asaf Yehudai & Gabriel Stanovsky
arxiv.org/abs/2406.00787
November 11, 2024 at 2:29 PM
Session 11 posters (10:30-12:00), Thursday, Nov 14th
Bar Iluz & Asaf Yehudai & Gabriel Stanovsky
arxiv.org/abs/2406.00787
Bar Iluz & Asaf Yehudai & Gabriel Stanovsky
arxiv.org/abs/2406.00787
Session 2 posters (11-12:30), Tuesday, Nov 12th
Chantal Shaib & @jessyjli.bsky.social & @byron.bsky.social
arxiv.org/abs/2407.00211
Learn more here - x.com/ChantalShaib...
Chantal Shaib & @jessyjli.bsky.social & @byron.bsky.social
arxiv.org/abs/2407.00211
Learn more here - x.com/ChantalShaib...
November 11, 2024 at 2:29 PM
Session 2 posters (11-12:30), Tuesday, Nov 12th
Chantal Shaib & @jessyjli.bsky.social & @byron.bsky.social
arxiv.org/abs/2407.00211
Learn more here - x.com/ChantalShaib...
Chantal Shaib & @jessyjli.bsky.social & @byron.bsky.social
arxiv.org/abs/2407.00211
Learn more here - x.com/ChantalShaib...
Session 2 oral (some time between 11-12:30), Tuesday Nov 12th
Will Merrill & Noah A. Smith
arxiv.org/abs/2406.13069
Learn more here - x.com/lambdaviking...
Will Merrill & Noah A. Smith
arxiv.org/abs/2406.13069
Learn more here - x.com/lambdaviking...
November 11, 2024 at 2:29 PM
Session 2 oral (some time between 11-12:30), Tuesday Nov 12th
Will Merrill & Noah A. Smith
arxiv.org/abs/2406.13069
Learn more here - x.com/lambdaviking...
Will Merrill & Noah A. Smith
arxiv.org/abs/2406.13069
Learn more here - x.com/lambdaviking...
In Miami for #EMNLP2024
My collaborators and I are presenting a bunch of interesting papers!
DM me if you wanna hear more about them, chat about life, job market, or other topics.
My collaborators and I are presenting a bunch of interesting papers!
DM me if you wanna hear more about them, chat about life, job market, or other topics.
November 11, 2024 at 2:29 PM
In Miami for #EMNLP2024
My collaborators and I are presenting a bunch of interesting papers!
DM me if you wanna hear more about them, chat about life, job market, or other topics.
My collaborators and I are presenting a bunch of interesting papers!
DM me if you wanna hear more about them, chat about life, job market, or other topics.