Jelle Zuidema 🟥
@wzuidema.bsky.social
Associate Professor of Natural Language Processing & Explainable AI, University of Amsterdam, ILLC
Dit Klavertje (dat mijn zoekmachine toont in een zoektocht naar iets heel anders), leest als een halve roman in één zin. Ik ben nu wel benieuwd naar de tweede helft, maar de link is dood (net als de ouders).
November 2, 2025 at 9:24 PM
Dit Klavertje (dat mijn zoekmachine toont in een zoektocht naar iets heel anders), leest als een halve roman in één zin. Ik ben nu wel benieuwd naar de tweede helft, maar de link is dood (net als de ouders).
Human individual judgements correlate even more strongly with the difference between a model's scores, but that says nothing about a model's abilities *in the wild*! This is contra Hu et al. '24 (www.pnas.org/doi/10.1073/...), & most importantly, provides a fresh dataset for use in this debate.
July 29, 2025 at 11:45 AM
Human individual judgements correlate even more strongly with the difference between a model's scores, but that says nothing about a model's abilities *in the wild*! This is contra Hu et al. '24 (www.pnas.org/doi/10.1073/...), & most importantly, provides a fresh dataset for use in this debate.
My favourite image from the paper, illustrating that LLMs are surprisingly weak at judging grammaticality. Human judgment correlates quite strongly with the *difference* in likelihood (or SLOR) that LLMs assign to pairs of grammatical & ungrammatical sentences, but that's the wrong measure.
July 29, 2025 at 11:19 AM
My favourite image from the paper, illustrating that LLMs are surprisingly weak at judging grammaticality. Human judgment correlates quite strongly with the *difference* in likelihood (or SLOR) that LLMs assign to pairs of grammatical & ungrammatical sentences, but that's the wrong measure.
Het stuk is van the Editorial Board, dus zonder specifieke auteursnamen.
En heeft m.i. schokkend beperkt beeld over de internationale conflicten van de afgelopen decennia.
En heeft m.i. schokkend beperkt beeld over de internationale conflicten van de afgelopen decennia.
June 15, 2025 at 8:48 AM
Het stuk is van the Editorial Board, dus zonder specifieke auteursnamen.
En heeft m.i. schokkend beperkt beeld over de internationale conflicten van de afgelopen decennia.
En heeft m.i. schokkend beperkt beeld over de internationale conflicten van de afgelopen decennia.
TIL that professor Kunihiko Fukushima, inventor of the neocognitron, is still an active researcher in his late 90s. The neocognitron is a convolutional neutral network (CNN); CNNs is turn were the model family with which deep learning revolution in Artificial Intelligence started. A living legend!
May 8, 2025 at 12:33 PM
TIL that professor Kunihiko Fukushima, inventor of the neocognitron, is still an active researcher in his late 90s. The neocognitron is a convolutional neutral network (CNN); CNNs is turn were the model family with which deep learning revolution in Artificial Intelligence started. A living legend!
Encouraging to see 5000+ protesters at our local Amsterdam protests against the budget cuts in higher education in the NL! In this time of geopolitical tensions, changing climate, democratic backsliding, & tech revolutions, governments should *invest* in knowledge & the future of the next generation
March 17, 2025 at 2:36 PM
Encouraging to see 5000+ protesters at our local Amsterdam protests against the budget cuts in higher education in the NL! In this time of geopolitical tensions, changing climate, democratic backsliding, & tech revolutions, governments should *invest* in knowledge & the future of the next generation
Bank robbery progressing as planned.
February 26, 2025 at 3:43 PM
Bank robbery progressing as planned.
Ha, ja, zo ken ik de uitdrukking ook (en dat klopt met wat Neerlandistiek.nl* erover schrijft).
Maar het voorbeeld illustreert vooral hoe DeepSeek redeneert (ipv bekende gegens oplepelt), en daar ligt de innovatie.
* neerlandistiek.nl/2023/04/je-w...
Maar het voorbeeld illustreert vooral hoe DeepSeek redeneert (ipv bekende gegens oplepelt), en daar ligt de innovatie.
* neerlandistiek.nl/2023/04/je-w...
January 28, 2025 at 10:27 PM
Ha, ja, zo ken ik de uitdrukking ook (en dat klopt met wat Neerlandistiek.nl* erover schrijft).
Maar het voorbeeld illustreert vooral hoe DeepSeek redeneert (ipv bekende gegens oplepelt), en daar ligt de innovatie.
* neerlandistiek.nl/2023/04/je-w...
Maar het voorbeeld illustreert vooral hoe DeepSeek redeneert (ipv bekende gegens oplepelt), en daar ligt de innovatie.
* neerlandistiek.nl/2023/04/je-w...
Today is Martin Luther King day in the US.
Makes me think back of a shop owner in Queens, who cheerfully sold me this calendar of 1998 (useful again in 2026!), the first time I visited NYC. Incredible to realise that the civil rights movement was as long ago then, as the nineties are for us now.
Makes me think back of a shop owner in Queens, who cheerfully sold me this calendar of 1998 (useful again in 2026!), the first time I visited NYC. Incredible to realise that the civil rights movement was as long ago then, as the nineties are for us now.
January 20, 2025 at 9:33 PM
Today is Martin Luther King day in the US.
Makes me think back of a shop owner in Queens, who cheerfully sold me this calendar of 1998 (useful again in 2026!), the first time I visited NYC. Incredible to realise that the civil rights movement was as long ago then, as the nineties are for us now.
Makes me think back of a shop owner in Queens, who cheerfully sold me this calendar of 1998 (useful again in 2026!), the first time I visited NYC. Incredible to realise that the civil rights movement was as long ago then, as the nineties are for us now.
Our video series is based on the tutorial "Transformer-specific Interpretability" that we presented at EACL last Spring. Code, slides and references linked here:
projects.illc.uva.nl/indeep/tutor...
5/n
projects.illc.uva.nl/indeep/tutor...
5/n
December 30, 2024 at 11:16 AM
Our video series is based on the tutorial "Transformer-specific Interpretability" that we presented at EACL last Spring. Code, slides and references linked here:
projects.illc.uva.nl/indeep/tutor...
5/n
projects.illc.uva.nl/indeep/tutor...
5/n
The good news is that people here are finally getting organized to protest, and that the 20,000+ people that showed up in The Hague last week surprised politicians. The spoiled, rich, fattened-up Dutch are learning to protest again :). Might be useful experience in the coming years.
December 4, 2024 at 8:19 AM
The good news is that people here are finally getting organized to protest, and that the 20,000+ people that showed up in The Hague last week surprised politicians. The spoiled, rich, fattened-up Dutch are learning to protest again :). Might be useful experience in the coming years.
Vegetable from another planet
November 10, 2024 at 7:50 PM
Vegetable from another planet
These are all relevant concepts for a mechanistic understanding of neural nets too, while "superposition" already has a technical definition (requiring linearity) that explicitly does not apply in modern NNs. And quantum superposition & observer effects also don't apply (see Dirac's explanation).
October 27, 2024 at 1:24 PM
These are all relevant concepts for a mechanistic understanding of neural nets too, while "superposition" already has a technical definition (requiring linearity) that explicitly does not apply in modern NNs. And quantum superposition & observer effects also don't apply (see Dirac's explanation).
I don't know what the perfect ref would be (and "the most relevant information based on context" might already be very rich). The latest blogpost from Anthropic's interpretability team has some interesting examples of differences due to fine-tuning.
transformer-circuits.pub/2024/crossco...
transformer-circuits.pub/2024/crossco...
October 27, 2024 at 12:23 PM
I don't know what the perfect ref would be (and "the most relevant information based on context" might already be very rich). The latest blogpost from Anthropic's interpretability team has some interesting examples of differences due to fine-tuning.
transformer-circuits.pub/2024/crossco...
transformer-circuits.pub/2024/crossco...
Thx! We use tSNE frequently to develop intuitions. For instance, this new paper proposes a new method to disentangle internal states of wav2vec2, into a text-like and an acoustics-only representation. tSNE nicely illustrates this works, even if it's by itself not firm proof
arxiv.org/abs/2410.03037
arxiv.org/abs/2410.03037
October 16, 2024 at 8:40 PM
Thx! We use tSNE frequently to develop intuitions. For instance, this new paper proposes a new method to disentangle internal states of wav2vec2, into a text-like and an acoustics-only representation. tSNE nicely illustrates this works, even if it's by itself not firm proof
arxiv.org/abs/2410.03037
arxiv.org/abs/2410.03037
Is er al een template voor een protest-reactie op verzoeken van het ministerie van Onderwijs? Ik krijg, als examinator, het verzoek een vragenlijst in te vullen. Ik ben geneigd te antwoorden dat ik daar geen tijd voor heb, i.e.g. tot de bezuinigingen van tafel zijn. Doen anderen dat ook?
September 5, 2024 at 9:58 AM
Is er al een template voor een protest-reactie op verzoeken van het ministerie van Onderwijs? Ik krijg, als examinator, het verzoek een vragenlijst in te vullen. Ik ben geneigd te antwoorden dat ik daar geen tijd voor heb, i.e.g. tot de bezuinigingen van tafel zijn. Doen anderen dat ook?
On why OpenAI’s head of alignment resigned
May 18, 2024 at 6:57 AM
On why OpenAI’s head of alignment resigned
the LinkedIn algorithms are not getting any better yet...
April 29, 2024 at 12:49 PM
the LinkedIn algorithms are not getting any better yet...
Is this spam or a serious attempt at academic bribery?
April 17, 2024 at 7:37 AM
Is this spam or a serious attempt at academic bribery?
Today's new job recommendations from (Microsoft-owned)
LinkedIn... Yes, I like Sicilian delicacies and coffee, yes I have called call centers and I've watched "The devil wears Prada", but I think I'll continue doing AI research at my local university for a bit longer. #AGI
LinkedIn... Yes, I like Sicilian delicacies and coffee, yes I have called call centers and I've watched "The devil wears Prada", but I think I'll continue doing AI research at my local university for a bit longer. #AGI
April 15, 2024 at 2:01 PM
Today's new job recommendations from (Microsoft-owned)
LinkedIn... Yes, I like Sicilian delicacies and coffee, yes I have called call centers and I've watched "The devil wears Prada", but I think I'll continue doing AI research at my local university for a bit longer. #AGI
LinkedIn... Yes, I like Sicilian delicacies and coffee, yes I have called call centers and I've watched "The devil wears Prada", but I think I'll continue doing AI research at my local university for a bit longer. #AGI
To be sure: nothing wrong with what Andrej writes
December 9, 2023 at 9:40 AM
To be sure: nothing wrong with what Andrej writes
Every serious NLP researcher for the last 12 month: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter:
Andrej Karpathy in December 2023: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter: what a brilliant take, such an amazing explanation!!!
Twitter:
Andrej Karpathy in December 2023: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter: what a brilliant take, such an amazing explanation!!!
December 9, 2023 at 9:36 AM
Every serious NLP researcher for the last 12 month: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter:
Andrej Karpathy in December 2023: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter: what a brilliant take, such an amazing explanation!!!
Twitter:
Andrej Karpathy in December 2023: "LLMs *always* hallucinate, truth is an accidental side-product"
Twitter: what a brilliant take, such an amazing explanation!!!