Juan Diego Rodriguez
@juand-r.bsky.social
CS PhD student at UT Austin in #NLP
Interested in language, reasoning, semantics and cognitive science. One day we'll have more efficient, interpretable and robust models!
Other interests: math, philosophy, cinema
https://www.juandiego-rodriguez.com/
Interested in language, reasoning, semantics and cognitive science. One day we'll have more efficient, interpretable and robust models!
Other interests: math, philosophy, cinema
https://www.juandiego-rodriguez.com/
GPT-5 being difficult
November 11, 2025 at 9:07 PM
GPT-5 being difficult
Some Halloween reading
October 31, 2025 at 6:16 PM
Some Halloween reading
Relatedly, I think this may be a better depiction of LLMs than the shoggoth meme
September 24, 2025 at 2:37 PM
Relatedly, I think this may be a better depiction of LLMs than the shoggoth meme
I keep coming back to this bit from John Stuart Mill's
"On Liberty":
"On Liberty":
August 6, 2025 at 3:47 AM
I keep coming back to this bit from John Stuart Mill's
"On Liberty":
"On Liberty":
These encompass three tiers of interpretive complexity:
1. Stylistic feature identification
2. Context retrieval (e.g., historical or literary context)
3. Multi-hop reasoning between style and boarder contexts.
1. Stylistic feature identification
2. Context retrieval (e.g., historical or literary context)
3. Multi-hop reasoning between style and boarder contexts.
July 27, 2025 at 7:19 PM
These encompass three tiers of interpretive complexity:
1. Stylistic feature identification
2. Context retrieval (e.g., historical or literary context)
3. Multi-hop reasoning between style and boarder contexts.
1. Stylistic feature identification
2. Context retrieval (e.g., historical or literary context)
3. Multi-hop reasoning between style and boarder contexts.
🎉 New Benchmark Alert: KRISTEVA – Close‑Reading for LLMs📚
I’m excited to announce a new paper accepted to ACL 2025, in collaboration with Patrick Sui, Philippe Laban, and others!
I’m excited to announce a new paper accepted to ACL 2025, in collaboration with Patrick Sui, Philippe Laban, and others!
July 27, 2025 at 7:19 PM
🎉 New Benchmark Alert: KRISTEVA – Close‑Reading for LLMs📚
I’m excited to announce a new paper accepted to ACL 2025, in collaboration with Patrick Sui, Philippe Laban, and others!
I’m excited to announce a new paper accepted to ACL 2025, in collaboration with Patrick Sui, Philippe Laban, and others!
Our final South by Semantics lecture at UT Austin is happening on Wednesday April 23!
April 21, 2025 at 1:39 PM
Our final South by Semantics lecture at UT Austin is happening on Wednesday April 23!
🔍 Results? We find RankAlign reduces the Generator-Validator gap by 31.8% (Pearson correlation) on average, significantly outperforming existing methods.
🎯 RankAlign also generalizes well to new, unseen tasks and lexical items, making LLMs more trustworthy!
🎯 RankAlign also generalizes well to new, unseen tasks and lexical items, making LLMs more trustworthy!
April 16, 2025 at 6:03 PM
🔍 Results? We find RankAlign reduces the Generator-Validator gap by 31.8% (Pearson correlation) on average, significantly outperforming existing methods.
🎯 RankAlign also generalizes well to new, unseen tasks and lexical items, making LLMs more trustworthy!
🎯 RankAlign also generalizes well to new, unseen tasks and lexical items, making LLMs more trustworthy!
🛠️ We define the Generator-Validator gap in a new way using the correlation between "generator" and “validator” scores: the likelihood of a completion vs. the likelihood of confirming it.
This extends prior work by reasoning about the entire distribution of possible outputs across all examples.
This extends prior work by reasoning about the entire distribution of possible outputs across all examples.
April 16, 2025 at 6:03 PM
🛠️ We define the Generator-Validator gap in a new way using the correlation between "generator" and “validator” scores: the likelihood of a completion vs. the likelihood of confirming it.
This extends prior work by reasoning about the entire distribution of possible outputs across all examples.
This extends prior work by reasoning about the entire distribution of possible outputs across all examples.
One of the ways that LLMs can be inconsistent is the "generator-validator gap," where LLMs deem their own answers incorrect.
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
April 16, 2025 at 6:03 PM
One of the ways that LLMs can be inconsistent is the "generator-validator gap," where LLMs deem their own answers incorrect.
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
Announcing another South by Semantics talk happening at UT Austin!
I'm looking forward to hearing Cameron Buckner (@cameronbuckner.bsky.social) talk about "Language Models as Models of Human Reasoning"on March 26th.
I'm looking forward to hearing Cameron Buckner (@cameronbuckner.bsky.social) talk about "Language Models as Models of Human Reasoning"on March 26th.
March 25, 2025 at 4:51 PM
Announcing another South by Semantics talk happening at UT Austin!
I'm looking forward to hearing Cameron Buckner (@cameronbuckner.bsky.social) talk about "Language Models as Models of Human Reasoning"on March 26th.
I'm looking forward to hearing Cameron Buckner (@cameronbuckner.bsky.social) talk about "Language Models as Models of Human Reasoning"on March 26th.
Gabe Dupre (Assistant Professor of Philosophy at UC Davis) will talk about LLMs and the linguistic system.
gabedupre.weebly.com
Date: Friday March 14, 2025
Time: 3:30 pm
Location: WAG 316
Title: 21st Century Wickelphones: Language Models and the Linguistic System
gabedupre.weebly.com
Date: Friday March 14, 2025
Time: 3:30 pm
Location: WAG 316
Title: 21st Century Wickelphones: Language Models and the Linguistic System
March 14, 2025 at 4:14 AM
Gabe Dupre (Assistant Professor of Philosophy at UC Davis) will talk about LLMs and the linguistic system.
gabedupre.weebly.com
Date: Friday March 14, 2025
Time: 3:30 pm
Location: WAG 316
Title: 21st Century Wickelphones: Language Models and the Linguistic System
gabedupre.weebly.com
Date: Friday March 14, 2025
Time: 3:30 pm
Location: WAG 316
Title: 21st Century Wickelphones: Language Models and the Linguistic System
Finally made a good amaretto sour, following Morgenthaler’s recipe
February 10, 2025 at 3:54 AM
Finally made a good amaretto sour, following Morgenthaler’s recipe
Musk takes control of the federal government.
The NYT fails to report what's happening yet again:
The NYT fails to report what's happening yet again:
February 3, 2025 at 3:22 PM
Musk takes control of the federal government.
The NYT fails to report what's happening yet again:
The NYT fails to report what's happening yet again:
My new favorite cocktail: the sherry flip
2 oz oloroso sherry, 1/2 oz Demerara syrup, 1 egg, and nutmeg
2 oz oloroso sherry, 1/2 oz Demerara syrup, 1 egg, and nutmeg
January 25, 2025 at 8:06 PM
My new favorite cocktail: the sherry flip
2 oz oloroso sherry, 1/2 oz Demerara syrup, 1 egg, and nutmeg
2 oz oloroso sherry, 1/2 oz Demerara syrup, 1 egg, and nutmeg
Today I found this illustration of a 17th century thermometer with Stranger Things vibes
January 2, 2025 at 11:47 PM
Today I found this illustration of a 17th century thermometer with Stranger Things vibes
