Juan Diego Rodriguez
@juand-r.bsky.social
CS PhD student at UT Austin in #NLP
Interested in language, reasoning, semantics and cognitive science. One day we'll have more efficient, interpretable and robust models!
Other interests: math, philosophy, cinema
https://www.juandiego-rodriguez.com/
Interested in language, reasoning, semantics and cognitive science. One day we'll have more efficient, interpretable and robust models!
Other interests: math, philosophy, cinema
https://www.juandiego-rodriguez.com/
Pinned
Juan Diego Rodriguez
@juand-r.bsky.social
· Apr 16
One of the ways that LLMs can be inconsistent is the "generator-validator gap," where LLMs deem their own answers incorrect.
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
🎯 We demonstrate that ranking-based discriminator training can significantly reduce this gap, and improvements on one task often generalize to others!
🧵👇
Reposted by Juan Diego Rodriguez
It's a useful reminder that sometimes tech can make a task more efficient for one side (applying for jobs), and more efficient for the other side (writing job adverts), and yet make the system as a whole completely inefficient.
November 14, 2025 at 10:14 AM
It's a useful reminder that sometimes tech can make a task more efficient for one side (applying for jobs), and more efficient for the other side (writing job adverts), and yet make the system as a whole completely inefficient.
Reposted by Juan Diego Rodriguez
November 12, 2025 at 12:27 AM
Reposted by Juan Diego Rodriguez
"Mission: Impossible" was featured in Quanta Magazine! Big thank you to @benbenbrubaker.bsky.social for the wonderful article covering our work on impossible languages. Ben was so thoughtful and thorough in all our conversations, and it really shows in his writing!
Large language models may not be so omnipotent after all. New research shows that LLMs, like humans, prefer to learn some linguistic patterns over others. @benbenbrubaker.bsky.social reports: www.quantamagazine.org/can-ai-model...
Can AI Models Show Us How People Learn? Impossible Languages Point a Way. | Quanta Magazine
Certain grammatical rules never appear in any known language. By constructing artificial languages that have these rules, linguists can use neural networks to explore how people learn.
www.quantamagazine.org
January 14, 2025 at 11:55 PM
"Mission: Impossible" was featured in Quanta Magazine! Big thank you to @benbenbrubaker.bsky.social for the wonderful article covering our work on impossible languages. Ben was so thoughtful and thorough in all our conversations, and it really shows in his writing!
GPT-5 being difficult
November 11, 2025 at 9:07 PM
GPT-5 being difficult
Generator-validator gap:
November 10, 2025 at 10:51 PM
Generator-validator gap:
Reposted by Juan Diego Rodriguez
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
November 10, 2025 at 10:11 PM
New work to appear @ TACL!
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Language models (LMs) are remarkably good at generating novel well-formed sentences, leading to claims that they have mastered grammar.
Yet they often assign higher probability to ungrammatical strings than to grammatical strings.
How can both things be true? 🧵👇
Reposted by Juan Diego Rodriguez
✨ Streaming now: Join us virtually at the Caltech and University of Chicago Conference on AI+Science!
🎥🔗 Livestream Link: aiscienceconference.caltech.edu
At 10:30am PST / 12:30pm CT, we’ll be awarding the Margot and Tom Pritzker Prize for AI in Science Research Excellence
🎥🔗 Livestream Link: aiscienceconference.caltech.edu
At 10:30am PST / 12:30pm CT, we’ll be awarding the Margot and Tom Pritzker Prize for AI in Science Research Excellence
AI+Science Conference
The California Institute of Technology and the University of Chicago are centers of gravity for the study, application, and use of AI and Machine Learning to enable scientific discovery across the physical and biological sciences, advancing core AI principles and training a new generation of interdisciplinary scientists. To both advance this scientific and technical pursuit and demonstrate the leadership of Caltech and UChicago in this space, we will host the The Caltech and University of Chicago Conference on AI+Science, Sponsored by the Margot and Tom Pritzker Foundation, at Caltech from November 10-11, 2025. This event will bring together an elite and diverse cohort of leading researchers in core AI and domain sciences to lead conversations and drive partnerships that will shape future inquiry, industry investment, and entrepreneurial opportunities.
aiscienceconference.caltech.edu
November 10, 2025 at 5:17 PM
✨ Streaming now: Join us virtually at the Caltech and University of Chicago Conference on AI+Science!
🎥🔗 Livestream Link: aiscienceconference.caltech.edu
At 10:30am PST / 12:30pm CT, we’ll be awarding the Margot and Tom Pritzker Prize for AI in Science Research Excellence
🎥🔗 Livestream Link: aiscienceconference.caltech.edu
At 10:30am PST / 12:30pm CT, we’ll be awarding the Margot and Tom Pritzker Prize for AI in Science Research Excellence
Reposted by Juan Diego Rodriguez
"The only thing we can do regarding the forthcoming bursting of the AI bubble is…pray? Are we so helpless in the face of our tech overlords that we must hope for the Almighty to save us, rather than, say, enacting some regulations and employing some critical thinking?"
Thou shalt not falsify the AI bubble
Serenity now, serenity now
buildcognitiveresonance.substack.com
November 10, 2025 at 12:36 PM
"The only thing we can do regarding the forthcoming bursting of the AI bubble is…pray? Are we so helpless in the face of our tech overlords that we must hope for the Almighty to save us, rather than, say, enacting some regulations and employing some critical thinking?"
Reposted by Juan Diego Rodriguez
Beautiful tree on my walk this morning
November 9, 2025 at 1:17 AM
Beautiful tree on my walk this morning
Reposted by Juan Diego Rodriguez
You should ignore any political analysis that fails to consider how the right wing has captured both legacy and social media to spread propaganda in lockstep with the Trump regime.
www.reuters.com/investigatio...
www.reuters.com/investigatio...
November 9, 2025 at 2:34 PM
You should ignore any political analysis that fails to consider how the right wing has captured both legacy and social media to spread propaganda in lockstep with the Trump regime.
www.reuters.com/investigatio...
www.reuters.com/investigatio...
Reposted by Juan Diego Rodriguez
Can LLMs accurately aggregate information over long, information-dense texts? Not yet…
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
November 7, 2025 at 5:07 PM
Can LLMs accurately aggregate information over long, information-dense texts? Not yet…
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
We introduce Oolong, a dataset of simple-to-verify information aggregation questions over long inputs. No model achieves >50% accuracy at 128K on Oolong!
Reposted by Juan Diego Rodriguez
This really was a pleasure to read.
This is fucking brilliant, and you should read it: lithub.com/maybe-dont-t...
Maybe Don’t Talk to the New York Times About Zohran Mamdani
It’s remarkable, the people you’ll hear from. Teach for even a little while at an expensive institution—the term they tend to prefer is “elite”—and odds are that eventually someone who was a studen…
lithub.com
November 8, 2025 at 5:04 PM
This really was a pleasure to read.
Reposted by Juan Diego Rodriguez
‘You should come back to Twitter instead of staying in a liberal echo chamber’
November 8, 2025 at 12:46 PM
‘You should come back to Twitter instead of staying in a liberal echo chamber’
Reposted by Juan Diego Rodriguez
Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP!
Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!
Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!
November 7, 2025 at 6:22 PM
Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP!
Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!
Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!
Reposted by Juan Diego Rodriguez
We wrote a thing about AI, fascism, and why framing this as "hype" is too apolitical
www.liberalcurrents.com/deflating-hy...
www.liberalcurrents.com/deflating-hy...
Deflating “Hype” Won’t Save Us
The problem with AI isn’t hype. The problem is who and what it’s useful for.
www.liberalcurrents.com
September 16, 2025 at 1:31 PM
We wrote a thing about AI, fascism, and why framing this as "hype" is too apolitical
www.liberalcurrents.com/deflating-hy...
www.liberalcurrents.com/deflating-hy...
Reposted by Juan Diego Rodriguez
Thrilled to release Gaperon, an open LLM suite for French, English and Coding 🧀
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
November 7, 2025 at 9:11 PM
Thrilled to release Gaperon, an open LLM suite for French, English and Coding 🧀
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
“Our mission is to ensure that artificial general intelligence benefits all of humanity.”
X and TikTok accounts are dedicated to posting AI-generated videos of women being strangled.
OpenAI’s Sora 2 Floods Social Media With Videos of Women Being Strangled
X and TikTok accounts are dedicated to posting AI-generated videos of women being strangled.
www.404media.co
November 7, 2025 at 5:28 PM
“Our mission is to ensure that artificial general intelligence benefits all of humanity.”
This is hilarious
AI could end scarcity, end humanity - or boost trend growth by 0.2 percentage points
November 7, 2025 at 3:08 PM
This is hilarious
Reposted by Juan Diego Rodriguez
Bombshell report exposes how Meta relied on scam ad profits to fund AI arstechnica.com/tech-policy/...
Bombshell report exposes how Meta relied on scam ad profits to fund AI
Meta goosed its revenue by targeting users likely to click on scam ads, docs show.
arstechnica.com
November 7, 2025 at 1:06 PM
Bombshell report exposes how Meta relied on scam ad profits to fund AI arstechnica.com/tech-policy/...
In the middle of debugging, Claude told me that a file was corrupted and then, in all caps, that this was VERY DANGEROUS, to BACKUP EVERYTHING and contact IT for help.
It was all made up. The file was fine. There was no problem. WTF Claude
It was all made up. The file was fine. There was no problem. WTF Claude
November 7, 2025 at 3:06 AM
In the middle of debugging, Claude told me that a file was corrupted and then, in all caps, that this was VERY DANGEROUS, to BACKUP EVERYTHING and contact IT for help.
It was all made up. The file was fine. There was no problem. WTF Claude
It was all made up. The file was fine. There was no problem. WTF Claude
Reposted by Juan Diego Rodriguez
This is a great idea!
Let CUNY Socialize EdTech for All of Us
Mamdani Win Could Be The First Step Towards Seizing The Means of Knowledge Production (Let CUNY Socialize EdTech for All of Us)
by Matt Seybold This essay originally appeared on Matt Seybold’s The American Vandal Substack. We are grateful for his generous permission to republish it here. An understandable response to …
moneyontheleft.org
November 7, 2025 at 1:10 AM
This is a great idea!
Why are they reporting on this garbage??
A new paper suggests a photo can tell a recruiter much about an applicant’s personality
Should facial analysis help determine whom companies hire?
A new paper suggests a photo can tell a recruiter much about an applicant’s personality
econ.st
November 6, 2025 at 10:57 PM
Why are they reporting on this garbage??
👏👏👏
No opinion about this specific claim, but I do think we need more humanities in our lives and hearts and minds. And more arts. More history, more literature, more painting, more moral philosophy, more poetry. More thinking about and cherishing what makes human life special.
I don’t know about you but I sincerely believe that Zohran Mamdani’s BA Major in Africana Studies enabled him to understand our current conjuncture & its demands of justice. Humanities shape minds & in his case for the better.
I know white tech bros disagree as they continue to collapse our worlds.
I know white tech bros disagree as they continue to collapse our worlds.
November 6, 2025 at 8:29 PM
👏👏👏
Reposted by Juan Diego Rodriguez
Vital piece of investigative reporting from Sky. They've uncovered the X algorithm which feeds users extremist right wing material from the moment they join the site. It is a far-right radicalisation engine, by design.
news.sky.com/story/the-x-...
news.sky.com/story/the-x-...
Elon Musk is boosting the British right - and this shows how
Elon Musk is boosting the British right - and this shows how
news.sky.com
November 6, 2025 at 7:23 AM
Vital piece of investigative reporting from Sky. They've uncovered the X algorithm which feeds users extremist right wing material from the moment they join the site. It is a far-right radicalisation engine, by design.
news.sky.com/story/the-x-...
news.sky.com/story/the-x-...
Very cool work
The secret life of an LM is defined by its internal data types. Inner layers transport abstractions that are more robust than words, like concepts, functions, or pointers.
In new work yesterday, @arnabsensharma.bsky.social et al identify a data type for *predicates*.
bsky.app/profile/arn...
In new work yesterday, @arnabsensharma.bsky.social et al identify a data type for *predicates*.
bsky.app/profile/arn...
Arnab Sen Sharma (@arnabsensharma.bsky.social)
How can a language model find the veggies in a menu?
New pre-print where we investigate the internal mechanisms of LLMs when filtering on a list of options.
Spoiler: turns out LLMs use strategies surprisingly similar to functional programming (think "filter" from python)! 🧵
bsky.app
November 6, 2025 at 7:24 PM
Very cool work