Ed Henry
@edhenry.bsky.social
Distinguished Scientist and Engineer @DellTech
Things I like : Mathematics, Machine Learning, Causality, Networks, and Philosophy.
Things I like : Mathematics, Machine Learning, Causality, Networks, and Philosophy.
Reposted by Ed Henry
The transformer was invented in Google. RLHF was not invented in industry labs, but came to prominence in OpenAI and DeepMind. I took 5 of the most influential papers (black dots) and visualized their references. Blue dots are papers that acknowledge federal funding (DARPA, NSF).
April 12, 2025 at 2:35 AM
The transformer was invented in Google. RLHF was not invented in industry labs, but came to prominence in OpenAI and DeepMind. I took 5 of the most influential papers (black dots) and visualized their references. Blue dots are papers that acknowledge federal funding (DARPA, NSF).
Reposted by Ed Henry
New Preprint! Interested in learning about how working memory is subserved by both compositional and generative mechanisms? Read on!
April 14, 2025 at 2:24 AM
New Preprint! Interested in learning about how working memory is subserved by both compositional and generative mechanisms? Read on!
Reposted by Ed Henry
After a short era in which people questioned the value of academia in ML, its value is more obvious than ever. Big labs stopped publishing the minute commercial incentives showed up and are relentlessly focused on a singular vision of scaling. Academia is a meaningful complement, bringing...
1/2
1/2
April 14, 2025 at 1:04 AM
After a short era in which people questioned the value of academia in ML, its value is more obvious than ever. Big labs stopped publishing the minute commercial incentives showed up and are relentlessly focused on a singular vision of scaling. Academia is a meaningful complement, bringing...
1/2
1/2
The AI research scene is dealing with a second order hardware lottery[1] effect right now, GPUs being the first, in that many papers being published are based on pretrained models trained using large research clusters available to only a few labs.
LinkedIn
This link will take you to a page that’s not on LinkedIn
lnkd.in
April 14, 2025 at 5:00 AM
The AI research scene is dealing with a second order hardware lottery[1] effect right now, GPUs being the first, in that many papers being published are based on pretrained models trained using large research clusters available to only a few labs.
Reposted by Ed Henry
“Philosophy would render us entirely skeptics, were not nature too strong for it.”
— David Hume, An Enquiry Concerning Human Understanding
#philosophy #philsky
— David Hume, An Enquiry Concerning Human Understanding
#philosophy #philsky
March 21, 2025 at 3:06 AM
“Philosophy would render us entirely skeptics, were not nature too strong for it.”
— David Hume, An Enquiry Concerning Human Understanding
#philosophy #philsky
— David Hume, An Enquiry Concerning Human Understanding
#philosophy #philsky
Reposted by Ed Henry
How it started / how it's going.....
March 18, 2025 at 2:44 AM
How it started / how it's going.....
Reposted by Ed Henry
we released olmo 32b today! ☺️
🐟our largest & best fully open model to-date
🐠right up there w similar size weights-only models from big companies on popular benchmarks
🐡but we used way less compute & all our data, ckpts, code, recipe are free & open
made a nice plot of our post-trained results!✌️
🐟our largest & best fully open model to-date
🐠right up there w similar size weights-only models from big companies on popular benchmarks
🐡but we used way less compute & all our data, ckpts, code, recipe are free & open
made a nice plot of our post-trained results!✌️
March 13, 2025 at 8:42 PM
we released olmo 32b today! ☺️
🐟our largest & best fully open model to-date
🐠right up there w similar size weights-only models from big companies on popular benchmarks
🐡but we used way less compute & all our data, ckpts, code, recipe are free & open
made a nice plot of our post-trained results!✌️
🐟our largest & best fully open model to-date
🐠right up there w similar size weights-only models from big companies on popular benchmarks
🐡but we used way less compute & all our data, ckpts, code, recipe are free & open
made a nice plot of our post-trained results!✌️
Reposted by Ed Henry
Some of his readers have asked Mike Masnick @mmasnick.bsky.social why his technology news site, Tech Dirt, has been covering politics so intensely lately. www.techdirt.com/2025/03/04/w...
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
March 7, 2025 at 12:09 AM
Some of his readers have asked Mike Masnick @mmasnick.bsky.social why his technology news site, Tech Dirt, has been covering politics so intensely lately. www.techdirt.com/2025/03/04/w...
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
I cannot recommend Mike's reply enough. It's exactly what readers need to hear, what journalists need to do.
Reposted by Ed Henry
Just FYI: I wrote about this and other issues involving AI benchmarks in this piece for Science: www.science.org/doi/10.1126/...
How do we know how smart AI systems are?
In 1967, Marvin Minksy, a founder of the field of artificial intelligence (AI), made a bold prediction: “Within a generation…the problem of creating ‘artificial intelligence’ will be substantially sol...
www.science.org
March 5, 2025 at 8:29 PM
Just FYI: I wrote about this and other issues involving AI benchmarks in this piece for Science: www.science.org/doi/10.1126/...
Reposted by Ed Henry
My new paper "Deep Learning is Not So Mysterious or Different": arxiv.org/abs/2503.02113. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12
March 5, 2025 at 3:38 PM
My new paper "Deep Learning is Not So Mysterious or Different": arxiv.org/abs/2503.02113. Generalization behaviours in deep learning can be intuitively understood through a notion of soft inductive biases, and formally characterized with countable hypothesis bounds! 1/12
Reposted by Ed Henry
Awesome LLM Post-training
This repository is a curated collection of the most influential papers, code implementations, benchmarks, and resources related to Large Language Models (LLMs) Post-Training Methodologies.
github.com/mbzuai-oryx/...
This repository is a curated collection of the most influential papers, code implementations, benchmarks, and resources related to Large Language Models (LLMs) Post-Training Methodologies.
github.com/mbzuai-oryx/...
March 4, 2025 at 12:03 AM
Awesome LLM Post-training
This repository is a curated collection of the most influential papers, code implementations, benchmarks, and resources related to Large Language Models (LLMs) Post-Training Methodologies.
github.com/mbzuai-oryx/...
This repository is a curated collection of the most influential papers, code implementations, benchmarks, and resources related to Large Language Models (LLMs) Post-Training Methodologies.
github.com/mbzuai-oryx/...
Reposted by Ed Henry
Thinking about John Locke lately. His core principles:
1) government serves the people,
2) they have the right to remove corrupt governments, and
3) checks and balances are necessary.
Finally: when these are violated, the people have the right of revolution.
iep.utm.edu/locke/#:~:te....
1) government serves the people,
2) they have the right to remove corrupt governments, and
3) checks and balances are necessary.
Finally: when these are violated, the people have the right of revolution.
iep.utm.edu/locke/#:~:te....
Locke, John | Internet Encyclopedia of Philosophy
John Locke was born in 1632 in Wrington, a small village in southwestern England. His father, also named John, was a legal clerk and served with the Parliamentary forces in the English Civil War. His family was well-to-do, but not of particularly high social or economic standing. Locke spent his childhood in the West Country and as a teenager was sent to Westminster School in London.
iep.utm.edu
February 27, 2025 at 2:52 AM
Thinking about John Locke lately. His core principles:
1) government serves the people,
2) they have the right to remove corrupt governments, and
3) checks and balances are necessary.
Finally: when these are violated, the people have the right of revolution.
iep.utm.edu/locke/#:~:te....
1) government serves the people,
2) they have the right to remove corrupt governments, and
3) checks and balances are necessary.
Finally: when these are violated, the people have the right of revolution.
iep.utm.edu/locke/#:~:te....
Reposted by Ed Henry
Starlink embedded in the FAA.
Grok used by the OPM.
Tesla contracts from the DoD.
SpaceX taking over NASA tasks.
We are "NeuraLink requirements for Social Security payments" away from a complete governmental parasitic symbiosis.
Grok used by the OPM.
Tesla contracts from the DoD.
SpaceX taking over NASA tasks.
We are "NeuraLink requirements for Social Security payments" away from a complete governmental parasitic symbiosis.
February 25, 2025 at 3:43 PM
Starlink embedded in the FAA.
Grok used by the OPM.
Tesla contracts from the DoD.
SpaceX taking over NASA tasks.
We are "NeuraLink requirements for Social Security payments" away from a complete governmental parasitic symbiosis.
Grok used by the OPM.
Tesla contracts from the DoD.
SpaceX taking over NASA tasks.
We are "NeuraLink requirements for Social Security payments" away from a complete governmental parasitic symbiosis.
Reposted by Ed Henry
Today we're launching a multi-lab open collaboration, the ARBOR project, to accelerate AI interpretability research for reasoning models. Please join us!
github.com/ARBORproject...
(ARBOR = Analysis of Reasoning Behavior through Open Research)
github.com/ARBORproject...
(ARBOR = Analysis of Reasoning Behavior through Open Research)
GitHub - ARBORproject/arborproject.github.io
Contribute to ARBORproject/arborproject.github.io development by creating an account on GitHub.
github.com
February 20, 2025 at 7:55 PM
Today we're launching a multi-lab open collaboration, the ARBOR project, to accelerate AI interpretability research for reasoning models. Please join us!
github.com/ARBORproject...
(ARBOR = Analysis of Reasoning Behavior through Open Research)
github.com/ARBORproject...
(ARBOR = Analysis of Reasoning Behavior through Open Research)
Reposted by Ed Henry
JUST IN: NASA says there's now a 3.1% chance an asteroid will hit Earth in 2032, up from 2.6% yesterday.
This is the highest risk assessment an asteroid has ever received, surpassing 2.7% in 2004
This is the highest risk assessment an asteroid has ever received, surpassing 2.7% in 2004
February 18, 2025 at 7:24 PM
JUST IN: NASA says there's now a 3.1% chance an asteroid will hit Earth in 2032, up from 2.6% yesterday.
This is the highest risk assessment an asteroid has ever received, surpassing 2.7% in 2004
This is the highest risk assessment an asteroid has ever received, surpassing 2.7% in 2004
Reposted by Ed Henry
Forget “tapestry” or “delve” these are the actual unique giveaway words for each model, relative to each other. arxiv.org/pdf/2502.12150
February 19, 2025 at 3:04 AM
Forget “tapestry” or “delve” these are the actual unique giveaway words for each model, relative to each other. arxiv.org/pdf/2502.12150
Reposted by Ed Henry
An uncensored version of R1 is released 🔥
“R1 1776 is a DeepSeek-R1 reasoning model that has been post-trained by Perplexity AI to remove CCP censorship. The model provides unbiased, accurate, and factual information while maintaining high reasoning capabilities.”
huggingface.co/perplexity-a...
“R1 1776 is a DeepSeek-R1 reasoning model that has been post-trained by Perplexity AI to remove CCP censorship. The model provides unbiased, accurate, and factual information while maintaining high reasoning capabilities.”
huggingface.co/perplexity-a...
perplexity-ai/r1-1776 · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
February 19, 2025 at 3:22 AM
An uncensored version of R1 is released 🔥
“R1 1776 is a DeepSeek-R1 reasoning model that has been post-trained by Perplexity AI to remove CCP censorship. The model provides unbiased, accurate, and factual information while maintaining high reasoning capabilities.”
huggingface.co/perplexity-a...
“R1 1776 is a DeepSeek-R1 reasoning model that has been post-trained by Perplexity AI to remove CCP censorship. The model provides unbiased, accurate, and factual information while maintaining high reasoning capabilities.”
huggingface.co/perplexity-a...
Reposted by Ed Henry
CRA statement about NSF firings cra.org/cuts-to-nsf-...
Cuts to NSF and CISE Directorate Jeopardize American Leadership in Computing
A statement from the Computing Research Association (CRA) The reported termination today of 10 percent of the National Science Foundation’s (NSF) workforce — including significant cuts to the Compu…
cra.org
February 18, 2025 at 11:52 PM
CRA statement about NSF firings cra.org/cuts-to-nsf-...
Reposted by Ed Henry
Why reasoning models will generalize
DeepSeek R1 is just the tip of the ice berg of rapid progress.
People underestimate the long-term potential of “reasoning.”
DeepSeek R1 is just the tip of the ice berg of rapid progress.
People underestimate the long-term potential of “reasoning.”
Why reasoning models will generalize
People underestimate the long-term potential of “reasoning.”
buff.ly
January 28, 2025 at 9:04 PM
Why reasoning models will generalize
DeepSeek R1 is just the tip of the ice berg of rapid progress.
People underestimate the long-term potential of “reasoning.”
DeepSeek R1 is just the tip of the ice berg of rapid progress.
People underestimate the long-term potential of “reasoning.”
Reposted by Ed Henry
Current me: It's only one more project/talk/paper/review...
Future me: Don't do this, I beg you.
Current me: Super interesting, could find a way to fit it in...
Future me: C'mon, remember the rule, just say no!
Current me: & loads of time before the deadline...
Future me: Wait, can you even hear me?
Future me: Don't do this, I beg you.
Current me: Super interesting, could find a way to fit it in...
Future me: C'mon, remember the rule, just say no!
Current me: & loads of time before the deadline...
Future me: Wait, can you even hear me?
January 17, 2025 at 4:02 PM
Current me: It's only one more project/talk/paper/review...
Future me: Don't do this, I beg you.
Current me: Super interesting, could find a way to fit it in...
Future me: C'mon, remember the rule, just say no!
Current me: & loads of time before the deadline...
Future me: Wait, can you even hear me?
Future me: Don't do this, I beg you.
Current me: Super interesting, could find a way to fit it in...
Future me: C'mon, remember the rule, just say no!
Current me: & loads of time before the deadline...
Future me: Wait, can you even hear me?
Reposted by Ed Henry
Super interesting extension of the o1 approach here with addition of search within external RAG-obtained documents for "reasoning within documents".
#LLM #AI #ML
Search-o1: Agentic Search-Enhanced Large Reasoning Models
www.alphaxiv.org/abs/2501.05366
#LLM #AI #ML
Search-o1: Agentic Search-Enhanced Large Reasoning Models
www.alphaxiv.org/abs/2501.05366
Search-o1: Agentic Search-Enhanced Large Reasoning Models | alphaXiv
View recent discussion. Abstract: Large reasoning models (LRMs) like OpenAI-o1 have demonstrated impressive long stepwise reasoning capabilities through large-scale reinforcement learning. However, th...
www.alphaxiv.org
January 11, 2025 at 4:17 PM
Super interesting extension of the o1 approach here with addition of search within external RAG-obtained documents for "reasoning within documents".
#LLM #AI #ML
Search-o1: Agentic Search-Enhanced Large Reasoning Models
www.alphaxiv.org/abs/2501.05366
#LLM #AI #ML
Search-o1: Agentic Search-Enhanced Large Reasoning Models
www.alphaxiv.org/abs/2501.05366
Reposted by Ed Henry
The state of post-training in 2025: a tutorial on modern post-training
A re-record of my NeurIPS tutorial on language modeling (plus some added content on the high level state of things)
Blog + extra context: https://buff.ly/424VvLm
YouTube: https://buff.ly/40808l5
Slides: https://buff.ly/404jGa9
A re-record of my NeurIPS tutorial on language modeling (plus some added content on the high level state of things)
Blog + extra context: https://buff.ly/424VvLm
YouTube: https://buff.ly/40808l5
Slides: https://buff.ly/404jGa9
The state of post-training in 2025
A re-record of my NeurIPS tutorial on language modeling (plus some added content).
buff.ly
January 8, 2025 at 3:38 PM
The state of post-training in 2025: a tutorial on modern post-training
A re-record of my NeurIPS tutorial on language modeling (plus some added content on the high level state of things)
Blog + extra context: https://buff.ly/424VvLm
YouTube: https://buff.ly/40808l5
Slides: https://buff.ly/404jGa9
A re-record of my NeurIPS tutorial on language modeling (plus some added content on the high level state of things)
Blog + extra context: https://buff.ly/424VvLm
YouTube: https://buff.ly/40808l5
Slides: https://buff.ly/404jGa9
Reposted by Ed Henry
In Solidarity with Ann Telnaes. ✊
“Democracy Dies in Darkness.”
anntelnaes.substack.com/p/why-im-qui...
@anntelnaes.bsky.social
“Democracy Dies in Darkness.”
anntelnaes.substack.com/p/why-im-qui...
@anntelnaes.bsky.social
January 5, 2025 at 4:36 PM
In Solidarity with Ann Telnaes. ✊
“Democracy Dies in Darkness.”
anntelnaes.substack.com/p/why-im-qui...
@anntelnaes.bsky.social
“Democracy Dies in Darkness.”
anntelnaes.substack.com/p/why-im-qui...
@anntelnaes.bsky.social