brendan chambers
@societyoftrees.bsky.social
Ithaca | prev Chicago | interested in interconnected systems and humans+computers | past and future: academic and industry research | currently: gardening
Reposted by brendan chambers
Goal-Conditioned Data Augmentation for Offline Reinforcement Learning
Xingshuai Huang, Di Wu, Benoit Boulet
Action editor: Baoxiang Wang
https://openreview.net/forum?id=8K16dplpE0
#reinforcement #conditioning #learns
Xingshuai Huang, Di Wu, Benoit Boulet
Action editor: Baoxiang Wang
https://openreview.net/forum?id=8K16dplpE0
#reinforcement #conditioning #learns
November 23, 2025 at 9:18 AM
Goal-Conditioned Data Augmentation for Offline Reinforcement Learning
Xingshuai Huang, Di Wu, Benoit Boulet
Action editor: Baoxiang Wang
https://openreview.net/forum?id=8K16dplpE0
#reinforcement #conditioning #learns
Xingshuai Huang, Di Wu, Benoit Boulet
Action editor: Baoxiang Wang
https://openreview.net/forum?id=8K16dplpE0
#reinforcement #conditioning #learns
Reposted by brendan chambers
🌊 Global Mangrove Watch is using OlmoEarth to refresh mangrove map baselines faster, with higher accuracy & less manual annotation—allowing orgs + governments to respond to threats more quickly.
Learn more → buff.ly/6xLHLk6
Learn more → buff.ly/6xLHLk6
November 4, 2025 at 2:53 PM
🌊 Global Mangrove Watch is using OlmoEarth to refresh mangrove map baselines faster, with higher accuracy & less manual annotation—allowing orgs + governments to respond to threats more quickly.
Learn more → buff.ly/6xLHLk6
Learn more → buff.ly/6xLHLk6
Reposted by brendan chambers
ml halloween costume concept
October 31, 2025 at 10:08 PM
ml halloween costume concept
Reposted by brendan chambers
Yes!! A POMDP world model benchmark with controlled test environments. So excited to play with this
October 29, 2025 at 12:47 PM
Yes!! A POMDP world model benchmark with controlled test environments. So excited to play with this
Reposted by brendan chambers
Really proud of this project led by @kylecranmer.bsky.social and created alongside these awesome collaborators and friends. I believe this is important work that will help researchers make grounded decisions when building multimodal models with many diverse inputs. Add it to your paper pile!
New paper, with @rkhashmani.me @marielpettee.bsky.social @garrettmerz.bsky.social Hellen Qu. We introduce a framework for generating realistic, highly multimodal datasets with explicitly calculable mutual information. This is helpful for studying self-supervised learning
arxiv.org/abs/2510.21686
arxiv.org/abs/2510.21686
October 28, 2025 at 8:13 PM
Really proud of this project led by @kylecranmer.bsky.social and created alongside these awesome collaborators and friends. I believe this is important work that will help researchers make grounded decisions when building multimodal models with many diverse inputs. Add it to your paper pile!
Reposted by brendan chambers
Cornell (NYC and Ithaca) is recruiting AI postdocs, apply by Nov 20, 2025! If you're interested in working with me on technical approaches to responsible AI (e.g., personalization, fairness), please email me.
academicjobsonline.org/ajo/jobs/30971
academicjobsonline.org/ajo/jobs/30971
Cornell University, Empire AI Fellows Program
Job #AJO30971, Postdoctoral Fellow, Empire AI Fellows Program, Cornell University, New York, New York, US
academicjobsonline.org
October 28, 2025 at 6:19 PM
Cornell (NYC and Ithaca) is recruiting AI postdocs, apply by Nov 20, 2025! If you're interested in working with me on technical approaches to responsible AI (e.g., personalization, fairness), please email me.
academicjobsonline.org/ajo/jobs/30971
academicjobsonline.org/ajo/jobs/30971
Here is the recipe from the latest Thinking Machines blogpost about late post-training:
- generate student rollouts
- query teacher distribution forced on student history
- update using the reverse KL divergence at each step
thinkingmachines.ai/blog/on-poli...
- generate student rollouts
- query teacher distribution forced on student history
- update using the reverse KL divergence at each step
thinkingmachines.ai/blog/on-poli...
October 28, 2025 at 6:37 PM
Here is the recipe from the latest Thinking Machines blogpost about late post-training:
- generate student rollouts
- query teacher distribution forced on student history
- update using the reverse KL divergence at each step
thinkingmachines.ai/blog/on-poli...
- generate student rollouts
- query teacher distribution forced on student history
- update using the reverse KL divergence at each step
thinkingmachines.ai/blog/on-poli...
Reposted by brendan chambers
Did an intro to tokenization lecture today and worked in this thread.
> be a language model
> all you see is tokens
> you don't care, it's all abstracted away
> you live for a world of pure ideas, chain of concepts, reasoning streams
> tokens don't exist.
> all you see is tokens
> you don't care, it's all abstracted away
> you live for a world of pure ideas, chain of concepts, reasoning streams
> tokens don't exist.
October 24, 2025 at 8:54 PM
Did an intro to tokenization lecture today and worked in this thread.
Reposted by brendan chambers
The first fantastic paper on scaling RL with LLMs just dropped. I strongly recommend taking a look and will be sharing more thoughts on the blog soon.
The Art of Scaling Reinforcement Learning Compute for LLMs
Khatri & Madaan et al.
buff.ly/olKwF3X
The Art of Scaling Reinforcement Learning Compute for LLMs
Khatri & Madaan et al.
buff.ly/olKwF3X
October 16, 2025 at 1:59 PM
The first fantastic paper on scaling RL with LLMs just dropped. I strongly recommend taking a look and will be sharing more thoughts on the blog soon.
The Art of Scaling Reinforcement Learning Compute for LLMs
Khatri & Madaan et al.
buff.ly/olKwF3X
The Art of Scaling Reinforcement Learning Compute for LLMs
Khatri & Madaan et al.
buff.ly/olKwF3X
Reposted by brendan chambers
NVIDIA sent me preview hardware of their new DGX Spark 128GB ARM64 4TB "AI supercomputer" - it's a very neat little device, here are my notes so far
simonwillison.net/2025/Oct/14/...
simonwillison.net/2025/Oct/14/...
NVIDIA DGX Spark: great hardware, early days for the ecosystem
NVIDIA sent me a preview unit of their new DGX Spark desktop “AI supercomputer”. I’ve never had hardware to review before! You can consider this my first ever sponsored post …
simonwillison.net
October 14, 2025 at 11:38 PM
NVIDIA sent me preview hardware of their new DGX Spark 128GB ARM64 4TB "AI supercomputer" - it's a very neat little device, here are my notes so far
simonwillison.net/2025/Oct/14/...
simonwillison.net/2025/Oct/14/...
Reposted by brendan chambers
Multi-Head Latent Attention
🔗 github.com/rasbt/LLMs-f...
🔗 github.com/rasbt/LLMs-f...
October 12, 2025 at 1:57 PM
Multi-Head Latent Attention
🔗 github.com/rasbt/LLMs-f...
🔗 github.com/rasbt/LLMs-f...
Reposted by brendan chambers
⚠️ You have marked yourself as an untrusted node in the epistemic network
October 11, 2025 at 1:55 PM
⚠️ You have marked yourself as an untrusted node in the epistemic network
improving pretrained LLMs by searching over iid-noised params, using a reward score (aka fitness criterion) for weight-merging
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning
Fine-tuning pre-trained large language models (LLMs) for down-stream tasks is a critical step in the AI deployment pipeline. Reinforcement learning (RL) is arguably the most prominent fine-tuning meth...
arxiv.org
October 7, 2025 at 5:02 PM
improving pretrained LLMs by searching over iid-noised params, using a reward score (aka fitness criterion) for weight-merging
Reposted by brendan chambers
We are excited to announce 4 outstanding papers 🏆🏆🏆🏆 --> 🧵
October 7, 2025 at 1:23 PM
We are excited to announce 4 outstanding papers 🏆🏆🏆🏆 --> 🧵
Reposted by brendan chambers
LLMs are currently this one big parameter block that stores all sort of facts. In our new preprint, we add context-specific memory parameters to the model, and pretrain the model along with a big bank of memories.
📑 arxiv.org/abs/2510.02375
[1/10]🧵
📑 arxiv.org/abs/2510.02375
[1/10]🧵
October 6, 2025 at 4:06 PM
LLMs are currently this one big parameter block that stores all sort of facts. In our new preprint, we add context-specific memory parameters to the model, and pretrain the model along with a big bank of memories.
📑 arxiv.org/abs/2510.02375
[1/10]🧵
📑 arxiv.org/abs/2510.02375
[1/10]🧵
October 6, 2025 at 3:39 PM
Reposted by brendan chambers
Paper: arxiv.org/pdf/2509.20328
arxiv.org
October 3, 2025 at 1:05 PM
Paper: arxiv.org/pdf/2509.20328
Reposted by brendan chambers
Spaced Scheduling for Large Language Model Training
Amine El hattami, Nicolas Chapados, Christopher Pal
Action editor: Colin Raffel
https://openreview.net/forum?id=p0KTYl2B9T
#scheduling #scheduled #training
Amine El hattami, Nicolas Chapados, Christopher Pal
Action editor: Colin Raffel
https://openreview.net/forum?id=p0KTYl2B9T
#scheduling #scheduled #training
October 2, 2025 at 4:18 AM
Spaced Scheduling for Large Language Model Training
Amine El hattami, Nicolas Chapados, Christopher Pal
Action editor: Colin Raffel
https://openreview.net/forum?id=p0KTYl2B9T
#scheduling #scheduled #training
Amine El hattami, Nicolas Chapados, Christopher Pal
Action editor: Colin Raffel
https://openreview.net/forum?id=p0KTYl2B9T
#scheduling #scheduled #training
Reposted by brendan chambers
really neat clear explainer for the new on “centralizing flows” to theoretically model learning dynamics
Understanding Optimization in Deep Learning with Central Flows
centralflows.github.io
October 1, 2025 at 12:20 PM
really neat clear explainer for the new on “centralizing flows” to theoretically model learning dynamics
Reposted by brendan chambers
Scaling laws don’t just show up in test error — they leave fingerprints in the weight spectrum.
In the feature learning regime, we map this connection: phase diagrams of scaling exponents <-> spectral signatures of trained weights. The paper is: arxiv.org/abs/2509.24882
In the feature learning regime, we map this connection: phase diagrams of scaling exponents <-> spectral signatures of trained weights. The paper is: arxiv.org/abs/2509.24882
September 30, 2025 at 11:02 AM
Scaling laws don’t just show up in test error — they leave fingerprints in the weight spectrum.
In the feature learning regime, we map this connection: phase diagrams of scaling exponents <-> spectral signatures of trained weights. The paper is: arxiv.org/abs/2509.24882
In the feature learning regime, we map this connection: phase diagrams of scaling exponents <-> spectral signatures of trained weights. The paper is: arxiv.org/abs/2509.24882
Reposted by brendan chambers
latent space opera
September 28, 2025 at 4:26 PM
latent space opera
Reposted by brendan chambers
New technical post from Thinky on optimizers but this is the main catch: conditional learning rate per layers.
thinkingmachines.ai/blog/modular...
thinkingmachines.ai/blog/modular...
September 26, 2025 at 6:00 PM
New technical post from Thinky on optimizers but this is the main catch: conditional learning rate per layers.
thinkingmachines.ai/blog/modular...
thinkingmachines.ai/blog/modular...
Reposted by brendan chambers
New (March) Schmidhuber I missed where they use a carefully engineered layer to track the information gained by each (prediction) token for solving problems that require computation. Hidden state is predictive of (a? not necessarily minimal?) description length.
Measuring In-Context Computation Complexity via Hidden State Prediction
Detecting when a neural sequence model does "interesting" computation is an open problem. The next token prediction loss is a poor indicator: Low loss can stem from trivially predictable sequences tha...
arxiv.org
September 9, 2025 at 12:06 AM
New (March) Schmidhuber I missed where they use a carefully engineered layer to track the information gained by each (prediction) token for solving problems that require computation. Hidden state is predictive of (a? not necessarily minimal?) description length.
the Perceptron folks are sharing design specs of their approach to serialize multimodal data as interleaved events
www.perceptron.inc/blog/tensors...
www.perceptron.inc/blog/tensors...
TensorStream - Perceptron
A layer of intelligence for the physical world.
We are a research company building the future of Physical AGI.
www.perceptron.inc
September 24, 2025 at 4:27 PM
the Perceptron folks are sharing design specs of their approach to serialize multimodal data as interleaved events
www.perceptron.inc/blog/tensors...
www.perceptron.inc/blog/tensors...