Deepak Ramachandran
@thesilverbail.bsky.social
Reposted by Deepak Ramachandran
#ICML2025
Is standard RLHF optimal in view of test-time scaling? Unsurprisingly no.
We show a simple change to standard RLHF framework that involves 𝐫𝐞𝐰𝐚𝐫𝐝 𝐜𝐚𝐥𝐢𝐛𝐫𝐚𝐭𝐢𝐨𝐧 and 𝐫𝐞𝐰𝐚𝐫𝐝 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 (suited to test-time procedure) is optimal!
Is standard RLHF optimal in view of test-time scaling? Unsurprisingly no.
We show a simple change to standard RLHF framework that involves 𝐫𝐞𝐰𝐚𝐫𝐝 𝐜𝐚𝐥𝐢𝐛𝐫𝐚𝐭𝐢𝐨𝐧 and 𝐫𝐞𝐰𝐚𝐫𝐝 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 (suited to test-time procedure) is optimal!
Inference-time procedures (e.g. Best-of-N, CoT) have been instrumental to recent development of LLMs. Standard RLHF focuses only on improving the trained model. This creates a train/inference mismatch.
𝘊𝘢𝘯 𝘸𝘦 𝘢𝘭𝘪𝘨𝘯 𝘰𝘶𝘳 𝘮𝘰𝘥𝘦𝘭 𝘵𝘰 𝘣𝘦𝘵𝘵𝘦𝘳 𝘴𝘶𝘪𝘵 𝘢 𝘨𝘪𝘷𝘦𝘯 𝘪𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦-𝘵𝘪𝘮𝘦 𝘱𝘳𝘰𝘤𝘦𝘥𝘶𝘳𝘦?
Check out below.
𝘊𝘢𝘯 𝘸𝘦 𝘢𝘭𝘪𝘨𝘯 𝘰𝘶𝘳 𝘮𝘰𝘥𝘦𝘭 𝘵𝘰 𝘣𝘦𝘵𝘵𝘦𝘳 𝘴𝘶𝘪𝘵 𝘢 𝘨𝘪𝘷𝘦𝘯 𝘪𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦-𝘵𝘪𝘮𝘦 𝘱𝘳𝘰𝘤𝘦𝘥𝘶𝘳𝘦?
Check out below.
May 9, 2025 at 12:20 AM
#ICML2025
Is standard RLHF optimal in view of test-time scaling? Unsurprisingly no.
We show a simple change to standard RLHF framework that involves 𝐫𝐞𝐰𝐚𝐫𝐝 𝐜𝐚𝐥𝐢𝐛𝐫𝐚𝐭𝐢𝐨𝐧 and 𝐫𝐞𝐰𝐚𝐫𝐝 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 (suited to test-time procedure) is optimal!
Is standard RLHF optimal in view of test-time scaling? Unsurprisingly no.
We show a simple change to standard RLHF framework that involves 𝐫𝐞𝐰𝐚𝐫𝐝 𝐜𝐚𝐥𝐢𝐛𝐫𝐚𝐭𝐢𝐨𝐧 and 𝐫𝐞𝐰𝐚𝐫𝐝 𝐭𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 (suited to test-time procedure) is optimal!
Reposted by Deepak Ramachandran
How do language models generalize from information they learn in-context vs. via finetuning? In arxiv.org/abs/2505.00661 we show that in-context learning can generalize more flexibly, illustrating key differences in the inductive biases of these modes of learning — and ways to improve finetuning. 1/
arxiv.org
May 2, 2025 at 5:02 PM
How do language models generalize from information they learn in-context vs. via finetuning? In arxiv.org/abs/2505.00661 we show that in-context learning can generalize more flexibly, illustrating key differences in the inductive biases of these modes of learning — and ways to improve finetuning. 1/
Gemini is now Pareto-optimal across most price points....Amazon Nova still seems to do a great job at the very lowest.
www.linkedin.com/posts/jonas-...
www.linkedin.com/posts/jonas-...
Gemini is finally Pareto optimal. Best model family at all price points… | Jonas Adler
Gemini is finally Pareto optimal. Best model family at all price points and we haven't even shopped the entire 2.5 family yet.
www.linkedin.com
April 16, 2025 at 1:07 PM
Gemini is now Pareto-optimal across most price points....Amazon Nova still seems to do a great job at the very lowest.
www.linkedin.com/posts/jonas-...
www.linkedin.com/posts/jonas-...
It's great to see fundamental research contributions from Indian institutions that aren't IIT/IISc/TIFR : telanganatoday.com/hyderabad-ba...
Hyderabad-based scientists play key role in Large Hadron Collider experiment that wins Breakthrough Prize
Group led by Dr Bhawna Gomber at CASEST, School of Physics, UoH, contributes in the form of data analysis, trigger electronics, cutting-edge research
telanganatoday.com
April 15, 2025 at 5:16 PM
It's great to see fundamental research contributions from Indian institutions that aren't IIT/IISc/TIFR : telanganatoday.com/hyderabad-ba...
We have always been at war with Eastasia.
April 9, 2025 at 6:38 PM
We have always been at war with Eastasia.
Gemini 2.0 Image output is Live on aistudio.google.com . This was an amazing effort by manygoo many people in the Gemini team and partners at GDM + rest of Google; and I'm so honoured and priveleged to have been part of it. 🧵->
Google AI Studio
Google AI Studio is the fastest way to start building with Gemini, our next generation family of multimodal generative AI models.
aistudio.google.com
March 12, 2025 at 5:11 PM
Gemini 2.0 Image output is Live on aistudio.google.com . This was an amazing effort by manygoo many people in the Gemini team and partners at GDM + rest of Google; and I'm so honoured and priveleged to have been part of it. 🧵->
Reposted by Deepak Ramachandran
New paper: Simulating Time With Square-Root Space
people.csail.mit.edu/rrw/time-vs-...
It's still hard for me to believe it myself, but I seem to have shown that TIME[t] is contained in SPACE[sqrt{t log t}].
To appear in STOC. Comments are very welcome!
people.csail.mit.edu/rrw/time-vs-...
It's still hard for me to believe it myself, but I seem to have shown that TIME[t] is contained in SPACE[sqrt{t log t}].
To appear in STOC. Comments are very welcome!
people.csail.mit.edu
February 21, 2025 at 10:19 PM
New paper: Simulating Time With Square-Root Space
people.csail.mit.edu/rrw/time-vs-...
It's still hard for me to believe it myself, but I seem to have shown that TIME[t] is contained in SPACE[sqrt{t log t}].
To appear in STOC. Comments are very welcome!
people.csail.mit.edu/rrw/time-vs-...
It's still hard for me to believe it myself, but I seem to have shown that TIME[t] is contained in SPACE[sqrt{t log t}].
To appear in STOC. Comments are very welcome!
Reposted by Deepak Ramachandran
The American Mathematical Society has also started a page to coordinate support for professional mathematics, so far focusing on executive orders impacting the National Science Foundation: www.ams.org/government/g...
AMS :: Take Action
www.ams.org
February 22, 2025 at 2:59 PM
The American Mathematical Society has also started a page to coordinate support for professional mathematics, so far focusing on executive orders impacting the National Science Foundation: www.ams.org/government/g...
Reposted by Deepak Ramachandran
A letter of support for the NIH funding of biomedical research, and the damage wrought by imposing severe caps on indirect costs: docs.google.com/forms/d/1Agz...
Protect NIH Research: Advocate for Full Funding
As researchers representing universities across the country, the last few weeks have been filled with uncertainty. As you are aware, the Trump administration’s National Institutes of Health (NIH) prop...
docs.google.com
February 22, 2025 at 2:57 PM
A letter of support for the NIH funding of biomedical research, and the damage wrought by imposing severe caps on indirect costs: docs.google.com/forms/d/1Agz...
A hard problem I found for LLMs to get right: 'Which of Quine's two dogmas is about the analytic- synthetic distinction?' it's a common misconception that it's the first. But it's actually *both* (deducible by reading en.m.wikipedia.org/wiki/Two_Dog... carefully)
Two Dogmas of Empiricism - Wikipedia
en.m.wikipedia.org
February 20, 2025 at 2:03 AM
A hard problem I found for LLMs to get right: 'Which of Quine's two dogmas is about the analytic- synthetic distinction?' it's a common misconception that it's the first. But it's actually *both* (deducible by reading en.m.wikipedia.org/wiki/Two_Dog... carefully)
Imagen 3 (deepmind.google/technologies...) is now the top ranking model on the lmsys image generation arena, by a significant amount. Proud to have been part of the team that built it (and there's even more to come soon !).
February 4, 2025 at 1:36 AM
Imagen 3 (deepmind.google/technologies...) is now the top ranking model on the lmsys image generation arena, by a significant amount. Proud to have been part of the team that built it (and there's even more to come soon !).
There are many content creators that have made it huge by putting in a lot of work. Good for them but I am confused by how unbalanced the content economy Is. So many small creators creating unique content out there that deserve far more love and support.
January 25, 2025 at 5:51 PM
There are many content creators that have made it huge by putting in a lot of work. Good for them but I am confused by how unbalanced the content economy Is. So many small creators creating unique content out there that deserve far more love and support.
Check out our new paper on Focus-N-Fix, a simple and effective approach to Fine-Tuning Text-to-Image Generation models by only fixing regions that were problematic in the image from the base model.
arxiv.org/abs/2501.06481
arxiv.org/abs/2501.06481
Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation
Text-to-image (T2I) generation has made significant advances in recent years, but challenges still remain in the generation of perceptual artifacts, misalignment with complex prompts, and safety. The ...
arxiv.org
January 19, 2025 at 3:48 PM
Check out our new paper on Focus-N-Fix, a simple and effective approach to Fine-Tuning Text-to-Image Generation models by only fixing regions that were problematic in the image from the base model.
arxiv.org/abs/2501.06481
arxiv.org/abs/2501.06481
An interesting tidbit about the late great Adlai Stevenson : Stevenson was approached by Soviet ambassador Menshikov who offered Soviet financial and public relations help to assist him in getting elected if he decided to run...
December 30, 2024 at 3:43 AM
An interesting tidbit about the late great Adlai Stevenson : Stevenson was approached by Soviet ambassador Menshikov who offered Soviet financial and public relations help to assist him in getting elected if he decided to run...
Reposted by Deepak Ramachandran
Super proud to have been part of the imagen 3 work and huge shout out to the veo 2 team !
Here's Veo 2, the latest version of our video generation model, as well as a substantial upgrade for Imagen 3 🧑🍳🚢
(Did I mention we are hiring on the Generative Media team, btw 👀)
blog.google/technology/g...
(Did I mention we are hiring on the Generative Media team, btw 👀)
blog.google/technology/g...
State-of-the-art video and image generation with Veo 2 and Imagen 3
We’re rolling out a new, state-of-the-art video model, Veo 2, and updates to Imagen 3. Plus, check out our new experiment, Whisk.
blog.google
December 16, 2024 at 7:26 PM
Super proud to have been part of the imagen 3 work and huge shout out to the veo 2 team !
Reposted by Deepak Ramachandran
We're hiring!
We need more part-time researchers to work with our interdisciplinary, all-virtual team of scientists and fellows studying climate solutions. Do you have research experience in climate +
#electricitygrid
#transportation
#oceans
#buildings
#agriculture
drawdown.org/careers/rese...
We need more part-time researchers to work with our interdisciplinary, all-virtual team of scientists and fellows studying climate solutions. Do you have research experience in climate +
#electricitygrid
#transportation
#oceans
#buildings
#agriculture
drawdown.org/careers/rese...
Research Fellow
Our mission is to help the world reach “Drawdown" as quickly, safely, and equitably as possible.
drawdown.org
December 5, 2024 at 10:14 PM
We're hiring!
We need more part-time researchers to work with our interdisciplinary, all-virtual team of scientists and fellows studying climate solutions. Do you have research experience in climate +
#electricitygrid
#transportation
#oceans
#buildings
#agriculture
drawdown.org/careers/rese...
We need more part-time researchers to work with our interdisciplinary, all-virtual team of scientists and fellows studying climate solutions. Do you have research experience in climate +
#electricitygrid
#transportation
#oceans
#buildings
#agriculture
drawdown.org/careers/rese...
Homeboy just took it down !
Gukesh has become the youngest World Chess Champion in history! Congratulations to the 18-year-old youngest World Champion ever!
December 12, 2024 at 2:30 PM
Homeboy just took it down !
Combining the reasoning and interactive power of an LLM with native Image Output enabled some magical new experiences. Proud to be part of the team that built this !
youtu.be/7RqFLp0TqV0?...
youtu.be/7RqFLp0TqV0?...
Building with Gemini 2.0: Native image output
YouTube video by Google for Developers
youtu.be
December 12, 2024 at 2:14 PM
Combining the reasoning and interactive power of an LLM with native Image Output enabled some magical new experiences. Proud to be part of the team that built this !
youtu.be/7RqFLp0TqV0?...
youtu.be/7RqFLp0TqV0?...
Reposted by Deepak Ramachandran
A woman who worked at an IHOP for 13 YEARS was fired for serving a homeless man a stack of pancakes and a glass of water
"I need my job, but I would still do it again. I truly would. I would still help somebody if I could."
"I need my job, but I would still do it again. I truly would. I would still help somebody if I could."
‘I need my job’: Server at Lakeland IHOP claims she was fired after feeding man in need
A Polk County woman said her simple act of kindness left her out of a job right before the holidays.
www.wfla.com
December 2, 2024 at 3:33 PM
A woman who worked at an IHOP for 13 YEARS was fired for serving a homeless man a stack of pancakes and a glass of water
"I need my job, but I would still do it again. I truly would. I would still help somebody if I could."
"I need my job, but I would still do it again. I truly would. I would still help somebody if I could."
Reposted by Deepak Ramachandran
Anne Gagneux, Ségolène Martin, @quentinbertrand.bsky.social Remi Emonet and I wrote a tutorial blog post on flow matching: dl.heeere.com/conditional-... with lots of illustrations and intuition!
We got this idea after their cool work on improving Plug and Play with FM: arxiv.org/abs/2410.02423
We got this idea after their cool work on improving Plug and Play with FM: arxiv.org/abs/2410.02423
November 27, 2024 at 9:00 AM
Anne Gagneux, Ségolène Martin, @quentinbertrand.bsky.social Remi Emonet and I wrote a tutorial blog post on flow matching: dl.heeere.com/conditional-... with lots of illustrations and intuition!
We got this idea after their cool work on improving Plug and Play with FM: arxiv.org/abs/2410.02423
We got this idea after their cool work on improving Plug and Play with FM: arxiv.org/abs/2410.02423
Reposted by Deepak Ramachandran
folks working on one or more of the following
🖼️ Image Descriptions to improve Image-Text alignment
AND/OR
💬Multi/Cross Lingual image-text understanding/generation
AND/OR
🌏Geo-Cultural representation and learning
Please DM if you are willing to discuss the current state/challenges/future-work.
🖼️ Image Descriptions to improve Image-Text alignment
AND/OR
💬Multi/Cross Lingual image-text understanding/generation
AND/OR
🌏Geo-Cultural representation and learning
Please DM if you are willing to discuss the current state/challenges/future-work.
November 25, 2024 at 6:57 AM
folks working on one or more of the following
🖼️ Image Descriptions to improve Image-Text alignment
AND/OR
💬Multi/Cross Lingual image-text understanding/generation
AND/OR
🌏Geo-Cultural representation and learning
Please DM if you are willing to discuss the current state/challenges/future-work.
🖼️ Image Descriptions to improve Image-Text alignment
AND/OR
💬Multi/Cross Lingual image-text understanding/generation
AND/OR
🌏Geo-Cultural representation and learning
Please DM if you are willing to discuss the current state/challenges/future-work.
Reposted by Deepak Ramachandran
Too many times as a reviewer and as an AC have I had to deal with this.
Your job is not just to handle your own review/response. You need to interact with other reviewers to come to a decision. In particular, if your review disagrees with everyone else, the burden is *on you* to engage.
Your job is not just to handle your own review/response. You need to interact with other reviewers to come to a decision. In particular, if your review disagrees with everyone else, the burden is *on you* to engage.
If you reviewed for #ICLR, please make sure to read other reviewers' comments too and reflect on whether you may have missed something.
The paper will need to have a single decision; the point of this exercise is not just about addressing each reviewer's concerns individually.
The paper will need to have a single decision; the point of this exercise is not just about addressing each reviewer's concerns individually.
November 23, 2024 at 9:37 PM
Too many times as a reviewer and as an AC have I had to deal with this.
Your job is not just to handle your own review/response. You need to interact with other reviewers to come to a decision. In particular, if your review disagrees with everyone else, the burden is *on you* to engage.
Your job is not just to handle your own review/response. You need to interact with other reviewers to come to a decision. In particular, if your review disagrees with everyone else, the burden is *on you* to engage.