Roei Herzig
@roeiherz.bsky.social
Research Scientist @ IBM Research. Postdoc @ Berkeley AI. PhD @ Tel Aviv University. Working on Compositionality, Multimodal Foundation Models, and Structured Physical Intelligence.
🔗 https://roeiherz.github.io/
📍Bay area 🇺🇲
🔗 https://roeiherz.github.io/
📍Bay area 🇺🇲
Pinned
Roei Herzig
@roeiherz.bsky.social
· Feb 24
What happens when vision🤝 robotics meet? 🚨 Happy to share our new work on Pretraining Robotic Foundational Models!🔥
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
Reposted by Roei Herzig
CVPR panel at the What is Next in Multimodal Foundation Models? workshop kicks off soon!
11:30AM, R207 A–D (Level 2)
Don't miss an amazing discussion with: Ludwig Schmidt, @andrewowens.bsky.social , Arsha Nagrani, and Ani Kembhavi 🔥
@cvprconference.bsky.social
sites.google.com/view/mmfm3rd...
11:30AM, R207 A–D (Level 2)
Don't miss an amazing discussion with: Ludwig Schmidt, @andrewowens.bsky.social , Arsha Nagrani, and Ani Kembhavi 🔥
@cvprconference.bsky.social
sites.google.com/view/mmfm3rd...
MMFM 3rd Workshop - Program
08:30am - Welcome (5min)
08:35am - Keynote Talk by Ludwig Schmidt (Stanford, Anthropic) (25min + 5min QA)
Title: LAION-5B & DataComp: In search of the next generation of multim...
sites.google.com
June 12, 2025 at 3:51 PM
CVPR panel at the What is Next in Multimodal Foundation Models? workshop kicks off soon!
11:30AM, R207 A–D (Level 2)
Don't miss an amazing discussion with: Ludwig Schmidt, @andrewowens.bsky.social , Arsha Nagrani, and Ani Kembhavi 🔥
@cvprconference.bsky.social
sites.google.com/view/mmfm3rd...
11:30AM, R207 A–D (Level 2)
Don't miss an amazing discussion with: Ludwig Schmidt, @andrewowens.bsky.social , Arsha Nagrani, and Ani Kembhavi 🔥
@cvprconference.bsky.social
sites.google.com/view/mmfm3rd...
What happens when vision🤝 robotics meet? 🚨 Happy to share our new work on Pretraining Robotic Foundational Models!🔥
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
February 24, 2025 at 3:49 AM
What happens when vision🤝 robotics meet? 🚨 Happy to share our new work on Pretraining Robotic Foundational Models!🔥
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
ARM4R is an Autoregressive Robotic Model that leverages low-level 4D Representations learned from human video data to yield a better robotic model.
BerkeleyAI 😊
The best friend of Auto-regressive Robotic Models is 4D representations...🤖😻❤️
February 20, 2025 at 5:01 AM
The best friend of Auto-regressive Robotic Models is 4D representations...🤖😻❤️
Our workshop "What is Next in Multimodal Foundation Models?" has been accepted to #CVPR for its 3rd time!
We are cooking amazing talks and an excellent panel for you, so stay tuned!
@cvprconference.bsky.social
We are cooking amazing talks and an excellent panel for you, so stay tuned!
@cvprconference.bsky.social
December 21, 2024 at 7:06 PM
Our workshop "What is Next in Multimodal Foundation Models?" has been accepted to #CVPR for its 3rd time!
We are cooking amazing talks and an excellent panel for you, so stay tuned!
@cvprconference.bsky.social
We are cooking amazing talks and an excellent panel for you, so stay tuned!
@cvprconference.bsky.social
For all our @neuripsconf.bsky.social friends🤖🦋, our work is presented NOW at POSTER #3701.
Come hear us talk our work on many-shot in-context learning and test-time scaling by leveraging the activations! You won't be disappointed😎
#Multimodal-InContextLearning #NeurIPS
Come hear us talk our work on many-shot in-context learning and test-time scaling by leveraging the activations! You won't be disappointed😎
#Multimodal-InContextLearning #NeurIPS
December 12, 2024 at 7:13 PM
For all our @neuripsconf.bsky.social friends🤖🦋, our work is presented NOW at POSTER #3701.
Come hear us talk our work on many-shot in-context learning and test-time scaling by leveraging the activations! You won't be disappointed😎
#Multimodal-InContextLearning #NeurIPS
Come hear us talk our work on many-shot in-context learning and test-time scaling by leveraging the activations! You won't be disappointed😎
#Multimodal-InContextLearning #NeurIPS
December 10, 2024 at 11:24 PM
🚨Excited to share for the first time our work here in 🦋 "Sparse Attention Vector (SAVs)" 🥳
We showed that when done properly, generative multimodal features can be discriminative vision-language classifiers.
A really fun & enjoyable collab w/ @CMU, @BAIR, and @MIT-IBM Lab
arxiv.org/abs/2410.12782
We showed that when done properly, generative multimodal features can be discriminative vision-language classifiers.
A really fun & enjoyable collab w/ @CMU, @BAIR, and @MIT-IBM Lab
arxiv.org/abs/2410.12782
In-Context Learning Enables Robot Action Prediction in LLMs
Recently, Large Language Models (LLMs) have achieved remarkable success using in-context learning (ICL) in the language domain. However, leveraging the ICL capabilities within LLMs to directly predict...
arxiv.org
December 4, 2024 at 9:24 PM
🚨Excited to share for the first time our work here in 🦋 "Sparse Attention Vector (SAVs)" 🥳
We showed that when done properly, generative multimodal features can be discriminative vision-language classifiers.
A really fun & enjoyable collab w/ @CMU, @BAIR, and @MIT-IBM Lab
arxiv.org/abs/2410.12782
We showed that when done properly, generative multimodal features can be discriminative vision-language classifiers.
A really fun & enjoyable collab w/ @CMU, @BAIR, and @MIT-IBM Lab
arxiv.org/abs/2410.12782
My hot take it is essential to have many-shot capabilities. In our NeurIPS work arxiv.org/abs/2406.15334, we showed how to use Multimodal Task Vectors for many-shot.
But, I'm not sure it makes sense to pretrain for this. ICL is an emergent property, not a downstream task...
Anyway, nice work!
But, I'm not sure it makes sense to pretrain for this. ICL is an emergent property, not a downstream task...
Anyway, nice work!
November 28, 2024 at 8:07 PM
My hot take it is essential to have many-shot capabilities. In our NeurIPS work arxiv.org/abs/2406.15334, we showed how to use Multimodal Task Vectors for many-shot.
But, I'm not sure it makes sense to pretrain for this. ICL is an emergent property, not a downstream task...
Anyway, nice work!
But, I'm not sure it makes sense to pretrain for this. ICL is an emergent property, not a downstream task...
Anyway, nice work!
🤔
VAEs vs GANs will go down in history, much like Bayesian vs Frequentist learning from long gone years. Let us enjoy this moment for what it is — a beautiful moment in time. Too soon it will be gone. Blue skies are smiling on all of us today. Nothing but blue skies do we see.
VAEs won test of time at @iclr-conf.bsky.social — I had the honor of presenting the award. Normalizing flows might get the short end of the stick though.
November 28, 2024 at 1:02 AM
🤔
You can access keynotes, papers, and spotlights on Robotic Learning for free! So cool!🤖
youtube.com/watch?v=0joZ...
#Robotics #DeepLearning #CoRL2024
youtube.com/watch?v=0joZ...
#Robotics #DeepLearning #CoRL2024
youtube.com
November 26, 2024 at 5:07 AM
You can access keynotes, papers, and spotlights on Robotic Learning for free! So cool!🤖
youtube.com/watch?v=0joZ...
#Robotics #DeepLearning #CoRL2024
youtube.com/watch?v=0joZ...
#Robotics #DeepLearning #CoRL2024
Reposted by Roei Herzig
📣 Stony Brook University’s Department of Computer Science invites applications for a tenure-track assistant professor position with an expected starting date of Fall 2025.
Link to the job post: careercenter.cra.org/job/assistan...
Link to the job post: careercenter.cra.org/job/assistan...
Assistant Professor (tenure-track) - Computer Science, College of Engineering and Applied Sciences
Tenure-Track Assistant Professor in AI Department of Computer Science Stony Brook University Stony Brook University’s Department of Computer Science invites applications for a tenure-track assistant p...
careercenter.cra.org
November 25, 2024 at 9:31 PM
📣 Stony Brook University’s Department of Computer Science invites applications for a tenure-track assistant professor position with an expected starting date of Fall 2025.
Link to the job post: careercenter.cra.org/job/assistan...
Link to the job post: careercenter.cra.org/job/assistan...
For all the ML/AI researchers, are you still tweeting both at X and BK at the same time? Is there a convenient way to do this?
a penguin with a fish on its head is on a wooden raft with the words little help written above it
ALT: a penguin with a fish on its head is on a wooden raft with the words little help written above it
media.tenor.com
November 24, 2024 at 5:57 PM
For all the ML/AI researchers, are you still tweeting both at X and BK at the same time? Is there a convenient way to do this?
