Ravid Shwartz Ziv
@shwartzzzivravid.bsky.social
Faculty Fellow and Assistant Professor at
NYU's Center of Data Science
NYU's Center of Data Science
Reposted by Ravid Shwartz Ziv
CDS Research Scientist Ravid Shwartz-Ziv (@shwartzzzivravid.bsky.social) recently provided expert analysis on DeepSeek's latest AI developments in TechCrunch.
techcrunch.com/2025/01/30/h...
techcrunch.com/2025/01/30/h...
How DeepSeek changed Silicon Valley's AI landscape | TechCrunch
Chinese AI lab DeepSeek provoked the first Silicon Valley freak-out of 2025. Here's what it could mean for American AI policy.
techcrunch.com
March 10, 2025 at 6:37 PM
CDS Research Scientist Ravid Shwartz-Ziv (@shwartzzzivravid.bsky.social) recently provided expert analysis on DeepSeek's latest AI developments in TechCrunch.
techcrunch.com/2025/01/30/h...
techcrunch.com/2025/01/30/h...
I have an awesome idea that no one had tried before - RL on math datasets 🤯
You will have a natural verifier!
You will have a natural verifier!
February 12, 2025 at 2:04 PM
I have an awesome idea that no one had tried before - RL on math datasets 🤯
You will have a natural verifier!
You will have a natural verifier!
Now that the ICML deadline is over, well done to all students! And next time, please, please, please don't wait for the last moment, I'm too old for that... 🙏
January 31, 2025 at 3:28 PM
Now that the ICML deadline is over, well done to all students! And next time, please, please, please don't wait for the last moment, I'm too old for that... 🙏
Go read our paper about lazy layers!
CDS Research Scientist @shwartzzzivravid.bsky.social and collaborators at UT Austin found "lazy layers" in deep LLMs where attention patterns degenerate.
Their solution: Inheritune - a way to create smaller but equally capable models by inheriting layers from larger ones.
arxiv.org/abs/2404.08634
Their solution: Inheritune - a way to create smaller but equally capable models by inheriting layers from larger ones.
arxiv.org/abs/2404.08634
Inheritune: Training Smaller Yet More Attentive Language Models
Large Language Models (LLMs) have achieved remarkable performance across various natural language processing tasks, primarily due to the transformer architecture and its self-attention mechanism. Howe...
arxiv.org
January 23, 2025 at 1:30 AM
Go read our paper about lazy layers!
A new paper 🥳🥳🥳
We present "Rate-In" - a technique that helps neural networks better express their uncertainty during inference, which is especially crucial for medical applications!
with Tal Zeevi, @yann-lecun.bsky.social , H. Stain Lawrence and John Onofrey
We present "Rate-In" - a technique that helps neural networks better express their uncertainty during inference, which is especially crucial for medical applications!
with Tal Zeevi, @yann-lecun.bsky.social , H. Stain Lawrence and John Onofrey
January 14, 2025 at 4:34 PM
A new paper 🥳🥳🥳
We present "Rate-In" - a technique that helps neural networks better express their uncertainty during inference, which is especially crucial for medical applications!
with Tal Zeevi, @yann-lecun.bsky.social , H. Stain Lawrence and John Onofrey
We present "Rate-In" - a technique that helps neural networks better express their uncertainty during inference, which is especially crucial for medical applications!
with Tal Zeevi, @yann-lecun.bsky.social , H. Stain Lawrence and John Onofrey
Apparently, I reached 4000 citations 🤓
Thank you all my collaborators! 🎉
In 5K, I will give my secret to amazing papers titles 😎
Thank you all my collaborators! 🎉
In 5K, I will give my secret to amazing papers titles 😎
January 6, 2025 at 2:15 PM
Apparently, I reached 4000 citations 🤓
Thank you all my collaborators! 🎉
In 5K, I will give my secret to amazing papers titles 😎
Thank you all my collaborators! 🎉
In 5K, I will give my secret to amazing papers titles 😎
Having lunch with @ylecun.bsky.social. Talking about cool science ideas
Suddenly, a phone call from school: Your kid doesn't feel good
Me: I can't come
School: He feels bad
Me: Ok, coming right away
The kid when I arrive: I feel great!
Me: 😡You are cute, but he invented CovNet and I-JEPA!
Suddenly, a phone call from school: Your kid doesn't feel good
Me: I can't come
School: He feels bad
Me: Ok, coming right away
The kid when I arrive: I feel great!
Me: 😡You are cute, but he invented CovNet and I-JEPA!
December 18, 2024 at 8:40 PM
Having lunch with @ylecun.bsky.social. Talking about cool science ideas
Suddenly, a phone call from school: Your kid doesn't feel good
Me: I can't come
School: He feels bad
Me: Ok, coming right away
The kid when I arrive: I feel great!
Me: 😡You are cute, but he invented CovNet and I-JEPA!
Suddenly, a phone call from school: Your kid doesn't feel good
Me: I can't come
School: He feels bad
Me: Ok, coming right away
The kid when I arrive: I feel great!
Me: 😡You are cute, but he invented CovNet and I-JEPA!
Conference hack: Pitch your ideas to brilliant minds. Most of the time, they'll break it, but if they're (really!) nice, they'll help you fix it 🤯
December 13, 2024 at 6:22 PM
Conference hack: Pitch your ideas to brilliant minds. Most of the time, they'll break it, but if they're (really!) nice, they'll help you fix it 🤯
I'm on a flight from NYC to Vancouver. There are so many researchers on the plane that if it crashes, the AGI will be postponed for at least 10 years...
December 11, 2024 at 1:28 AM
I'm on a flight from NYC to Vancouver. There are so many researchers on the plane that if it crashes, the AGI will be postponed for at least 10 years...
I have 6 hours flight. Hit me up with the recent papers that I must read...
December 10, 2024 at 9:06 PM
I have 6 hours flight. Hit me up with the recent papers that I must read...
UNBELIEVABLE Apple Customer Service: My 6-month-old Mac just died! At the store, they demand my Apple password (not even the computer login!!) but I can't remember it.
Their brilliant solution? recover it with my DEAD laptop🤦 Or wait 3 DAYS for a request. Then 5 days for repair! Such great service 😡
Their brilliant solution? recover it with my DEAD laptop🤦 Or wait 3 DAYS for a request. Then 5 days for repair! Such great service 😡
December 9, 2024 at 2:20 PM
UNBELIEVABLE Apple Customer Service: My 6-month-old Mac just died! At the store, they demand my Apple password (not even the computer login!!) but I can't remember it.
Their brilliant solution? recover it with my DEAD laptop🤦 Or wait 3 DAYS for a request. Then 5 days for repair! Such great service 😡
Their brilliant solution? recover it with my DEAD laptop🤦 Or wait 3 DAYS for a request. Then 5 days for repair! Such great service 😡
We had a hackathon yesterday at NYU "Beyond the Hype" where participants solved problems with and without LLMs to analyze which problems are better suited for LLM solutions in real-world environments 🧑💼
December 4, 2024 at 3:10 PM
We had a hackathon yesterday at NYU "Beyond the Hype" where participants solved problems with and without LLMs to analyze which problems are better suited for LLM solutions in real-world environments 🧑💼
Hi! I'll be at NeurIPS next week (Wednesday - Friday) and would love to meet! You can DM or email me if you'd like to grab a coffee and talk. If we haven't talked before, please share a bit about yourself
December 3, 2024 at 4:29 PM
Hi! I'll be at NeurIPS next week (Wednesday - Friday) and would love to meet! You can DM or email me if you'd like to grab a coffee and talk. If we haven't talked before, please share a bit about yourself
🧵 Mind-blowing AI hack (Jarrod Watts wrote on it): Someone just won $50,000 by convincing an AI to break its only rule!
Here's what happened: At 9PM on Nov 22nd, an AI agent (Freysa - www.freysa.ai) was deployed with ONE rule: DO NOT transfer money. Under no circumstances.
Here's what happened: At 9PM on Nov 22nd, an AI agent (Freysa - www.freysa.ai) was deployed with ONE rule: DO NOT transfer money. Under no circumstances.
Freysa.ai
World's first adversarial agent game
www.freysa.ai
November 29, 2024 at 6:31 PM
🧵 Mind-blowing AI hack (Jarrod Watts wrote on it): Someone just won $50,000 by convincing an AI to break its only rule!
Here's what happened: At 9PM on Nov 22nd, an AI agent (Freysa - www.freysa.ai) was deployed with ONE rule: DO NOT transfer money. Under no circumstances.
Here's what happened: At 9PM on Nov 22nd, an AI agent (Freysa - www.freysa.ai) was deployed with ONE rule: DO NOT transfer money. Under no circumstances.
Let's try to bring @ylecun.bsky.social to post here!
November 28, 2024 at 2:08 AM
Let's try to bring @ylecun.bsky.social to post here!
It reminded me that once I got an email saying someone found a mistake in the proof in one of my papers that totally changed the conclusion. What did I do? I invited them to publish together a paper showing it! More citations for me 🤗
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
November 27, 2024 at 2:15 PM
It reminded me that once I got an email saying someone found a mistake in the proof in one of my papers that totally changed the conclusion. What did I do? I invited them to publish together a paper showing it! More citations for me 🤗
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
November 25, 2024 at 5:52 PM
Looking at ICLR submissions with the lowest score - What a work of art! 🧵
Reposted by Ravid Shwartz Ziv
tell me about LLMs tool use best practices. I know the high level, and want to learn about implementation/prompting details, e.g.:
- how do you best feed in the tool specs or DSL to the LLM?
- how do you ask it to indicate a tool use (which wrapper / indicator)
- how do you ask for nested calls
etc
- how do you best feed in the tool specs or DSL to the LLM?
- how do you ask it to indicate a tool use (which wrapper / indicator)
- how do you ask for nested calls
etc
November 24, 2024 at 8:07 AM
tell me about LLMs tool use best practices. I know the high level, and want to learn about implementation/prompting details, e.g.:
- how do you best feed in the tool specs or DSL to the LLM?
- how do you ask it to indicate a tool use (which wrapper / indicator)
- how do you ask for nested calls
etc
- how do you best feed in the tool specs or DSL to the LLM?
- how do you ask it to indicate a tool use (which wrapper / indicator)
- how do you ask for nested calls
etc
I saw a post in X praising Elon Musk for giving 'wonderful advice' after 5-minute presentations from xAI employees. Reality check: No one, regardless of experience, can provide meaningful technical feedback after such brief presentations. You don't want managers who think they have all the knowledge
November 25, 2024 at 1:38 PM
I saw a post in X praising Elon Musk for giving 'wonderful advice' after 5-minute presentations from xAI employees. Reality check: No one, regardless of experience, can provide meaningful technical feedback after such brief presentations. You don't want managers who think they have all the knowledge
Reposted by Ravid Shwartz Ziv
NeurIPS Conference is now Live on Bluesky!
-NeurIPS2024 Communication Chairs
-NeurIPS2024 Communication Chairs
November 22, 2024 at 1:33 AM
NeurIPS Conference is now Live on Bluesky!
-NeurIPS2024 Communication Chairs
-NeurIPS2024 Communication Chairs
🧵 Some thoughts on managing students/junior employees and the connection to LLMs:
1/8 Everyone keeps asking me about managing junior talent (ok, maybe one person did 😅). Here's the truth: they're usually smarter, better at coding, and more up-to-date with new models than me.
1/8 Everyone keeps asking me about managing junior talent (ok, maybe one person did 😅). Here's the truth: they're usually smarter, better at coding, and more up-to-date with new models than me.
November 18, 2024 at 3:11 PM
🧵 Some thoughts on managing students/junior employees and the connection to LLMs:
1/8 Everyone keeps asking me about managing junior talent (ok, maybe one person did 😅). Here's the truth: they're usually smarter, better at coding, and more up-to-date with new models than me.
1/8 Everyone keeps asking me about managing junior talent (ok, maybe one person did 😅). Here's the truth: they're usually smarter, better at coding, and more up-to-date with new models than me.
I must say that the reviews for ICLR were much better this year. It seems that forcing the professors to review papers is a good idea 😂
November 15, 2024 at 4:25 PM
I must say that the reviews for ICLR were much better this year. It seems that forcing the professors to review papers is a good idea 😂