@liang-weixin.bsky.social
Pinned
How can we reduce pretraining costs for multi-modal models without sacrificing quality? We study this Q in our new work: arxiv.org/abs/2411.04996
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!
🎉 Excited to share: "𝐌𝐢𝐱𝐭𝐮𝐫𝐞-𝐨𝐟-𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 (𝐌𝐨𝐓)" has been officially accepted to TMLR (March 2025) and the code is now open-sourced!
📌 GitHub repo: github.com/facebookrese...
📄 Paper: arxiv.org/abs/2411.04996
📌 GitHub repo: github.com/facebookrese...
📄 Paper: arxiv.org/abs/2411.04996
May 9, 2025 at 5:35 AM
🎉 Excited to share: "𝐌𝐢𝐱𝐭𝐮𝐫𝐞-𝐨𝐟-𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 (𝐌𝐨𝐓)" has been officially accepted to TMLR (March 2025) and the code is now open-sourced!
📌 GitHub repo: github.com/facebookrese...
📄 Paper: arxiv.org/abs/2411.04996
📌 GitHub repo: github.com/facebookrese...
📄 Paper: arxiv.org/abs/2411.04996
🚨 Our new study reveals widespread LLM adoption across society:
📊By late 2024, LLMs assist in writing:
- 18% of financial consumer complaints
- 24% of corporate press releases
- Up to 15% of job postings (esp. in small/young firms)
- 14% of UN press releases
arxiv.org/abs/2502.09747
📊By late 2024, LLMs assist in writing:
- 18% of financial consumer complaints
- 24% of corporate press releases
- Up to 15% of job postings (esp. in small/young firms)
- 14% of UN press releases
arxiv.org/abs/2502.09747
March 1, 2025 at 10:28 PM
🚨 Our new study reveals widespread LLM adoption across society:
📊By late 2024, LLMs assist in writing:
- 18% of financial consumer complaints
- 24% of corporate press releases
- Up to 15% of job postings (esp. in small/young firms)
- 14% of UN press releases
arxiv.org/abs/2502.09747
📊By late 2024, LLMs assist in writing:
- 18% of financial consumer complaints
- 24% of corporate press releases
- Up to 15% of job postings (esp. in small/young firms)
- 14% of UN press releases
arxiv.org/abs/2502.09747
🚀 Want 2x faster pretraining for your multi-modal LLM?
🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)!
✅ Mamba w/ Transfusion setting (image + text): Dense-level performance with just 34.76% of the FLOPs
Full paper: 📚 arxiv.org/abs/2501.16295
🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)!
✅ Mamba w/ Transfusion setting (image + text): Dense-level performance with just 34.76% of the FLOPs
Full paper: 📚 arxiv.org/abs/2501.16295
February 6, 2025 at 5:30 AM
🚀 Want 2x faster pretraining for your multi-modal LLM?
🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)!
✅ Mamba w/ Transfusion setting (image + text): Dense-level performance with just 34.76% of the FLOPs
Full paper: 📚 arxiv.org/abs/2501.16295
🧵 Following up on Mixture-of-Transformers (MoT), we're excited to share Mixture-of-Mamba (MoM)!
✅ Mamba w/ Transfusion setting (image + text): Dense-level performance with just 34.76% of the FLOPs
Full paper: 📚 arxiv.org/abs/2501.16295
📢 Can LLMs program themselves to run faster? 🏃⏱️
LLM self-taught to code for next-gen AI hardware!
arxiv.org/abs/2502.02534
1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x better results.
LLM self-taught to code for next-gen AI hardware!
arxiv.org/abs/2502.02534
1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x better results.
February 6, 2025 at 4:41 AM
📢 Can LLMs program themselves to run faster? 🏃⏱️
LLM self-taught to code for next-gen AI hardware!
arxiv.org/abs/2502.02534
1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x better results.
LLM self-taught to code for next-gen AI hardware!
arxiv.org/abs/2502.02534
1/ Programming AI accelerators is a major bottleneck in ML. Our self-improving LLM agent learns to write optimized code for new hardware, achieving 3.9x better results.
Honored that #Nature has highlighted our work again in their latest piece examining #ChatGPT's transformative impact on scientific research and academia over the past two years. h/t @natureportfolio.bsky.social
www.nature.com/articles/d41...
www.nature.com/articles/d41...
December 4, 2024 at 5:15 PM
Honored that #Nature has highlighted our work again in their latest piece examining #ChatGPT's transformative impact on scientific research and academia over the past two years. h/t @natureportfolio.bsky.social
www.nature.com/articles/d41...
www.nature.com/articles/d41...
How well can #GPT4 provide scientific feedback on research projects? We study this in: arxiv.org/abs/2310.01783
We created a pipeline using GPT4 to read 1000s papers (from #Nature, #ICLR, etc.) and generate feedback (eg suggestions for improvement). Then we compare with human expert reviews.
(1/n)
We created a pipeline using GPT4 to read 1000s papers (from #Nature, #ICLR, etc.) and generate feedback (eg suggestions for improvement). Then we compare with human expert reviews.
(1/n)
Can large language models provide useful feedback on research papers? A large-scale empirical analysis
Expert feedback lays the foundation of rigorous research. However, the rapid growth of scholarly production and intricate knowledge specialization challenge the conventional scientific feedback mechan...
arxiv.org
November 22, 2024 at 8:51 PM
How well can #GPT4 provide scientific feedback on research projects? We study this in: arxiv.org/abs/2310.01783
We created a pipeline using GPT4 to read 1000s papers (from #Nature, #ICLR, etc.) and generate feedback (eg suggestions for improvement). Then we compare with human expert reviews.
(1/n)
We created a pipeline using GPT4 to read 1000s papers (from #Nature, #ICLR, etc.) and generate feedback (eg suggestions for improvement). Then we compare with human expert reviews.
(1/n)
Our new study in #COLM2024 estimates that ~17% of recent CS arXiv papers used #LLMs substantially in its writing.
Around 8% for bioRxiv papers.
Paper: arxiv.org/abs/2404.01268 🧵
Around 8% for bioRxiv papers.
Paper: arxiv.org/abs/2404.01268 🧵
November 22, 2024 at 8:49 PM
Our new study in #COLM2024 estimates that ~17% of recent CS arXiv papers used #LLMs substantially in its writing.
Around 8% for bioRxiv papers.
Paper: arxiv.org/abs/2404.01268 🧵
Around 8% for bioRxiv papers.
Paper: arxiv.org/abs/2404.01268 🧵
Excited that our paper quantifying #LLMs usage in paper reviews is selected as an #ICML2024 oral (top 1.5% of submissions)! 🚀
Main results👇
proceedings.mlr.press/v235/liang24...
Media Coverage: The New York Times
nyti.ms/3vwQhdi
Main results👇
proceedings.mlr.press/v235/liang24...
Media Coverage: The New York Times
nyti.ms/3vwQhdi
November 22, 2024 at 8:48 PM
Excited that our paper quantifying #LLMs usage in paper reviews is selected as an #ICML2024 oral (top 1.5% of submissions)! 🚀
Main results👇
proceedings.mlr.press/v235/liang24...
Media Coverage: The New York Times
nyti.ms/3vwQhdi
Main results👇
proceedings.mlr.press/v235/liang24...
Media Coverage: The New York Times
nyti.ms/3vwQhdi
Excited to share that our recent work on LLM in peer review and responsible LLM use is featured in #Nature!
Many thanks to my collaborators for their insights and dedication to advancing fair and ethical AI practices in scientific publishing. #AI #PeerReview
www.nature.com/articles/d41...
Many thanks to my collaborators for their insights and dedication to advancing fair and ethical AI practices in scientific publishing. #AI #PeerReview
www.nature.com/articles/d41...
ChatGPT is transforming peer review — how can we use it responsibly?
At major computer-science publication venues, up to 17% of the peer reviews are now written by artificial intelligence. We need guidelines before things get out of hand.
www.nature.com
November 22, 2024 at 5:07 PM
Excited to share that our recent work on LLM in peer review and responsible LLM use is featured in #Nature!
Many thanks to my collaborators for their insights and dedication to advancing fair and ethical AI practices in scientific publishing. #AI #PeerReview
www.nature.com/articles/d41...
Many thanks to my collaborators for their insights and dedication to advancing fair and ethical AI practices in scientific publishing. #AI #PeerReview
www.nature.com/articles/d41...
How can we reduce pretraining costs for multi-modal models without sacrificing quality? We study this Q in our new work: arxiv.org/abs/2411.04996
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!
November 22, 2024 at 1:36 AM
How can we reduce pretraining costs for multi-modal models without sacrificing quality? We study this Q in our new work: arxiv.org/abs/2411.04996
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!
✅ MoT achieves dense-level 7B performance with up to 66% fewer FLOPs!