We built CopilotArena this fall in order to evaluate coding models in realistic, interactive environments.
Check out our recent writeup describing the results, as well as details of the system itself.
Work led by @waynechi.bsky.social and Valerie Chen.
.
Check out our recent writeup describing the results, as well as details of the system itself.
Work led by @waynechi.bsky.social and Valerie Chen.
.
What do developers 𝘳𝘦𝘢𝘭𝘭𝘺 think of AI coding assistants?
In October, we launched Copilot Arena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint.
Here's what we have learned /🧵
In October, we launched Copilot Arena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint.
Here's what we have learned /🧵
March 5, 2025 at 4:54 PM
We built CopilotArena this fall in order to evaluate coding models in realistic, interactive environments.
Check out our recent writeup describing the results, as well as details of the system itself.
Work led by @waynechi.bsky.social and Valerie Chen.
.
Check out our recent writeup describing the results, as well as details of the system itself.
Work led by @waynechi.bsky.social and Valerie Chen.
.
great to see more specialized ML conferences! Mega conferences are fun, but at least in my experience with MLSys, I've had much better scientific conversations at smaller ones.
🦕The 19th conference on Neurosymbolic AI will be in beautiful Santa Cruz (CA, USA), September 8-10, 2025!
CFP is now out: 2025.nesyconf.org/call-for-pap...
🚨 Paper deadline: Feb 28 (abstract), March 7 (full)
#neurosymbolic #NeSy2025
CFP is now out: 2025.nesyconf.org/call-for-pap...
🚨 Paper deadline: Feb 28 (abstract), March 7 (full)
#neurosymbolic #NeSy2025
Call for papers
19th International Conference on Neurosymbolic Learning (NeSy 2025, 8-10 September 2025, Santa Cruz, CA, USA)
2025.nesyconf.org
December 11, 2024 at 7:51 PM
great to see more specialized ML conferences! Mega conferences are fun, but at least in my experience with MLSys, I've had much better scientific conversations at smaller ones.
Excited about L2G, led by Wenduo Cheng. We leverage LLMs to beat genomic FMs and strong supervised baselines on a wide range of benchmarks. L2G uses cross-modal transfer (rather than vanilla fine-tuning), and neural architecture search to learn a genomic-specific embedder model.
Can we bypass the resource bottleneck of pretraining genomic Foundation Models? Our work L2G repurposes language LLMs for genomics via cross-modal transfer, matching fine-tuned genomic FMs. Kudos to Wenduo & fantastic collab w/ @atalwalkar.bsky.social. L2G, language to genome; L2G, life’s too good!
L2G: Repurposing Language Models for Genomics Tasks https://www.biorxiv.org/content/10.1101/2024.12.09.627422v1
December 11, 2024 at 7:36 PM
Excited about L2G, led by Wenduo Cheng. We leverage LLMs to beat genomic FMs and strong supervised baselines on a wide range of benchmarks. L2G uses cross-modal transfer (rather than vanilla fine-tuning), and neural architecture search to learn a genomic-specific embedder model.
Reposted by Ameet Talwalkar
Can we bypass the resource bottleneck of pretraining genomic Foundation Models? Our work L2G repurposes language LLMs for genomics via cross-modal transfer, matching fine-tuned genomic FMs. Kudos to Wenduo & fantastic collab w/ @atalwalkar.bsky.social. L2G, language to genome; L2G, life’s too good!
L2G: Repurposing Language Models for Genomics Tasks https://www.biorxiv.org/content/10.1101/2024.12.09.627422v1
December 11, 2024 at 1:41 PM
Can we bypass the resource bottleneck of pretraining genomic Foundation Models? Our work L2G repurposes language LLMs for genomics via cross-modal transfer, matching fine-tuned genomic FMs. Kudos to Wenduo & fantastic collab w/ @atalwalkar.bsky.social. L2G, language to genome; L2G, life’s too good!
Great writeup on the Chatbot Arena team, including a nice photo of @waynechi.bsky.social's back (in the purple shirt). It's been fun collaborating with this team via CoPilot Arena (blog.lmarena.ai/blog/2024/co...), and I'm super impressed with their hustle!
www.wsj.com/tech/ai/the-...
www.wsj.com/tech/ai/the-...
The UC Berkeley Project That Is the AI Industry’s Obsession
Chatbot Arena ranks the world’s best AI models on a leaderboard based on user voting in head-to-head competitions between bots.
www.wsj.com
December 6, 2024 at 7:42 PM
Great writeup on the Chatbot Arena team, including a nice photo of @waynechi.bsky.social's back (in the purple shirt). It's been fun collaborating with this team via CoPilot Arena (blog.lmarena.ai/blog/2024/co...), and I'm super impressed with their hustle!
www.wsj.com/tech/ai/the-...
www.wsj.com/tech/ai/the-...
Check out @junhongshen1.bsky.social's blog post describing this project in more detail:
blog.ml.cmu.edu/2024/12/06/s...
blog.ml.cmu.edu/2024/12/06/s...
December 6, 2024 at 7:37 PM
Check out @junhongshen1.bsky.social's blog post describing this project in more detail:
blog.ml.cmu.edu/2024/12/06/s...
blog.ml.cmu.edu/2024/12/06/s...
Excited to share this work! This was a fun project in collaboration with Scribe, and a great example of the power of open-source FMs when coupled with rich domain-specific data!
1/ Introducing ScribeAgent 🤖! Using 𝗿𝗲𝗮𝗹-𝘄𝗼𝗿𝗹𝗱 𝘄𝗲𝗯 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄 𝗱𝗮𝘁𝗮, we at @scsatcmu.bsky.social and Scribe scribehow.com/ have adapted 𝗴𝗲𝗻𝗲𝗿𝗮𝗹-𝗽𝘂𝗿𝗽𝗼𝘀𝗲 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 𝗟𝗟𝗠𝘀 into 𝘀𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝘄𝗲𝗯 𝗮𝗴𝗲𝗻𝘁𝘀, outperforming agents that rely on proprietary models like GPT-4 and o1-preview. More in 🧵.
December 3, 2024 at 9:17 PM
Excited to share this work! This was a fun project in collaboration with Scribe, and a great example of the power of open-source FMs when coupled with rich domain-specific data!
Reposted by Ameet Talwalkar
if you're a PhD student at CMU doing AI/ML, lmk if you want to be added to this starter pack.
(I don't belong in this list, but I don't know how to remove myself from this pack 😂)
go.bsky.app/9APVxQQ
(I don't belong in this list, but I don't know how to remove myself from this pack 😂)
go.bsky.app/9APVxQQ
December 3, 2024 at 6:27 PM
if you're a PhD student at CMU doing AI/ML, lmk if you want to be added to this starter pack.
(I don't belong in this list, but I don't know how to remove myself from this pack 😂)
go.bsky.app/9APVxQQ
(I don't belong in this list, but I don't know how to remove myself from this pack 😂)
go.bsky.app/9APVxQQ