Michael Hu
@michahu.bsky.social
PhD student at NYU. NLP & training data.
michahu.github.io
michahu.github.io
Boavista album by Stephan Bodzin:
open.spotify.com/track/7ujvbI...
open.spotify.com/track/7ujvbI...
Nothing Like You
Stephan Bodzin, Luna Semara · Boavista · Song · 2021
open.spotify.com
November 26, 2024 at 1:29 AM
Boavista album by Stephan Bodzin:
open.spotify.com/track/7ujvbI...
open.spotify.com/track/7ujvbI...
Is this #1 in your Spotify wrapped 😆
November 26, 2024 at 1:14 AM
Is this #1 in your Spotify wrapped 😆
thanks for featuring this work!
November 19, 2024 at 2:04 AM
thanks for featuring this work!
In joint work with @MayeeChen @NickLourie @kchonyc @HazyResearch, we use our optimization framework to analyze failures of existing methods. We then turn these insights into:
Aioli 🧄, a fully-online data mixing algorithm!
paper: arxiv.org/abs/2411.05735
code: github.com/HazyResearch...
Aioli 🧄, a fully-online data mixing algorithm!
paper: arxiv.org/abs/2411.05735
code: github.com/HazyResearch...
Aioli: A Unified Optimization Framework for Language Model Data Mixing
Language model performance depends on identifying the optimal mixture of data groups to train on (e.g., law, code, math). Prior work has proposed a diverse set of methods to efficiently learn mixture ...
arxiv.org
November 12, 2024 at 5:04 PM
In joint work with @MayeeChen @NickLourie @kchonyc @HazyResearch, we use our optimization framework to analyze failures of existing methods. We then turn these insights into:
Aioli 🧄, a fully-online data mixing algorithm!
paper: arxiv.org/abs/2411.05735
code: github.com/HazyResearch...
Aioli 🧄, a fully-online data mixing algorithm!
paper: arxiv.org/abs/2411.05735
code: github.com/HazyResearch...
metropolis-hastings:
1️⃣ sample from your proposal function
2️⃣ run the sample through your filter, proportional to the desired pdf
3️⃣ use the kept samples to initialize the next round
i wonder if we can connect iterative approaches to synthetic data as making specific choices in an MCMC framework...
1️⃣ sample from your proposal function
2️⃣ run the sample through your filter, proportional to the desired pdf
3️⃣ use the kept samples to initialize the next round
i wonder if we can connect iterative approaches to synthetic data as making specific choices in an MCMC framework...
November 10, 2024 at 2:24 AM
metropolis-hastings:
1️⃣ sample from your proposal function
2️⃣ run the sample through your filter, proportional to the desired pdf
3️⃣ use the kept samples to initialize the next round
i wonder if we can connect iterative approaches to synthetic data as making specific choices in an MCMC framework...
1️⃣ sample from your proposal function
2️⃣ run the sample through your filter, proportional to the desired pdf
3️⃣ use the kept samples to initialize the next round
i wonder if we can connect iterative approaches to synthetic data as making specific choices in an MCMC framework...