Tzu-Heng (Brian) Huang
@zihengh1.bsky.social
CS Ph.D. Student @UWMadison. Research Intern @Apple AIML. Focusing on multimodal models, data curation, and data-centric AI.
zihengh1.github.io
zihengh1.github.io
This was my intern work Apple! Huge thanks to Manjot Bilkhu, my advisor @fredsala.bsky.social, and Javier Movellan! Check out the full paper for all the details:
arxiv.org/abs/2501.06708.
If you found this interesting, feel free to spread the word!
arxiv.org/abs/2501.06708.
If you found this interesting, feel free to spread the word!
Evaluating Sample Utility for Data Selection by Mimicking Model Weights
Foundation models are trained on large-scale web-crawled datasets, which often contain noise, biases, and irrelevant information. This motivates the use of data selection techniques, which can be divi...
arxiv.org
February 9, 2025 at 9:08 PM
This was my intern work Apple! Huge thanks to Manjot Bilkhu, my advisor @fredsala.bsky.social, and Javier Movellan! Check out the full paper for all the details:
arxiv.org/abs/2501.06708.
If you found this interesting, feel free to spread the word!
arxiv.org/abs/2501.06708.
If you found this interesting, feel free to spread the word!
Our results are exciting! 🎉
✔️ Pretrained model weights are reliable guides for data selection.
✔️ Grad-Mimic identifies noisy samples and estimates training dataset quality.
✔️ It even complements other filtering methods, boosting CLIP performance with less data!
✔️ Pretrained model weights are reliable guides for data selection.
✔️ Grad-Mimic identifies noisy samples and estimates training dataset quality.
✔️ It even complements other filtering methods, boosting CLIP performance with less data!
February 9, 2025 at 9:08 PM
Our results are exciting! 🎉
✔️ Pretrained model weights are reliable guides for data selection.
✔️ Grad-Mimic identifies noisy samples and estimates training dataset quality.
✔️ It even complements other filtering methods, boosting CLIP performance with less data!
✔️ Pretrained model weights are reliable guides for data selection.
✔️ Grad-Mimic identifies noisy samples and estimates training dataset quality.
✔️ It even complements other filtering methods, boosting CLIP performance with less data!
Mimic Score helps identify samples that can misguide weight updates. We can automatically filter these out, improving training. Here is an identified example using noisy web datasets!
February 9, 2025 at 9:08 PM
Mimic Score helps identify samples that can misguide weight updates. We can automatically filter these out, improving training. Here is an identified example using noisy web datasets!
🛠️ Using Mimic Score, we develop Grad-Mimic, a two-stage framework:
1️⃣ Training Phase: Prioritizes which samples to learn, boosting data efficiency.
2️⃣ Post-Training Phase: Evaluates sample utility across training steps, creating an ensemble filter using weak supervision.
1️⃣ Training Phase: Prioritizes which samples to learn, boosting data efficiency.
2️⃣ Post-Training Phase: Evaluates sample utility across training steps, creating an ensemble filter using weak supervision.
February 9, 2025 at 9:08 PM
🛠️ Using Mimic Score, we develop Grad-Mimic, a two-stage framework:
1️⃣ Training Phase: Prioritizes which samples to learn, boosting data efficiency.
2️⃣ Post-Training Phase: Evaluates sample utility across training steps, creating an ensemble filter using weak supervision.
1️⃣ Training Phase: Prioritizes which samples to learn, boosting data efficiency.
2️⃣ Post-Training Phase: Evaluates sample utility across training steps, creating an ensemble filter using weak supervision.
📣 We propose the Mimic Score: a new data quality metric. It leverages reference model weights to assess sample utility, relying on the alignment between gradients and a target direction induced by the reference model.
February 9, 2025 at 9:08 PM
📣 We propose the Mimic Score: a new data quality metric. It leverages reference model weights to assess sample utility, relying on the alignment between gradients and a target direction induced by the reference model.
Data selection is crucial!
⚠️ Existing methods have limitations: Model-free approaches are hard to design, while model-based ones can be computationally expensive.
✅ Grad-Mimic offers a better way using existing model weights!
arxiv.org/abs/2501.06708
⚠️ Existing methods have limitations: Model-free approaches are hard to design, while model-based ones can be computationally expensive.
✅ Grad-Mimic offers a better way using existing model weights!
arxiv.org/abs/2501.06708
Evaluating Sample Utility for Data Selection by Mimicking Model Weights
Foundation models are trained on large-scale web-crawled datasets, which often contain noise, biases, and irrelevant information. This motivates the use of data selection techniques, which can be divi...
arxiv.org
February 9, 2025 at 9:08 PM
Data selection is crucial!
⚠️ Existing methods have limitations: Model-free approaches are hard to design, while model-based ones can be computationally expensive.
✅ Grad-Mimic offers a better way using existing model weights!
arxiv.org/abs/2501.06708
⚠️ Existing methods have limitations: Model-free approaches are hard to design, while model-based ones can be computationally expensive.
✅ Grad-Mimic offers a better way using existing model weights!
arxiv.org/abs/2501.06708