@albertge.bsky.social
Online data mixing reduces training costs for foundation models, but faces challenges:
⚠️ Human-defined domains miss semantic nuances
⚠️ Limited eval accessibility
⚠️ Poor scalability
Introducing 🎵R&B: first regroup data, then dynamically reweight domains during training!
⚠️ Human-defined domains miss semantic nuances
⚠️ Limited eval accessibility
⚠️ Poor scalability
Introducing 🎵R&B: first regroup data, then dynamically reweight domains during training!
May 8, 2025 at 5:01 PM
Online data mixing reduces training costs for foundation models, but faces challenges:
⚠️ Human-defined domains miss semantic nuances
⚠️ Limited eval accessibility
⚠️ Poor scalability
Introducing 🎵R&B: first regroup data, then dynamically reweight domains during training!
⚠️ Human-defined domains miss semantic nuances
⚠️ Limited eval accessibility
⚠️ Poor scalability
Introducing 🎵R&B: first regroup data, then dynamically reweight domains during training!
Reposted
First up at #NeurIPS2024 from our group, our work on labeling via programmatic distillation (a spotlight!). Label your data orders of magnitude faster and cheaper — come join us today at Poster Session 2 East for a demo!
December 11, 2024 at 11:15 PM
First up at #NeurIPS2024 from our group, our work on labeling via programmatic distillation (a spotlight!). Label your data orders of magnitude faster and cheaper — come join us today at Poster Session 2 East for a demo!