Zach Mueller
@muellerzr.bsky.social
Technical Lead on Accelerate @ Hugging Face | Passionate about Open Source | https://muellerzr.github.io
DataLoader Dispatching
When constrained by a variety of reasons to where you can't include multiple copies (or mmaps) of datasets in memory, be it too many concurrent streams, low resource availability, or a slow CPU, dispatching is here to help.
When constrained by a variety of reasons to where you can't include multiple copies (or mmaps) of datasets in memory, be it too many concurrent streams, low resource availability, or a slow CPU, dispatching is here to help.
October 5, 2025 at 12:02 PM
DataLoader Dispatching
When constrained by a variety of reasons to where you can't include multiple copies (or mmaps) of datasets in memory, be it too many concurrent streams, low resource availability, or a slow CPU, dispatching is here to help.
When constrained by a variety of reasons to where you can't include multiple copies (or mmaps) of datasets in memory, be it too many concurrent streams, low resource availability, or a slow CPU, dispatching is here to help.
Batch Sampler Sharding
October 3, 2025 at 11:24 AM
Batch Sampler Sharding
Dataset Sharding
When performing distributed data parallelism, we split the dataset every batch so every device sees a different chunk of the data. There are different methods for doing so. One example is sharding at the *dataset* level, shown here.
When performing distributed data parallelism, we split the dataset every batch so every device sees a different chunk of the data. There are different methods for doing so. One example is sharding at the *dataset* level, shown here.
October 2, 2025 at 11:05 AM
Dataset Sharding
When performing distributed data parallelism, we split the dataset every batch so every device sees a different chunk of the data. There are different methods for doing so. One example is sharding at the *dataset* level, shown here.
When performing distributed data parallelism, we split the dataset every batch so every device sees a different chunk of the data. There are different methods for doing so. One example is sharding at the *dataset* level, shown here.
Hi all 👋
Back very briefly to mention I’m working on a new course, and there’s a star-studded set of guest speakers 🎉
From Scratch to Scale: Distributed Training (from the ground up).
From now until I’m done writing the course material, it’s 25% off :)
maven.com/walk-with-co...
Back very briefly to mention I’m working on a new course, and there’s a star-studded set of guest speakers 🎉
From Scratch to Scale: Distributed Training (from the ground up).
From now until I’m done writing the course material, it’s 25% off :)
maven.com/walk-with-co...
June 22, 2025 at 11:18 PM
Hi all 👋
Back very briefly to mention I’m working on a new course, and there’s a star-studded set of guest speakers 🎉
From Scratch to Scale: Distributed Training (from the ground up).
From now until I’m done writing the course material, it’s 25% off :)
maven.com/walk-with-co...
Back very briefly to mention I’m working on a new course, and there’s a star-studded set of guest speakers 🎉
From Scratch to Scale: Distributed Training (from the ground up).
From now until I’m done writing the course material, it’s 25% off :)
maven.com/walk-with-co...
Recently did my first lifting competition, overall quite happy where things ended up (bar bench) and was a ton of fun. Might wait a year and give it another go, but I did get first in the weight class!
145kg squat
180kg deadlift
90kg bench
145kg squat
180kg deadlift
90kg bench
March 2, 2025 at 4:20 PM
Recently did my first lifting competition, overall quite happy where things ended up (bar bench) and was a ton of fun. Might wait a year and give it another go, but I did get first in the weight class!
145kg squat
180kg deadlift
90kg bench
145kg squat
180kg deadlift
90kg bench
Day one of new program: ✅
February 24, 2025 at 1:33 PM
Day one of new program: ✅
Me post seeing SmolLM2 quant is fast enough to run on my tiny laptop
February 10, 2025 at 12:58 AM
Me post seeing SmolLM2 quant is fast enough to run on my tiny laptop
Beautiful, non-ai search results
February 9, 2025 at 4:57 PM
Beautiful, non-ai search results
Or just fully delete it by tweaking the search engine you want to use
(Couldn’t find the original post that had the directions, just this saved screenshot)
(Couldn’t find the original post that had the directions, just this saved screenshot)
February 9, 2025 at 4:56 PM
Or just fully delete it by tweaking the search engine you want to use
(Couldn’t find the original post that had the directions, just this saved screenshot)
(Couldn’t find the original post that had the directions, just this saved screenshot)
Me: huh where’s the cell signal go. Where’d the internet go
Verizon:
I suppose I’m forcibly done with work for the day, too bad I had to go to the next town over to say that 😅
Verizon:
I suppose I’m forcibly done with work for the day, too bad I had to go to the next town over to say that 😅
January 24, 2025 at 8:05 PM
Me: huh where’s the cell signal go. Where’d the internet go
Verizon:
I suppose I’m forcibly done with work for the day, too bad I had to go to the next town over to say that 😅
Verizon:
I suppose I’m forcibly done with work for the day, too bad I had to go to the next town over to say that 😅
If anything, I’m consistent
December 4, 2024 at 2:24 PM
If anything, I’m consistent
I'm looking for an intern!
If you are:
* Driven
* Love OSS
* Interested in distributed PyTorch training/FSDPv2/DeepSpeed
Come work with me!
Fully remote, more details to apply in the comments
If you are:
* Driven
* Love OSS
* Interested in distributed PyTorch training/FSDPv2/DeepSpeed
Come work with me!
Fully remote, more details to apply in the comments
November 26, 2024 at 4:01 PM
I'm looking for an intern!
If you are:
* Driven
* Love OSS
* Interested in distributed PyTorch training/FSDPv2/DeepSpeed
Come work with me!
Fully remote, more details to apply in the comments
If you are:
* Driven
* Love OSS
* Interested in distributed PyTorch training/FSDPv2/DeepSpeed
Come work with me!
Fully remote, more details to apply in the comments
I'm a simple man. I see that Oura has an API, I then try to recreate the app so I don't have to pay $6/mo just to see my data and write custom visualizations
November 17, 2024 at 2:10 PM
I'm a simple man. I see that Oura has an API, I then try to recreate the app so I don't have to pay $6/mo just to see my data and write custom visualizations
Another lovely Hugging Face offsite comes to a close 🥹😭
November 13, 2024 at 4:45 PM
Another lovely Hugging Face offsite comes to a close 🥹😭
Coworkers who dive together stay together 🫡🤗
November 11, 2024 at 4:32 PM
Coworkers who dive together stay together 🫡🤗
The pretty graph I’m most proud of:
October 29, 2024 at 4:09 PM
The pretty graph I’m most proud of:
*Gets a bunch of github notifications*
Huh. Weird. I wonder what's u-
Oh. Cool ☺️
Huh. Weird. I wonder what's u-
Oh. Cool ☺️
July 13, 2023 at 6:50 PM
*Gets a bunch of github notifications*
Huh. Weird. I wonder what's u-
Oh. Cool ☺️
Huh. Weird. I wonder what's u-
Oh. Cool ☺️
Whelp guess I'm back here back here. Someone mentioned I've been a recluse the last few months (fair) so prob expect some madness now
(Such as oooo, pretty flowers from yesterday)
(Such as oooo, pretty flowers from yesterday)
July 1, 2023 at 5:37 PM
Whelp guess I'm back here back here. Someone mentioned I've been a recluse the last few months (fair) so prob expect some madness now
(Such as oooo, pretty flowers from yesterday)
(Such as oooo, pretty flowers from yesterday)
So, how's your weekend going?
June 4, 2023 at 2:04 AM
So, how's your weekend going?
Summer time, plant time 🥰😍🌞
May 27, 2023 at 6:20 PM
Summer time, plant time 🥰😍🌞
Blink-182 tonight with the cousin 🤩
May 27, 2023 at 12:28 AM
Blink-182 tonight with the cousin 🤩