Hi! I’m thinking increasingly often during your videos “She would *love* reading ‘the dawn of everything’!” (By David Graeber and David Wengrow). It touches on so many topics from your videos and will blow your mind about history!

Agreed on both counts. Also with brotli and zstd there are two recent, established general-purpose compression algorithms to investigate!

Definitely! Above a certain scale, only out-of-core is still possible. That’s why we started with AnnData’s “backed” mode in 2018 and switched to the established Dask library recently.

I’m unsure if eking out max compression is worth it. Using half the space isn’t that impactful, especially when we‘re mostly talking about disk space, since Dask will load things chunkwise anyway (and needs to decompress to do computation).