https://blosc.org
www.youtube.com/watch?v=BdpT... #DataScience #BigData #HPC #OpenSource
www.youtube.com/watch?v=BdpT... #DataScience #BigData #HPC #OpenSource
Blosc2 is part of the NumPy array library ecosystem: numpy.org
In v3.11, Blosc2 has become even more flexible; all functions can accept basically any array object:
𝚋𝚕𝚘𝚜𝚌𝟸.𝚖𝚊𝚝𝚖𝚞𝚕(𝙰, 𝙱)
for 𝙰, 𝙱 PyTorch, Jax, Tensorflow, Zarr... arrays!
#Compute
#Tensor
#PyTorch
See how Blosc2 powers heavy-duty linear algebra (100GB!) workflows
⚡1.5-2x faster than PyTorch + h5py!
🧱 optimised chunking for your cache hierarchy
🐍 one line syntax 𝚋𝚕𝚘𝚜𝚌𝟸.𝚖𝚊𝚝𝚖𝚞𝚕(𝙰, 𝙱, 𝚞𝚛𝚕𝚙𝚊𝚝𝚑='𝚘𝚞𝚝.𝚋𝟸𝚗𝚍')
See blog here: ironarray.io/blog/la-blosc
#Blosc2
#Data
#LinearAlgebra
See how Blosc2 powers heavy-duty linear algebra (100GB!) workflows
⚡1.5-2x faster than PyTorch + h5py!
🧱 optimised chunking for your cache hierarchy
🐍 one line syntax 𝚋𝚕𝚘𝚜𝚌𝟸.𝚖𝚊𝚝𝚖𝚞𝚕(𝙰, 𝙱, 𝚞𝚛𝚕𝚙𝚊𝚝𝚑='𝚘𝚞𝚝.𝚋𝟸𝚗𝚍')
See blog here: ironarray.io/blog/la-blosc
#Blosc2
#Data
#LinearAlgebra
Check out our new video 🎬 to see how #Blosc2 eliminates these temporaries⚡
Compress Better, Compute Larger
Blosc2 calculates Fourier approximations:
⚡5x faster than NumPy
👣 fraction of the memory footprint
🐍 one-line syntax 𝚜𝚞𝚖(𝚊 * 𝚌𝚘𝚜(𝚝) + 𝚋 * 𝚜𝚒𝚗(𝚝), 𝚊𝚡𝚒𝚜=𝟷)
Full notebook: github.com/Blosc/python...
(inspiration: towardsdatascience.com/numexpr-the-...)
#Blosc2
#Data
#SignalProcessing
Check out our new video 🎬 to see how #Blosc2 eliminates these temporaries⚡
Compress Better, Compute Larger
Lazy evaluation for ALL Blosc2 functions in version 3.10!
Blosc2 powers:
- Zarr/HDF5 computations (see image and blosc.org/posts/tensor...)
- Server compuation via Cat2Cloud (demo site here cat2.cloud/demo/)!
#Zarr
#HDF5
#Python
Lazy evaluation for ALL Blosc2 functions in version 3.10!
Blosc2 powers:
- Zarr/HDF5 computations (see image and blosc.org/posts/tensor...)
- Server compuation via Cat2Cloud (demo site here cat2.cloud/demo/)!
#Zarr
#HDF5
#Python
At SciPy 2023, we showcased visualizing a massive 7.3 TB Gaia dataset on a standard laptop by compressing it to 1.8 GB using #Blosc2. Big data analysis for everyone! 🚀
Dataset: digistorage.es/links/652403...
Paper: www.blosc.org/docs/Explori...
#napari
Blosc2 as a mature computation engine >>>
• Array API standard compliance - 70 new functions
• Fast linalg: tensordot 12x 𝐅𝐀𝐒𝐓𝐄𝐑 than NumPy for large arrays !
• Fast calc: Funcs like hypot 10x 𝐅𝐀𝐒𝐓𝐄𝐑 than NumPy!
Blog: ironarray.io/blog/array-api
#Python
#linalg
#computation
For ultimate speed, native Blosc2 containers are still king. 🏎️
Code: gist.github.com/FrancescAlte...
#Python #DataScience #NumPy #Performance #Blosc
For ultimate speed, native Blosc2 containers are still king. 🏎️
Code: gist.github.com/FrancescAlte...
#Python #DataScience #NumPy #Performance #Blosc
A step closer to compliance with the array-api standard: data-apis.org/array-api!
This is an effort across all array-based libraries so that your code works (e.g. for both blosc2 and NumPy) by simply changing the import statement below!
#arrayapi #Blosc2
A step closer to compliance with the array-api standard: data-apis.org/array-api!
This is an effort across all array-based libraries so that your code works (e.g. for both blosc2 and NumPy) by simply changing the import statement below!
#arrayapi #Blosc2
This is a maintenance release, with a few fixes and some optimizations. Thanks to Barak Ugav and Preeyan Parmar.
For more info, see the release notes in:
github.com/Blosc/c-blos...
Compress Better, Compute Bigger
Enjoy!
This is a maintenance release, with a few fixes and some optimizations. Thanks to Barak Ugav and Preeyan Parmar.
For more info, see the release notes in:
github.com/Blosc/c-blos...
Compress Better, Compute Bigger
Enjoy!
We can help! 💪 We've spent countless hours profiling and micro-benchmarking Blosc2 for you. A thing that LLMs cannot do (yet).
Ready to dive in? 🚀
🔹 EuroSciPy Talk: www.blosc.org/docs/2025-Eu...
🔹 Tutorial: github.com/Blosc/EuroSc...
#HPC #Blosc2
We can help! 💪 We've spent countless hours profiling and micro-benchmarking Blosc2 for you. A thing that LLMs cannot do (yet).
Ready to dive in? 🚀
🔹 EuroSciPy Talk: www.blosc.org/docs/2025-Eu...
🔹 Tutorial: github.com/Blosc/EuroSc...
#HPC #Blosc2
Check out compress-image: a new C++/Python library for working with compressed images directly in memory.
Kudos to Emil Dohne for this fantastic work!
Project here: github.com/EmilDohne/co...
#Blosc2 #FOSS #ImageProcessing
Check out compress-image: a new C++/Python library for working with compressed images directly in memory.
Kudos to Emil Dohne for this fantastic work!
Project here: github.com/EmilDohne/co...
#Blosc2 #FOSS #ImageProcessing
www.blosc.org/docs/2025-Eu...
EuroSciPy 2025 was great! A human sized event with lots of interesting talks and people ready to help you. Thanks to the organizers for such a fantastic event!
www.blosc.org/docs/2025-Eu...
EuroSciPy 2025 was great! A human sized event with lots of interesting talks and people ready to help you. Thanks to the organizers for such a fantastic event!
📝 We've blogged about it:
www.blosc.org/posts/new-tr...
It's in beta, and available in Python-Blosc2 v3.7.2. Enjoy!
#Python #Blosc2 #TreeStore #DataScience #OpenSource
📝 We've blogged about it:
www.blosc.org/posts/new-tr...
It's in beta, and available in Python-Blosc2 v3.7.2. Enjoy!
#Python #Blosc2 #TreeStore #DataScience #OpenSource
But you can always bypass this mechanism for more appropriate fine-tuning in different scenarios 🚀
www.blosc.org/python-blosc...
#Compression #Performance
But you can always bypass this mechanism for more appropriate fine-tuning in different scenarios 🚀
www.blosc.org/python-blosc...
#Compression #Performance
¡Unlock new levels of data manipulation with #Blosc2! 🚀
We've tamed the complexity of fancy indexing to make it intuitive, efficient, and consistent with NumPy's behavior.
👉 www.blosc.org/posts/blosc2...
Enjoy
#Python #DataScience #BigData #NumPy #Performance
¡Unlock new levels of data manipulation with #Blosc2! 🚀
We've tamed the complexity of fancy indexing to make it intuitive, efficient, and consistent with NumPy's behavior.
👉 www.blosc.org/posts/blosc2...
Enjoy
#Python #DataScience #BigData #NumPy #Performance
✨ Why memory-mapping?
1️⃣ No system call overhead for each read/write
2️⃣ Data goes straight from page cache to user space—much faster than traditional I/O!
👉 github.com/Blosc/python...
#DataScience #Performance #BigData 🚀💾
✨ Why memory-mapping?
1️⃣ No system call overhead for each read/write
2️⃣ Data goes straight from page cache to user space—much faster than traditional I/O!
👉 github.com/Blosc/python...
#DataScience #Performance #BigData 🚀💾
Compress Better, Compute Bigger, Share Faster
#WebAssembly #DataCompression #WebDevelopment #DataScience
Compress Better, Compute Bigger, Share Faster
#WebAssembly #DataCompression #WebDevelopment #DataScience
This introduces significant performance and memory optimizations, enhancing the experience of computing with large, compressed datasets using lazy expressions.
Compress Better, Compute Bigger!
#Performance #MemoryEfficiency #DataScience
This introduces significant performance and memory optimizations, enhancing the experience of computing with large, compressed datasets using lazy expressions.
Compress Better, Compute Bigger!
#Performance #MemoryEfficiency #DataScience
👉 www.blosc.org/posts/pytabl...
#Performance #DataScience
👉 www.blosc.org/posts/pytabl...
#Performance #DataScience
This release introduces the powerful stack() function! 🥞 You can now stack multiple Blosc2 NDarrays along a new axis, similar to np.stack(), while getting great performance.
We have blogged about it here: www.blosc.org/posts/blosc2...
Compress Better, Compute Bigger
This release introduces the powerful stack() function! 🥞 You can now stack multiple Blosc2 NDarrays along a new axis, similar to np.stack(), while getting great performance.
We have blogged about it here: www.blosc.org/posts/blosc2...
Compress Better, Compute Bigger
We've updated our recent blog post:
Check it out! 👇
www.blosc.org/posts/blosc2...
Compress Better, Compute Bigger
#Python #DataScience #Performance #OpenSource
We've updated our recent blog post:
Check it out! 👇
www.blosc.org/posts/blosc2...
Compress Better, Compute Bigger
#Python #DataScience #Performance #OpenSource
Users are rapidly adopting #Blosc2, which now accounts for over 95% of downloads compared to Blosc1. 📈 Thanks to our amazing users. 🙏 🚀
Our motto: Compress Better, Compute Bigger! 💪
#Milestone #CommunitySupport #DataCompression
Users are rapidly adopting #Blosc2, which now accounts for over 95% of downloads compared to Blosc1. 📈 Thanks to our amazing users. 🙏 🚀
Our motto: Compress Better, Compute Bigger! 💪
#Milestone #CommunitySupport #DataCompression
Slides:
www.blosc.org/docs/2025-Eu...
Slides:
www.blosc.org/docs/2025-Eu...