Max Gabrielsson
@maxxen.bsky.social
Software Engineer @ DuckDB Labs
Ive touched on this a couple times, but in short I think it would be very hard to put raster processing into duckdb and maintain acceptable performance/memory usage. Huge blobs (rasters) and vectorized execution does not go well together. E.g. a single duckdb vector with 512x512x4 tiles is 2GB RAM.
February 12, 2025 at 10:04 PM
Ive touched on this a couple times, but in short I think it would be very hard to put raster processing into duckdb and maintain acceptable performance/memory usage. Huge blobs (rasters) and vectorized execution does not go well together. E.g. a single duckdb vector with 512x512x4 tiles is 2GB RAM.
As for the linux issues, thats a more general problem with all sorts of binary distributions on linux, which is why we're following pythons footsteps and using the many-linux system images when compiling. There's more details e.g. here peps.python.org/pep-0513/
PEP 513 – A Platform Tag for Portable Linux Built Distributions | peps.python.org
This PEP proposes the creation of a new platform tag for Python package built distributions, such as wheels, called manylinux1_{x86_64,i686} with external dependencies limited to a standardized, restr...
peps.python.org
January 20, 2025 at 12:45 PM
As for the linux issues, thats a more general problem with all sorts of binary distributions on linux, which is why we're following pythons footsteps and using the many-linux system images when compiling. There's more details e.g. here peps.python.org/pep-0513/
AFAIK the extension side of the C-API doesn't require any c++ code to function, you could theoretically just program against the header without any linking with all the api callbacks passed as function pointers at runtime. That's why we're generally trying to (slowly) invest more in the C-API.
January 20, 2025 at 12:42 PM
AFAIK the extension side of the C-API doesn't require any c++ code to function, you could theoretically just program against the header without any linking with all the api callbacks passed as function pointers at runtime. That's why we're generally trying to (slowly) invest more in the C-API.
Reposted by Max Gabrielsson
Full excel support also dropped in DuckDB 3 days ago (not through GDAL) 👀
Add support for reading/writing `.xlsx` files by Maxxen · Pull Request #3 · duckdb/duckdb_excel
This PR adds support for reading and writing .xlsx files through COPY TO/FROM ... FORMAT (XLSX), a new read_xlsx() table function as well as a .xlsx file replacement scan.
The XLSX file parsing is ...
github.com
December 9, 2024 at 2:43 PM
Full excel support also dropped in DuckDB 3 days ago (not through GDAL) 👀