Lightnews — Scholar-powered news

DuckDB

@duckdb.org

DuckLake now supports data inlining with PostgreSQL and SQLite catalogs! You can avoid the small files problem in your lakehouse with any DuckLake supported catalog.

Update your DuckLake extension in any client on DuckDB 1.4.x to use this feature.

P.S. Don't forget to flush!

December 12, 2025 at 12:23 PM

Tobias Müller

@tobilg.com

I think the issue that DuckLake "only" supports for local multi-user support will remain, as they expect SQLite to be accessible from a file path as far as I understood... This wouldn't help regarding D1.

I'd really love if they had a serverless Postgres service as well though.

June 1, 2025 at 12:21 PM

Jordan Tigani

@jrdntgn.bsky.social

A DuckLake reader for PySpark in ~30 lines of code. Despite looking simple, still partitions work across Spark nodes. Zero external dependencies, just the DuckDB JDBC driver. Try that with your favorite lakehouse technology :-)

Hannes Mühleisen @hannes.muehleisen.org · Jul 1

Want to read #ducklake from APACHE SPARK? Check this out: gist.github.com/hannes/395ac... #butdoesitscale #yesitdoes

pyspark-ducklake-2.py

GitHub Gist: instantly share code, notes, and snippets.

gist.github.com

July 2, 2025 at 7:20 PM

Marshal of the Riddermark

@edorasmarshal.bsky.social

Und für den modernen Datalake einfach Hetzner S3 + Ducklake 👌

November 1, 2025 at 8:40 AM

dhirschfeld.bsky.social

@dhirschfeld.bsky.social

An interesting writeup on how ducklake compares to existing lakehouse table formats:

www.pracdata.io/p/is-ducklak...

Is DuckLake a Step Backward?

Examining the new open table format’s return to relational metadata management

www.pracdata.io

December 18, 2025 at 12:40 AM

Tigris Data

@tigrisdata.com

Time travel! Ducklake and Tigris let you travel through time to view your analytics data as it existed in the past so you can fork the timeline and create a better future.

Read more on our blog: www.tigrisdata.com/blog/ducklak...

Data Time Travel with DuckLake and Tigris | Tigris Object Storage

Learn how to implement data time travel with Ducklake and Tigris to easily recover from database mishaps. This guide demonstrates how Ducklake creates snapshots for every `INSERT` or `DELETE`…

www.tigrisdata.com

June 25, 2025 at 8:01 PM

MotherDuck

@motherduck.com

MotherDuck now supports DuckLake! 🦆

New open table format for extreme scale with database-backed metadata.

Two options:
🚀 Fully managed
☁️ Bring your own bucket

Ready to dive in (the lake 🥁)?

CREATE DATABASE (TYPE ducklake);

motherduck.com/blog/announc...

MotherDuck Managed DuckLakes Now in Preview: Scale to Petabytes - MotherDuck Blog

Preview support of MotherDuck includes both fully-managed DuckLake support and ability to bring your own bucket. Combined with MotherDuck's storage, you get both high-speed access to recent data and s...

motherduck.com

July 1, 2025 at 9:39 AM

__Alex Monahan__

@a13x.bsky.social

Meet the elegant, modern evolution of the Data Lakehouse - DuckLake!
Store data in files on object storage and metadata in ~any SQL database! Full ACID transactions across tables and far faster and easier!

DuckDB @duckdb.org · May 27

Today we're launching DuckLake, an integrated data lake and catalog format powered by SQL. DuckLake unlocks next-generation data warehousing where compute is local, consistency central, and storage scales till infinity. ⁠ducklake is an open standard and we implemented it in the "ducklake" extension.

May 27, 2025 at 1:59 PM

hrbrmstr 🇺🇦 🇬🇱 🇨🇦 🏳️‍🌈

@hrbrmstr.dev

More magicks, this time for DuckDB's new DuckLake!

Use local or SharePoint-ed Excel as a DuckLake metadata catalog.

github.com/gregwdata/du...

GitHub - gregwdata/ducklakexl: Use Excel as a metadata catalog for DuckLake 🤪

Use Excel as a metadata catalog for DuckLake 🤪. Contribute to gregwdata/ducklakexl development by creating an account on GitHub.

github.com

June 4, 2025 at 1:41 PM

ttunguz.bsky.social

@ttunguz.bsky.social

By returning to first principles, DuckLake delivers the power of a modern data lake without the complexity. Its simplicity and performance make it a vital part of the future of data.

tomtunguz.com/ducklake/

DuckLake - Subsecond Latency on a Petabyte by @ttunguz

DuckLake is one of the most exciting technologies in data. While data lakes are powerful, the formats that manage them have become notoriously difficult to work with. “I think one of the things in Duc...

tomtunguz.com

June 18, 2025 at 9:32 PM

【非公式】増田bot

@masuda-bot.jiskey.dev.ap.brid.gy

新しい記事が投稿されました

DuckDBでR2 Data Catalog & DuckLakeを試す(with Neon)
DuckDBでR2 Data Catalog &am...
https://anond.hatelabo.jp/20250726021025

DuckDBでR2 Data Catalog & DuckLakeを試す(with Neon)

DuckDBでR2DataCatalog&DuckLakeを試す(withNeon)まずはIcebergを扱えるR2DataCatalogを使う(DWHとして使う)次はDuckLakeを試すR2はストレージとして扱いメタデータは…

anond.hatelabo.jp

July 25, 2025 at 5:12 PM

Paul Watson 🌍

@paulmwatson.com

Put together a @duckdb.org DuckLake SQL lakehouse example with a 2 million row CSV. github.com/paulmwatson/...

Screenshot of DuckDB UI with SQL and table results.

June 6, 2025 at 9:14 PM

Matthew Mullins

@mmullins.coginiti.co

AWS Glue is based on HMS and most users never think about Hive. Someone else will implement the ducklake metastore and rebrand it too. Also, think we should call it a metastore and not an otf

June 7, 2025 at 7:20 PM

Delta Lake

@deltalakeoss.bsky.social

What’s next?
🔹 October 30: Webinar — The Failed Promises of Data Lineage: Why More Metadata Isn’t the Answer
🔹 November 13: Mini Summit — Mountain View
🔹 November 24: Meetup — Paris

🔗 RSVP and details: luma.com/openlakehouse

#opensource #oss #openlakehouse #ai #ducklake #duckdb

Open Lakehouse + AI · Events Calendar

View and subscribe to events from Open Lakehouse + AI on Luma. Open Lakehouse + AI is a global community advancing open lakehouse and AI through adoption, sharing real-world use cases, and collaborati...

luma.com

October 15, 2025 at 7:13 PM

デイリーZennトレンド

@dailyzenntrends.bsky.social

今日のZennトレンド

pg_duckdbとDuckLakeがもたらすOLAP統合の未来
この記事は、PostgreSQLと分析処理（OLAP）の統合が今後の潮流だと解説する。
PostgreSQLの拡張機能pg_duckdbは、DuckDBを統合し、データレイク上のファイルから直接高速分析できるHTAPを実現。
しかし、pg_duckdbには高度なデータ管理に課題がある。
これを補完するため、SQLエンジンにDuckDB、カタログデータベースにPostgreSQL、データファイルにオブジェクトストレージを用いるDuckLakeが登場。

pg_duckdbとDuckLakeがもたらすOLAP統合の未来

注目を集めるPostgreSQL＋Analytics先日、SnowflakeとDatabricksのそれぞれの年次イベントでPostgreSQLに関連する企業の買収が大々的に発表されました。両社は分析系（OLAP）のソリューションを提供する比較的大きなベンダーであり、過去にはOLTP系への進出を目指したデータストアの開発が注目されたこともありました（SnowflakeのUnistoreが典型です

zenn.dev

June 28, 2025 at 9:21 PM

Coba

@cobalente.bsky.social

"ducklake is not a replacement for iceberg/delta, but for the whole lakehouse stack" (catalog, storage, [compute])

and transactionality is provided at the catalog level, not at the table level

May 31, 2025 at 6:04 PM

Ananth Packkildurai

@ananthdurai.bsky.social

How Tables Grew a Brain: Iceberg → DuckLake
Snapshots → incremental → stream-native → catalog-first.
Metadata is the bottleneck.

More insights → www.dataengineeringw...

Full story → medium.com/fresha-da...

September 25, 2025 at 12:33 PM

Estuary

@estuarydev.bsky.social

💡 Did you know? You can now stream data directly into #DuckLake in real-time!

This webinar walks you through the entire process. It's time to say goodbye to clunky, slow data pipelines and hello to lightning-fast insights!

⚡ https://www.youtube.com/watch?v=8uce9V9VnjY