Craig
@craigkerstiens.com
Product @crunchydata previously MSFT/Citus/Heroku. Talk a lot about Postgres and startups. Why Postgres? https://www.crunchydata.com/why-postgres
Five years ago I joined @crunchydata.com, shortly after I wrote about having unfinished business with Postgres. Today as part of Snowflake that journey is continuing. We've built some amazing things, but are just getting started.
www.crunchydata.com/blog/crunchy...
www.crunchydata.com/blog/crunchy...
Crunchy Data Joins Snowflake | Crunchy Data Blog
We are excited to announce that Crunchy Data is joining Snowflake to bring Postgres to the AI Data Cloud.
www.crunchydata.com
June 2, 2025 at 8:44 PM
Five years ago I joined @crunchydata.com, shortly after I wrote about having unfinished business with Postgres. Today as part of Snowflake that journey is continuing. We've built some amazing things, but are just getting started.
www.crunchydata.com/blog/crunchy...
www.crunchydata.com/blog/crunchy...
We've got food duty at the first kids travel tournament of the year, was thinking about trying to pull off good quality pour over coffee. Anyone have good mechanisms when unclear a good power source for kettle exists?
May 23, 2025 at 2:48 PM
We've got food duty at the first kids travel tournament of the year, was thinking about trying to pull off good quality pour over coffee. Anyone have good mechanisms when unclear a good power source for kettle exists?
Way back at Heroku when were having a company onsite... we took the entire company to the SF MOMA for a Dieter Rams exhibit.
It wasn't just the designers focused on design it was everyone.
It wasn't just the designers focused on design it was everyone.
May 19, 2025 at 5:15 PM
Way back at Heroku when were having a company onsite... we took the entire company to the SF MOMA for a Dieter Rams exhibit.
It wasn't just the designers focused on design it was everyone.
It wasn't just the designers focused on design it was everyone.
Production ready Postgres.
That's it.
That's it.
May 19, 2025 at 3:59 PM
Production ready Postgres.
That's it.
That's it.
The details matter. In this case, completely revamping our newsletter signup screen ahead of an upcoming conference.
I mean why wouldn't you execute SQL to sign up for a database newsletter?
I mean why wouldn't you execute SQL to sign up for a database newsletter?
May 7, 2025 at 5:52 PM
The details matter. In this case, completely revamping our newsletter signup screen ahead of an upcoming conference.
I mean why wouldn't you execute SQL to sign up for a database newsletter?
I mean why wouldn't you execute SQL to sign up for a database newsletter?
New release of pg_parquet including:
* Amazon S3
* Azure Blob Storage
* Google Cloud Storage
* http(s) stores
* local files
Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together.
www.crunchydata.com/blog/announc...
* Amazon S3
* Azure Blob Storage
* Google Cloud Storage
* http(s) stores
* local files
Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together.
www.crunchydata.com/blog/announc...
Announcing pg_parquet v.0.4.0: Google Cloud Storage, https storage, and more | Crunchy Data Blog
pg_parquet is a copy/to from for Postgres and Parquet. We're excited to announce integration with Google Cloud storage, https, and additional formats.
www.crunchydata.com
May 7, 2025 at 3:36 PM
New release of pg_parquet including:
* Amazon S3
* Azure Blob Storage
* Google Cloud Storage
* http(s) stores
* local files
Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together.
www.crunchydata.com/blog/announc...
* Amazon S3
* Azure Blob Storage
* Google Cloud Storage
* http(s) stores
* local files
Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together.
www.crunchydata.com/blog/announc...
Reposted by Craig
Iceberg has gone from being the thing Netflix (and then Apple) built for their own enormous data lakes to a collaborative open standard where even the competition is learning to co-operate and align: I dug into the Iceberg summit and asked @craigkerstiens.com and others why it's so useful
The Iceberg revolution: A catalyst for data transformation?
"It took two commands, and it saved us $30,000 a month on our cloud bill"
www.thestack.technology
May 6, 2025 at 7:45 PM
Iceberg has gone from being the thing Netflix (and then Apple) built for their own enormous data lakes to a collaborative open standard where even the competition is learning to co-operate and align: I dug into the Iceberg summit and asked @craigkerstiens.com and others why it's so useful
Happy revenge of the sith day to all who celebrate.
May 5, 2025 at 3:27 PM
Happy revenge of the sith day to all who celebrate.
Updating my nulls today with:
\pset null 🤖
Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days...
\pset null 🤖
Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days...
May 4, 2025 at 4:11 PM
Updating my nulls today with:
\pset null 🤖
Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days...
\pset null 🤖
Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days...
A few weeks in, but still every time I demo this it feels absolutely magical. Finally convergence of transactional and analytical data. www.crunchydata.com/blog/logical...
Logical replication from Postgres to Iceberg | Crunchy Data Blog
We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.
www.crunchydata.com
May 2, 2025 at 11:56 PM
A few weeks in, but still every time I demo this it feels absolutely magical. Finally convergence of transactional and analytical data. www.crunchydata.com/blog/logical...
Now live on the Crunchy Bridge dashboard the ability to seamlessly sync data between your operational database over to your data warehouse for analytics - docs.crunchybridge.com/changelog#da...
Changelog
docs.crunchybridge.com
April 28, 2025 at 4:49 PM
Now live on the Crunchy Bridge dashboard the ability to seamlessly sync data between your operational database over to your data warehouse for analytics - docs.crunchybridge.com/changelog#da...
Reposted by Craig
My guess would be people are just using Iceberg connectors, see things seem to work, and compaction is an after-thought. That's why IMO good Iceberg support is not a connector feature but an engine feature. Like what Crunchy is doing with their DWH, or, for Kafka, Confluent with Tableflow.
April 23, 2025 at 7:56 PM
My guess would be people are just using Iceberg connectors, see things seem to work, and compaction is an after-thought. That's why IMO good Iceberg support is not a connector feature but an engine feature. Like what Crunchy is doing with their DWH, or, for Kafka, Confluent with Tableflow.
A shocking take-away for me from a few weeks ago at Iceberg Summit and similarly at Data Council today is for all those using Iceberg yet so few doing compaction on their data lake files.
To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months.
To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months.
April 23, 2025 at 5:55 PM
A shocking take-away for me from a few weeks ago at Iceberg Summit and similarly at Data Council today is for all those using Iceberg yet so few doing compaction on their data lake files.
To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months.
To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months.
Yesterday met someone at Data Council that was very familiar with @crunchydata.com team and described us as basically being team Avengers for Postgres/databases.
That's a new one, but will totally take it.
That's a new one, but will totally take it.
April 23, 2025 at 3:39 PM
Yesterday met someone at Data Council that was very familiar with @crunchydata.com team and described us as basically being team Avengers for Postgres/databases.
That's a new one, but will totally take it.
That's a new one, but will totally take it.
In talking with customers that were using CDC tools to get data from Postgres into a data warehouse, 80% of their spend on ETL was the Postgres data movement.
Further many of these tools often caused production outages.
We knew as soon as we launched Crunchy Data Warehouse we had to solve that.
Further many of these tools often caused production outages.
We knew as soon as we launched Crunchy Data Warehouse we had to solve that.
April 23, 2025 at 2:56 PM
In talking with customers that were using CDC tools to get data from Postgres into a data warehouse, 80% of their spend on ETL was the Postgres data movement.
Further many of these tools often caused production outages.
We knew as soon as we launched Crunchy Data Warehouse we had to solve that.
Further many of these tools often caused production outages.
We knew as soon as we launched Crunchy Data Warehouse we had to solve that.
Sitting in Ryan Blue’s talk at Data Council and about to leave from Q&A to head to office hours and question comes up about CDC from databases to Iceberg… @marcoslot.com makes a hard U-turn to tune in
April 22, 2025 at 11:29 PM
Sitting in Ryan Blue’s talk at Data Council and about to leave from Q&A to head to office hours and question comes up about CDC from databases to Iceberg… @marcoslot.com makes a hard U-turn to tune in
@andypavlo.bsky.social on HN sums it up well
OLAP vs. OLTP isn't right vs. wrong, they're designed for different things.
Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres.
OLAP vs. OLTP isn't right vs. wrong, they're designed for different things.
Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres.
April 22, 2025 at 10:40 PM
@andypavlo.bsky.social on HN sums it up well
OLAP vs. OLTP isn't right vs. wrong, they're designed for different things.
Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres.
OLAP vs. OLTP isn't right vs. wrong, they're designed for different things.
Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres.
Reposted by Craig
And there it is: Native logical replication from any Postgres server to Iceberg managed by Crunchy Data Warehouse.
Speed up Postgres analytical queries 100x with 2 commands.
Speed up Postgres analytical queries 100x with 2 commands.
April 22, 2025 at 2:48 PM
And there it is: Native logical replication from any Postgres server to Iceberg managed by Crunchy Data Warehouse.
Speed up Postgres analytical queries 100x with 2 commands.
Speed up Postgres analytical queries 100x with 2 commands.
Reposted by Craig
Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse.
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Logical replication from Postgres to Iceberg | Crunchy Data Blog
We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.
www.crunchydata.com
April 22, 2025 at 2:26 PM
Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse.
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Reposted by Craig
If you're using Postgres for your app data you can stop stitching together a myriad of ETL tools and analytics data stores and use one database for everything. This is both awesome and a lot of fun to use.
Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse.
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Logical replication from Postgres to Iceberg | Crunchy Data Blog
We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.
www.crunchydata.com
April 22, 2025 at 3:30 PM
If you're using Postgres for your app data you can stop stitching together a myriad of ETL tools and analytics data stores and use one database for everything. This is both awesome and a lot of fun to use.
The way this converges operational and analytical systems is nothing short of magical.
bsky.app/profile/crun...
bsky.app/profile/crun...
Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse.
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Now you can seamlessly move data and stream changes from your operational database into an analytical system.
www.crunchydata.com/blog/logical...
Logical replication from Postgres to Iceberg | Crunchy Data Blog
We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.
www.crunchydata.com
April 22, 2025 at 2:38 PM
The way this converges operational and analytical systems is nothing short of magical.
bsky.app/profile/crun...
bsky.app/profile/crun...
🐘 meets 🧊
End to end, under 2 minutes.
- Two commands to replicate data from Postgres -> Iceberg
- Synced over 10m rows under a minute
- Data is continually processed and updated in Iceberg
- count(*) in Postgres over 300ms down to under 20ms
End to end, under 2 minutes.
- Two commands to replicate data from Postgres -> Iceberg
- Synced over 10m rows under a minute
- Data is continually processed and updated in Iceberg
- count(*) in Postgres over 300ms down to under 20ms
April 22, 2025 at 2:38 PM
🐘 meets 🧊
End to end, under 2 minutes.
- Two commands to replicate data from Postgres -> Iceberg
- Synced over 10m rows under a minute
- Data is continually processed and updated in Iceberg
- count(*) in Postgres over 300ms down to under 20ms
End to end, under 2 minutes.
- Two commands to replicate data from Postgres -> Iceberg
- Synced over 10m rows under a minute
- Data is continually processed and updated in Iceberg
- count(*) in Postgres over 300ms down to under 20ms
Now playing: Eye of the tiger
April 22, 2025 at 1:55 PM
Now playing: Eye of the tiger
We now expose the physical zone directly to users on Crunchy Bridge. Most think they know what an availability zone is, but did you know by default AWS randomizes the names between accounts. You have to get to the physical zone id to colocate things - docs.crunchybridge.com/changelog#fe...
Changelog
docs.crunchybridge.com
April 21, 2025 at 11:47 PM
We now expose the physical zone directly to users on Crunchy Bridge. Most think they know what an availability zone is, but did you know by default AWS randomizes the names between accounts. You have to get to the physical zone id to colocate things - docs.crunchybridge.com/changelog#fe...
I've learned over the years when @louisemeta.bsky.social tells me she has an idea for a blog post I don't hesitate, just simply ask when can I see it because I know it's going to be awesome. This is no exception
Postgres does a good job of keeping internal statistics of your which are used to plan how queries are executed. But Postgres doesn't always know how columns are related. In this deep dive we look at hacking the statistics for improved performance
Hacking the Postgres Statistics Tables for Faster Queries | Crunchy Data Blog
Adding extended statistics can add information about how columns are related. Louise has some real life example queries and tips for working with extended table stats that can dramatically improve…
www.crunchydata.com
April 16, 2025 at 2:01 PM
I've learned over the years when @louisemeta.bsky.social tells me she has an idea for a blog post I don't hesitate, just simply ask when can I see it because I know it's going to be awesome. This is no exception