colton
@colton.boo
Data Engineering / Music / Photography / Tinkering
Developer Advocate @ Dagster
cmpadden.github.io
Developer Advocate @ Dagster
cmpadden.github.io
Reposted by colton
“Flexibility without fragmentation”
In addition to some solid alteration, it’s also a great explanation of how to build a data team and how your system can and should scale.
@colton.boo and Dennis Hume have an awesome new ebook on this topic, check it out!
#dataBS
dagster.io/how-to-scale...
In addition to some solid alteration, it’s also a great explanation of how to build a data team and how your system can and should scale.
@colton.boo and Dennis Hume have an awesome new ebook on this topic, check it out!
#dataBS
dagster.io/how-to-scale...
How to Scale Data Teams | Free eBook from Dagster
Download Dagster's free eBook to learn how to build systems that scale with clarity, reliability, and confidence.
dagster.io
November 7, 2025 at 2:40 AM
“Flexibility without fragmentation”
In addition to some solid alteration, it’s also a great explanation of how to build a data team and how your system can and should scale.
@colton.boo and Dennis Hume have an awesome new ebook on this topic, check it out!
#dataBS
dagster.io/how-to-scale...
In addition to some solid alteration, it’s also a great explanation of how to build a data team and how your system can and should scale.
@colton.boo and Dennis Hume have an awesome new ebook on this topic, check it out!
#dataBS
dagster.io/how-to-scale...
Excited to be at All Things Open tomorrow!
Come hang out at the @dagster.io booth, or see my talk “Enabling education with a little help from AI” at 1:45 EST on Tuesday!
Come hang out at the @dagster.io booth, or see my talk “Enabling education with a little help from AI” at 1:45 EST on Tuesday!
October 12, 2025 at 9:43 PM
Excited to be at All Things Open tomorrow!
Come hang out at the @dagster.io booth, or see my talk “Enabling education with a little help from AI” at 1:45 EST on Tuesday!
Come hang out at the @dagster.io booth, or see my talk “Enabling education with a little help from AI” at 1:45 EST on Tuesday!
Reposted by colton
Read this post you work in data analytics.
A similar mindset problem pervades sales engineering as well, especially in the demo / early stages of discovery.
Understand your prospect’s context. Who are they? What do they care about? What value can you provide them? That’s what you show.
A similar mindset problem pervades sales engineering as well, especially in the demo / early stages of discovery.
Understand your prospect’s context. Who are they? What do they care about? What value can you provide them? That’s what you show.
Technical writing is hard bcs "writing is thinking" but we often should tell our story not in the order we worked. Solution? I wrote a quick post on how @quarto.org 's embed shortcodes can reframe technical writing as reproducible evidence curation
www.emilyriederer.com/post/quarto-...
🧵 (1/n)
www.emilyriederer.com/post/quarto-...
🧵 (1/n)
How Quarto embed fixes data science storytelling | Emily Riederer
Literate programming excels at capturing our stream of conscience. Our stream of conscience does not excel at explaining the impact of our work. Notebooks enable some of data scientists’ worst tendenc...
www.emilyriederer.com
July 28, 2025 at 1:39 AM
Read this post you work in data analytics.
A similar mindset problem pervades sales engineering as well, especially in the demo / early stages of discovery.
Understand your prospect’s context. Who are they? What do they care about? What value can you provide them? That’s what you show.
A similar mindset problem pervades sales engineering as well, especially in the demo / early stages of discovery.
Understand your prospect’s context. Who are they? What do they care about? What value can you provide them? That’s what you show.
MotherDuck is live now!
DuckLake & The Future of Open Table Formats ft. Jordan Tigani & Hannes Mühleisen
www.youtube.com/live/yn07s-_...
DuckLake & The Future of Open Table Formats ft. Jordan Tigani & Hannes Mühleisen
www.youtube.com/live/yn07s-_...
DuckLake & The Future of Open Table Formats ft. Jordan Tigani & Hannes Mühleisen
YouTube video by MotherDuck
www.youtube.com
June 17, 2025 at 3:03 PM
MotherDuck is live now!
DuckLake & The Future of Open Table Formats ft. Jordan Tigani & Hannes Mühleisen
www.youtube.com/live/yn07s-_...
DuckLake & The Future of Open Table Formats ft. Jordan Tigani & Hannes Mühleisen
www.youtube.com/live/yn07s-_...
New Oxidise Your Command Line just dropped.
Some that stand out to me are: xh, fselect, and presenterm!
www.youtube.com/watch?v=rWMQ...
Some that stand out to me are: xh, fselect, and presenterm!
www.youtube.com/watch?v=rWMQ...
Oxidise Your Command Line (2025 Edition)
YouTube video by No Boilerplate
www.youtube.com
May 23, 2025 at 3:28 AM
New Oxidise Your Command Line just dropped.
Some that stand out to me are: xh, fselect, and presenterm!
www.youtube.com/watch?v=rWMQ...
Some that stand out to me are: xh, fselect, and presenterm!
www.youtube.com/watch?v=rWMQ...
Reposted by colton
I'm writing about #opentableformats. With the newest entries of AWS S3 tables and Cloudflare R2, it's interesting to reassess the market and see where the future with managed Iceberg Tables leads us.
What's your take? And anything missing on the evolution below?
What's your take? And anything missing on the evolution below?
May 2, 2025 at 12:28 PM
I'm writing about #opentableformats. With the newest entries of AWS S3 tables and Cloudflare R2, it's interesting to reassess the market and see where the future with managed Iceberg Tables leads us.
What's your take? And anything missing on the evolution below?
What's your take? And anything missing on the evolution below?
Refactored `chatblade.nvim` to use `llm`, and renamed the project to `llm.nvim`. It's archaic compared these agentic MCP workflows, but it's completely replaced Googling programming related questions.
April 9, 2025 at 2:29 AM
Refactored `chatblade.nvim` to use `llm`, and renamed the project to `llm.nvim`. It's archaic compared these agentic MCP workflows, but it's completely replaced Googling programming related questions.
I sat down to chat with Chris about "Project Airlift", a way to easily observe Airflow environments directly from Dagster.
With a couple of lines of code, you can get a holistic picture into all of your environments, and then optionally migrate them!
With a couple of lines of code, you can get a holistic picture into all of your environments, and then optionally migrate them!
Dagster's project Airlift makes it easy to observe and migrate Apache Airflow pipelines to Dagster, bringing a ton of value!
youtube.com/shorts/9UF64...
youtube.com/shorts/9UF64...
April 3, 2025 at 2:35 PM
I sat down to chat with Chris about "Project Airlift", a way to easily observe Airflow environments directly from Dagster.
With a couple of lines of code, you can get a holistic picture into all of your environments, and then optionally migrate them!
With a couple of lines of code, you can get a holistic picture into all of your environments, and then optionally migrate them!
Reposted by colton
🚨 New Course Alert 🚨
We are pleased to announce that the new "Testing with Dagster" course is now live on Dagster University.
dagster.io/blog/dagster...
We are pleased to announce that the new "Testing with Dagster" course is now live on Dagster University.
dagster.io/blog/dagster...
April 1, 2025 at 6:00 PM
🚨 New Course Alert 🚨
We are pleased to announce that the new "Testing with Dagster" course is now live on Dagster University.
dagster.io/blog/dagster...
We are pleased to announce that the new "Testing with Dagster" course is now live on Dagster University.
dagster.io/blog/dagster...
Tinkering with dltHub's REST API source to ingest data without having to write any bespoke code, and DuckDB's new local web UI, and I feel so spoiled with how good data tooling is getting!
March 21, 2025 at 3:43 PM
Tinkering with dltHub's REST API source to ingest data without having to write any bespoke code, and DuckDB's new local web UI, and I feel so spoiled with how good data tooling is getting!
Reposted by colton
🚀 Want to preview the brand new `dg` cli, and a framework for building and working with YAML DSLs built on top of Dagster called "Components"?
You can find a link to demo videos and a GitHub discussion in the thread!
You can find a link to demo videos and a GitHub discussion in the thread!
March 13, 2025 at 9:19 PM
🚀 Want to preview the brand new `dg` cli, and a framework for building and working with YAML DSLs built on top of Dagster called "Components"?
You can find a link to demo videos and a GitHub discussion in the thread!
You can find a link to demo videos and a GitHub discussion in the thread!
Joe Naso and I just published an e-book covering the essential topics needed to build a data platform!
Check it out if you're looking to learn more about data modeling, ingestion patterns, and common data architectures.
dagster.io/how-to-build...
Check it out if you're looking to learn more about data modeling, ingestion patterns, and common data architectures.
dagster.io/how-to-build...
February 25, 2025 at 7:26 PM
Joe Naso and I just published an e-book covering the essential topics needed to build a data platform!
Check it out if you're looking to learn more about data modeling, ingestion patterns, and common data architectures.
dagster.io/how-to-build...
Check it out if you're looking to learn more about data modeling, ingestion patterns, and common data architectures.
dagster.io/how-to-build...
Ok, Claude Code, consider me impressed!
February 24, 2025 at 10:04 PM
Ok, Claude Code, consider me impressed!
Reposted by colton
I think this is one of the most important books data people could be reading right now, especially if you need to work language models which need all the semantics they can get.
Matthew Mullins on LinkedIn: Data modeling is very important for any number of reasons, but much of the…
Data modeling is very important for any number of reasons, but much of the discourse on data modeling focuses on the shape of the date. How easy is it to…
www.linkedin.com
February 13, 2025 at 12:51 AM
I think this is one of the most important books data people could be reading right now, especially if you need to work language models which need all the semantics they can get.
Reposted by colton
Announcing the Data.gov Archive lil.law.harvard.edu/blog/2025/02...
February 8, 2025 at 6:39 PM
Announcing the Data.gov Archive lil.law.harvard.edu/blog/2025/02...
Come join us if you’d like to learn more about LLM routing from experts in the field!
Are you ready to get into the weeds on AI development best practices to reduce costs and improve accuracy? Join us on a Deep Dive on February 11 at 9 a.m. PT with the Not Diamond Team.
https://buff.ly/3EjhlAK
https://buff.ly/3EjhlAK
February 3, 2025 at 7:52 PM
Come join us if you’d like to learn more about LLM routing from experts in the field!
Reposted by colton
We’re building a new static type checker for Python, from scratch, in Rust.
From a technical perspective, it’s probably our most ambitious project yet. We’re about 800 PRs deep!
From a technical perspective, it’s probably our most ambitious project yet. We’re about 800 PRs deep!
January 29, 2025 at 5:18 PM
We’re building a new static type checker for Python, from scratch, in Rust.
From a technical perspective, it’s probably our most ambitious project yet. We’re about 800 PRs deep!
From a technical perspective, it’s probably our most ambitious project yet. We’re about 800 PRs deep!
Reposted by colton
As someone who works at an open core company similar to Preset / Superset, this post really resonates with my experience. Buying a “hosted version of an open source library” made by that libraries creators has more benefits than “just” the hosting. preset.io/blog/running...
Running Apache Superset on the Open Internet: A Report from the Fireline
Running Apache Superset on the open internet requires sophisticated solutions to security and performance challenges. Here we explore how to navigate these challenges while keeping data safe (or opt o...
preset.io
January 25, 2025 at 2:31 AM
As someone who works at an open core company similar to Preset / Superset, this post really resonates with my experience. Buying a “hosted version of an open source library” made by that libraries creators has more benefits than “just” the hosting. preset.io/blog/running...
Reposted by colton
I'm really proud to share that the new Dagster docs are live; what an effort!
The new Dagster docs experience is here, complete with dark mode!
This makes it easier than ever to get started with Dagster. Our focus was on creating a more user-friendly docs structure to ensure that all of the information you need is available within just a few clicks https://buff.ly/4awa90j
This makes it easier than ever to get started with Dagster. Our focus was on creating a more user-friendly docs structure to ensure that all of the information you need is available within just a few clicks https://buff.ly/4awa90j
January 23, 2025 at 10:53 PM
I'm really proud to share that the new Dagster docs are live; what an effort!
This is a great resource covering the tools you need as a data engineer!
Today, I'm sharing «The Data Engineering Toolkit» a set of tools and fundamentals for any data engineer who is getting started or wants to understand the role of a modern data engineer.
Maybe you remember the book: The DWH Toolkit. I take a step back, focusing on the essential toolset and knowledge
Maybe you remember the book: The DWH Toolkit. I take a step back, focusing on the essential toolset and knowledge
The Data Engineering Toolkit: Essential Tools for Your Machine - MotherDuck Blog
A comprehensive list of essential tools and environments every data engineer needs, from Linux commands to Docker and modern programming languages | Reading time: 19 min read
motherduck.com
January 23, 2025 at 4:28 PM
This is a great resource covering the tools you need as a data engineer!
Loved this discussion between @barrald.bsky.social from Hex and @pedramnavid.com from Dagster on the data ecosystem, and the impact of AI on the work of data engineers and analysts!
www.youtube.com/watch?v=8JxD...
www.youtube.com/watch?v=8JxD...
Friends of Data: Pedram Navid, Chief Dashboard Officer at Dagster
YouTube video by Hex
www.youtube.com
January 16, 2025 at 10:16 PM
Loved this discussion between @barrald.bsky.social from Hex and @pedramnavid.com from Dagster on the data ecosystem, and the impact of AI on the work of data engineers and analysts!
www.youtube.com/watch?v=8JxD...
www.youtube.com/watch?v=8JxD...
Reposted by colton
I'm always really enjoying these presentations, sharing all the code and showcasing what's possible today.
Amazing integrating BI tools such as Power BI, Looker, Tableau, or Sigma. I believe that's the first time an open-source orchestrator is fully end-to-end.
📺 youtu.be/z3trqkKPbsI?...
Amazing integrating BI tools such as Power BI, Looker, Tableau, or Sigma. I believe that's the first time an open-source orchestrator is fully end-to-end.
📺 youtu.be/z3trqkKPbsI?...
January 14, 2025 at 8:55 PM
I'm always really enjoying these presentations, sharing all the code and showcasing what's possible today.
Amazing integrating BI tools such as Power BI, Looker, Tableau, or Sigma. I believe that's the first time an open-source orchestrator is fully end-to-end.
📺 youtu.be/z3trqkKPbsI?...
Amazing integrating BI tools such as Power BI, Looker, Tableau, or Sigma. I believe that's the first time an open-source orchestrator is fully end-to-end.
📺 youtu.be/z3trqkKPbsI?...