Adrian Brudaru
@datateam.bsky.social
Data engineer & Cofounder @dlthub. Building out the tooling i wish i had.
This isn't about AI taking jobs. It's about AI exposing what the real job should have been all along.
Building systems of trust. Asking "do I trust this output?" instead of "how do I write this script?"
The intern is here. It's waiting for a manager.
Building systems of trust. Asking "do I trust this output?" instead of "how do I write this script?"
The intern is here. It's waiting for a manager.
Surviving the AI code Deluge: Data quality in the Spotlight
This is, we’re told, the great democratization of data engineering. The tedious work is gone. The barrier to entry is gone. Everyone can now be a data engineer.
dlthub.com
October 14, 2025 at 12:50 PM
This isn't about AI taking jobs. It's about AI exposing what the real job should have been all along.
Building systems of trust. Asking "do I trust this output?" instead of "how do I write this script?"
The intern is here. It's waiting for a manager.
Building systems of trust. Asking "do I trust this output?" instead of "how do I write this script?"
The intern is here. It's waiting for a manager.
Then the LLMs arrived.
Suddenly, there's an AI intern who can write that heroic query in seconds. An intern who can build a thousand handcrafted pipelines a day.
It's here to automate the firefighter's job into oblivion.
Suddenly, there's an AI intern who can write that heroic query in seconds. An intern who can build a thousand handcrafted pipelines a day.
It's here to automate the firefighter's job into oblivion.
October 14, 2025 at 12:50 PM
Then the LLMs arrived.
Suddenly, there's an AI intern who can write that heroic query in seconds. An intern who can build a thousand handcrafted pipelines a day.
It's here to automate the firefighter's job into oblivion.
Suddenly, there's an AI intern who can write that heroic query in seconds. An intern who can build a thousand handcrafted pipelines a day.
It's here to automate the firefighter's job into oblivion.
5/5 Read Erfan's full post: https://pipeline2insights.substack.com/p/introduction-to-data-load-tool-dlt
#DataEngineering #Python #dlt #OpenSource #FridayReads
#DataEngineering #Python #dlt #OpenSource #FridayReads
Introduction to data load tool (dlt): A Python Library for Simple Data Ingestion
Discover the basics of dlt and its role in modern data engineering workflows
pipeline2insights.substack.com
October 10, 2025 at 11:50 AM
5/5 Read Erfan's full post: https://pipeline2insights.substack.com/p/introduction-to-data-load-tool-dlt
#DataEngineering #Python #dlt #OpenSource #FridayReads
#DataEngineering #Python #dlt #OpenSource #FridayReads
4/5 Whether you're 😤 tired of rebuilding ingestion pipelines, 🔧 looking to integrate Python tools into your data stack or just 💡 curious about modern ELT approaches, this guide has something for you.
October 10, 2025 at 11:50 AM
4/5 Whether you're 😤 tired of rebuilding ingestion pipelines, 🔧 looking to integrate Python tools into your data stack or just 💡 curious about modern ELT approaches, this guide has something for you.
3/5 The best part? Anyone who knows Python can build senior-level pipelines.
No new frameworks. No specialized expertise. Just Python doing what it does best.
No new frameworks. No specialized expertise. Just Python doing what it does best.
October 10, 2025 at 11:50 AM
3/5 The best part? Anyone who knows Python can build senior-level pipelines.
No new frameworks. No specialized expertise. Just Python doing what it does best.
No new frameworks. No specialized expertise. Just Python doing what it does best.
2/5 What Erfan's guide covers:
⚡ ETL vs ELT & how to avoid data swamps
🐍 Why Python devs need simpler data tools
⚙️ dlt automates schema evolution, incremental loading & data contracts
💻 A practical MySQL-to-DuckDB example you can try today
☁️ Plus real deployment options: Lambda, Airflow, Kubernetes
⚡ ETL vs ELT & how to avoid data swamps
🐍 Why Python devs need simpler data tools
⚙️ dlt automates schema evolution, incremental loading & data contracts
💻 A practical MySQL-to-DuckDB example you can try today
☁️ Plus real deployment options: Lambda, Airflow, Kubernetes
October 10, 2025 at 11:50 AM
2/5 What Erfan's guide covers:
⚡ ETL vs ELT & how to avoid data swamps
🐍 Why Python devs need simpler data tools
⚙️ dlt automates schema evolution, incremental loading & data contracts
💻 A practical MySQL-to-DuckDB example you can try today
☁️ Plus real deployment options: Lambda, Airflow, Kubernetes
⚡ ETL vs ELT & how to avoid data swamps
🐍 Why Python devs need simpler data tools
⚙️ dlt automates schema evolution, incremental loading & data contracts
💻 A practical MySQL-to-DuckDB example you can try today
☁️ Plus real deployment options: Lambda, Airflow, Kubernetes
2/2
In this blog post, I describe an approach to benchmarking EL pipeline creation by separating the generation problem into 3 parts - Extraction, intuitive memory, and code creation.
dlthub.com/blog/towards...
In this blog post, I describe an approach to benchmarking EL pipeline creation by separating the generation problem into 3 parts - Extraction, intuitive memory, and code creation.
dlthub.com/blog/towards...
Towards a Benchmark for AI-Generated Data Pipelines
Building pipelines with AI isn’t one task, it's many. In this post, we explore how to split and test them individually, so failures are easier to diagnose and fix.
dlthub.com
April 1, 2025 at 9:53 AM
2/2
In this blog post, I describe an approach to benchmarking EL pipeline creation by separating the generation problem into 3 parts - Extraction, intuitive memory, and code creation.
dlthub.com/blog/towards...
In this blog post, I describe an approach to benchmarking EL pipeline creation by separating the generation problem into 3 parts - Extraction, intuitive memory, and code creation.
dlthub.com/blog/towards...