Data ingestion with dlt and Dagster: An end-to-end pipeline tutorial:

Curious like us to see what people are sharing with #dataBS and #datasky? Check out this post to learn how to do it using dlt!"
@matthausk.bsky.social
@datateam.bsky.social
@hgeren.bsky.social
@hopefanhe.bsky.social
#dlt

We are starting a 32-week Data Engineering Interview Guide program, covering everything from fundamentals to advanced topics, with sessions every Saturday.
Do you think we're missing any critical topics? We're curious about your opinions😊
#dataBS
#datasky

As a Data Engineer, understanding the data storage lifecycle and data retention policies is critical for designing efficient, cost-effective, and compliant data systems.
@joereis.bsky.social
#dataBS #datasky

substack.com/@pipeline2in...

In our new post, we've covered 10 of the most popular data pipeline design patterns.

We’d love to hear your thoughts. For more details, please check out the full post created by (@hgeren.bsky.social and @hopefanhe.bsky.social ): open.substack.com/pub/pipeline...

#dataBS #datasky

Discover how dlt simplifies data ingestion.
Learn its origins and real-world use cases. Follow a step-by-step guide to build your first pipeline and join the growing dlt community!
@matthausk.bsky.social
@datateam.bsky.social
@hgeren.bsky.social
@hopefanhe.bsky.social

#dataBS #datasky

Hi, wishing everyone a great Thanksgiving!

Recently we wrote about how SQL queries are executed behind the scenes.

If you are interested, check out our post: open.substack.com/pub/pipeline...

#dataBS #datasky

Storage is at the heart of Data Engineering.
In this post, we explore the hierarchy of data storage from the ground up, drawing inspiration from Fundamentals of Data Engineering by
@joereis.bsky.social
and Matt Housley, as well as insights from the DE Professionals on Coursera.
#dataBS #datasky

Hey #dataBS and #datasky folks,

Our new post about "how understanding Big O Notation & Execution Plans can optimize SQL queries" has just been posted.

Check it out if you're interested, and we'd love to hear your thoughts! @hopefanhe.bsky.social
open.substack.com/pub/pipeline...

Hey #dataBS, I've been thinking of an analogy for Data Teams' roles.

Imagine a company as a vehicle. How would you map Data Engineering, Analytics, and Science to vehicle parts? Teams could have multiple parts or overlap with other Teams.

Curious about your thoughts!

Looking for a distraction? Try this great interview between @hannes.muehleisen.org and @medriscoll.bsky.social covering all things @duckdb.org. I especially enjoyed the philosophy around improving SQL usability. www.youtube.com/watch?v=a-Rm... #databs

#dstaBS can you repost?

Filled up the first 150 and so am creating a second starter pack! Let’s all keep finding each other and make this place the best for all things data

Week 1 of "100 Days of SQL Optimisation" covered key techniques like column selection, multicolumn indexes, filtering, window functions, Rank, CTE and composite indexes with IMDb data.

Check out the full post for more!
@hgeren.bsky.social
#dataBS #datasky

I made an infra engineer starter pack. Folks posting about databases, stream processing, durable execution, orchestrators, service meshes, and more.

go.bsky.app/SCZe42X

Hello everyone! I’m Hasan.

I transitioned from Industrial Engineering to Data Science, then found my passion in Data Engineering. Currently, doing a PhD in distributed stream processing while working as a Data Engineer.

Looking forward to connecting with fellow data enthusiasts to learn and share.

Just joined and heard #dataBS and #datasky are where the cool kids hang.

Wanted to introduce our blog where we regularly write about Data Engineering concepts, news, and tools.

pipeline2insights.substack.com