LightNews LightNews
jonathanbhaskar.bsky.social
@jonathanbhaskar.bsky.social
17 followers 160 following 3 posts
Posts Replies Media Videos
jonathanbhaskar.bsky.social
I wrote an article exploring how we could make data pipelines easier to build and maintain by moving from code to configuration, drawing lessons from how DevOps evolved:

blog.jonathanb.me/declarative-...

#databs #datasky
Declarative Data Pipelines: Moving from Code to Configuration
Learn how declarative data pipelines can transform data engineering by separating implementation from business logic, making pipelines more maintainable
blog.jonathanb.me
February 5, 2025 at 10:28 AM
jonathanbhaskar.bsky.social
I created a library to build data pipelines with YAML/JSON

Works on dagster

Integrates with dlthub for ingestion and soda for data quality

#dataBS

github.com/runodp/dagst...
GitHub - runodp/dagster-odp: A configuration-driven framework for building Dagster pipelines that enables teams to create and manage data workflows using YAML/JSON instead of code
A configuration-driven framework for building Dagster pipelines that enables teams to create and manage data workflows using YAML/JSON instead of code - runodp/dagster-odp
github.com
December 2, 2024 at 2:39 PM