What to watch in the eight days before Ignite—and how to translate the signals into architecture bets that age well.

A recent conversation with a colleague reminded me of how important applying structure to an agile project team is. Most industries can't take, to quote the old Blizzard line, the option to release it "when it's ready," because they've made commitments to customers or other parts of the business. Agile frees teams from big-batch planning, but it doesn’t free them from consequences.

Schema changes have always been risky because a schema isn’t just columns—it’s the interface between data producers and data consumers. Historically, that interface was rigid, which made any change expensive. Modern lakehouse design solves the problem structurally: a Medallion architecture separates where variation is tolerated (Bronze) from where commitment is made (Silver) and relied upon (Gold). In Microsoft Fabric, those roles map cleanly to Lakehouse, Warehouse, and Power BI’s semantic layer, with governance and domain‑oriented (data‑product) design tying it all together.

I’ve never been fond of the phrase data‑driven. It can imply that people should surrender the wheel to whatever the chart says. I prefer data‑enabled: a culture where evidence is visible, disputable, and useful—where humans steer and data is the headlight, not the driver. That shift doesn’t start with a platform; it starts with literacy, and it grows when more people can do a little analysis for themselves.

Architecture diagrams look clean—until they collide with the place your system actually lives. That place is not just “production.” It’s your organization: a dense system of systems made of teams, processes, policies, budgets, and tools. The primary reason to understand the organization in which a system will reside is precisely these systems. Your software must integrate with them as surely as it integrates with databases and queues.

If software requirements really are a “black art,” Software Requirements (3rd Edition) is the field guide that turns on the lights. Karl Wiegers and Joy Beatty’s update for Microsoft Press remains the most practical, end‑to‑end treatment of requirements work I’ve seen—thorough enough for architects and business analysts, yet approachable enough to hand to a junior developer or data engineer without scaring them off.

If you agree that data should be judged by the decisions it improves, the way you build with data changes. You stop trying to pour every table from every system into one giant lake and assume value will appear later. Instead, you ship small, finished data products that help with one decision at a time. The first approach optimizes for storage.

In a recent post, I argued that a real release is a declaration—a line in the sand that says, this is the version we stand behind. A declaration begs a follow‑up: what exactly are we declaring? The honest answer is: requirements. A release without requirements is just a pile of diffs; a release grounded in requirements is a promise we can audit, test, and keep.

I keep saying the quiet part out loud: a modern warehouse ships meaning and trust, not just tables. If meaning changes invisibly, trust evaporates. Releases, Release Flow, and CI/CD in Microsoft Fabric are how you move quickly and keep confidence—by making change observable, reversible, and governed. Fabric’s Variable Library and a deliberate, database‑level metadata library are the glue that make this work day to day.

In an era where everything markets itself as “continuous”—continuous integration, continuous delivery, continuous retraining—it can feel quaint to talk about releases. But if we care about reliability and governance, we should talk about them more, not less. A true software‑style release is not nostalgia; it’s a commitment device. It’s the point where we say: this is the version we stand behind…

Modern data platforms live in tension: Source systems evolve faster than dimensional models can absorb. Audit and lineage are mandatory, but teams still need velocity. Cloud lakehouses, streaming, and domain ownership do not slot neatly into yesterday’s warehouse playbooks. Data Vault is a response to those pressures. It is both a modeling approach and a delivery method designed to (1) 

I like to say that the first product of a data team isn’t a table or a dashboard—it’s trust. Trust is built the same way in data as it is in software: through tests that catch regressions, encode intent, and make change safe. If pipelines are code, then they deserve the same rigor as code. That means unit tests you can run in seconds, integration tests that respect the messy edges of reality, comprehensive tests that exercise the platform end‑to‑end, and user acceptance testing that proves the system answers the questions people actually have.

For years, “Bronze” quietly became a parking lot for periodic snapshots: copy a slice from the source every hour/day, write new files, repeat. It worked, but it was noisy and expensive—lots of hot storage, lots of ingest compute, and a tendency to let “temporary” landing data turn into de‑facto history. Fabric upends that with two primitives that encourage Zero Unmanaged…

Governance, done well, accelerates innovation. That sounds counterintuitive because “governance” often conjures gatekeeping and delay. But in complex systems, enabling constraints—clear aims, decision rights, evidence standards, and risk guardrails—reduce thrash. They let teams move faster with less politics, less ambiguity, and fewer expensive reworks. Put simply: Governed innovation = purposeful exploration + disciplined decisions + explicit guardrails. Purposeful exploration means we start from outcomes the organization actually cares about (growth, safety, quality, equity, cost-to-serve) and frame hypotheses against those aims.

I love the promise of data mesh: push data ownership to the edges, let domain teams ship data as products, and watch the organization move faster. But here’s the unglamorous truth we keep repeating in classrooms and boardrooms: a mesh without strong, distributed data and analytics governance is just a tangle. Autonomy without agreed‑upon rules yields incompatible data products, brittle integrations, and an ever‑growing integration tax.

A Materialized Lake View (MLV) is a table in your Fabric lakehouse that’s defined by a SQL query and kept up‑to‑date by the service. You write one CREATE MATERIALIZED LAKE VIEW … AS SELECT … statement; Fabric figures out dependencies, materializes the result into your lakehouse, and refreshes it on a schedule. Today, MLVs are in preview, SQL‑first (Spark SQL), and designed to make Medallion layers (Bronze → Silver → Gold) declarative instead of hand‑assembled pipelines.

In Microsoft Fabric you’re sitting on top of Delta Lake tables in OneLake. If you flip on Delta Change Data Feed (CDF) for those tables, Delta will record row‑level inserts, deletes, and updates (including pre‑/post‑images for updates) and let you read just the changes between versions. That makes incremental processing for SCDs (Type 1/2) and Data Vault satellites dramatically simpler and cheaper because you aren’t rescanning entire tables—just consuming the “diff.” Fabric’s Lakehouse fully supports this because it’s natively Delta; Mirrored databases land in OneLake as Delta too, but (as of September 2025) Microsoft hasn’t documented a supported way to 

Most organizations have matured data governance (quality, ownership, catalogs) and are racing to formalize AI governance (risk, bias, safety, model monitoring). Application governance (SDLC, access, change control) keeps production systems stable. But the layer where business decisions actually touch numbers—analytics—often sits in a gray zone. KPI definitions live in wikis, dashboards implement subtle variations of the “same” metric, and spreadsheets quietly fork the math.

If you live in spreadsheets or SQL all day, the “one big table” (OBT) feels like home. Everything you need is right there: one row per thing, a column for every attribute, and no joins to worry about. It’s a great way to explore data fast—until it isn’t. This post explains, in plain language, why the star schema pays you back every day you analyze data, and how it keeps the speed you love without the headaches you’ve learned to live with.

When teams build warehouses the old way—source → bronze → silver → gold → semantic—visualization and semantic specialists are invited in at the end. Their job looks reactive: wire up a few visuals, name some measures, make it load fast enough. They inherit whatever the pipeline produced, then try to make meaning out of it. The failure mode is predictable: pixel‑perfect charts sitting on semantic quicksand, with definitions that shift underfoot and performance that depends on structures no one designed for the questions at hand.

Real‑Time Intelligence (RTI) is the part of Fabric that treats events and logs as first‑class citizens: you connect live streams, shape them, persist them, query them with KQL or SQL, visualize them, and trigger actions—all without leaving the SaaS surface. Concretely, RTI centers on Eventstream (ingest/transform/route), Eventhouse (KQL databases), Real‑Time Dashboards / Map, and Activator (detect patterns and act). That tight loop—capture → analyze → visualize/act—now covers everything from IoT telemetry to operational logs and clickstream analytics.

Information governance (IG) is the strategy, accountability, and control system for how an organization collects, classifies, uses, protects, shares, retains, and disposes of information across its entire lifecycle. It is: Scope‑wide: Covers structured data, unstructured content, model artifacts, code, dashboards, and records (including legal/records management and privacy). Lifecycle‑aware: From intake and creation → active use → archival → retention/disposition and legal holds.

If you’ve built data systems long enough, you’ve lived through at least three architectural moods: the tidy certainty of Kimball and Inmon, the anarchic freedom of “throw everything in the data lake to ingest quickly,” and today’s lakehouse, which tries to keep our speed without losing our sanity. I've always cared less about labels and more about baselines—clear, durable expectations that make change safe.

Most data teams still run a tidy assembly line: ingest sources into bronze, standardize into silver, curate into gold, and only then wire up a semantic model for BI. That sounds rigorous—but it puts the business contract (grain, conformed dimensions, measure logic, security scope, and SLOs) at the very end. By the time the organization finally argues about what “AUM” or a compliant “time‑weighted return” …

On edudatasci.net, I keep data mesh grounded in four behaviors: domains own data; data as a product; a small self‑serve platform; and federated governance (policies expressed as code and applied consistently). I also use foundational vs derived data products as a practical way to think about scope and ownership, and I recommend publishing products in Purview’s Unified Catalog so ownership, access and SLOs are discoverable to the org, not just the team that built them.