Most teams understand the Bronze-Silver-Gold concept within five minutes of reading about it. The architecture looks simple on paper. Three layers, data moves left to right, quality improves at each step.
What those same teams discover six months into production is that the concept is simple and the execution is where everything breaks. Silver tables accumulate without enforcement. Gold tables multiply faster than anyone can govern them. A schema change in a source system silently propagates through all three layers because nobody defined what each layer is actually responsible for stopping.
This article is about the execution, not the concept.
It is part of the Modern Data Engineering: The Complete Guide series. The architecture foundation is in Databricks for Data Engineering: Architecture, Components, and Best Practices. The storage layer that makes all three tiers work is covered in Delta Lake Explained for Data Engineers.
What Medallion Architecture Is and Why Databricks Uses It
Medallion architecture organizes data into layers of increasing quality and trustworthiness. Raw data lands in Bronze exactly as received. Silver cleans and validates it. Gold aggregates it for specific consumers.
As the Databricks official medallion documentation describes it: the goal is to incrementally and progressively improve the structure and quality of data as it flows through each layer. Each hop adds validation, transformation, and business logic.
The reason this matters practically: without it, every consumer of a data platform builds their own version of "clean." One analyst filters nulls before their query. Another does not. A data scientist drops duplicates in a notebook. A BI dashboard does not. The same source data produces different numbers depending on who is querying it and what they happened to clean in their own code.
Medallion architecture makes cleaning a platform responsibility, not a per-consumer responsibility. As Tacnode's March 2026 lakehouse architecture analysis explains, the critical distinction is that medallion architecture is a logical pattern, not a product. You can implement it on any platform. What makes it work is the contract each layer enforces, not the labels applied to the tables.
The Three Layers: What Each One Actually Does
| Layer | Data State | Who Writes | Who Reads | Key Rule |
|---|---|---|---|---|
| Bronze | Raw, unmodified | Lakeflow Connect, Auto Loader | Silver pipelines only | Never transform, never drop fields |
| Silver | Cleaned, validated, deduplicated | Silver pipelines | Gold pipelines, data scientists, ML teams | Schema enforced, expectations applied |
| Gold | Aggregated, use-case optimized | Gold pipelines | BI tools, analysts, dashboards, ML serving | One table per defined use case |
Bronze: Raw and Untouched
Bronze is your source of truth for raw data. Not a useful truth. Not a clean truth. The raw truth exactly as the source system delivered it, with all its nulls, duplicates, inconsistent formats, and schema surprises intact.
The rule for Bronze is simple: do not transform, do not drop fields, do not apply business logic. Write everything. If a source system adds a new column tomorrow, Bronze absorbs it automatically via schema evolution. Nothing downstream breaks immediately because Bronze makes no promises about what the data looks like.
What Bronze does enforce: ACID writes (via Delta Lake), schema evolution for new columns, and a ingestiontimestamp metadata column on every row so you can always trace when a record arrived. Auto Loader handles incremental file-based ingestion. Lakeflow Connect handles enterprise application and database sources. Both land data in Bronze without custom ingestion code.
Retention for Bronze is typically 30 to 90 days, not forever. As reintech.io's 2026 medallion implementation guide recommends: set explicit retention policies for each layer. Bronze might retain 90 days, Silver a year, Gold indefinitely. Bronze is not an archive. It is a recovery layer. If a Silver transformation produces wrong results, you reprocess from Bronze. Once that reprocessing window has passed, old Bronze data serves no operational purpose and continues costing storage.
Silver: The Contract Layer
Silver is where Bronze's chaos becomes something trustworthy. It is also where most teams get the architecture wrong.
The mistake: treating Silver as "a bit cleaner than Bronze." Teams drop obvious nulls, maybe remove a few duplicates, and call it Silver. There are no defined rules. There are no enforced expectations. Silver is just Bronze with some manual cleanup applied.
That is not Silver. That is Bronze with extra steps.
Real Silver has explicit contracts:
- Schema is defined and enforced. A row that violates the schema fails, visibly, at write time.
- Deduplication is applied with a defined deduplication key and strategy.
- Data quality expectations are declared and monitored. A customer_id field that allows nulls is not a Silver-quality field.
- Business keys are reconciled across sources that use different identifiers for the same entity.
Inside Lakeflow Declarative Pipelines, expectations enforce these rules at write time. A row that violates an expectation either gets quarantined to a separate table for investigation, or fails the pipeline run depending on severity. Teams that skip expectations in Silver are running without a net. Bad data passes through and reaches Gold silently.
Data Quality and Reliability Patterns for Databricks Pipelines covers how to design expectations at each tier boundary, including quarantine patterns and data quality monitoring with Unity Catalog.
Gold: One Table, One Use Case
Gold is for consumers. BI analysts, dashboards, ML feature stores, operational APIs. Gold tables are optimized for the people and systems that read them, not for the engineers who build them.
The problem that kills most medallion implementations at the Gold layer is proliferation. As Apurva Patil's January 2026 medallion guide in Towards Data Engineering puts it directly: creating too many Gold tables without ownership is one of the top failure modes in production lakehouse implementations. Every team requests a custom Gold table. Within six months there are 60 Gold tables. Some cover nearly identical logic with slightly different business definitions. Different tables give different answers to the same question. Analysts argue about which one is "right." Trust in the platform collapses.
The rule: one Gold table per defined consumer use case, with a named owner. If two teams want similar aggregations, either they share one Gold table with a clear definition, or the difference in their requirements is significant enough to justify a separate table with its own documented purpose.
Gold tables should have the shortest compute time of any layer because the work was done in Silver. If a Gold transformation is complex and slow, the complexity belongs in Silver.
Data Quality at Tier Boundaries: How Expectations Work
The boundary between Bronze and Silver is where quality enforcement begins. The boundary between Silver and Gold is where it is verified.
Lakeflow Spark Declarative Pipelines supports three expectation actions:
- WARN: Log the violation, allow the row to pass. Use this for monitoring Bronze layer anomalies.
- DROP: Drop violating rows silently. Dangerous if overused. Best for known noise in Bronze.
- FAIL: Fail the pipeline run. Use this at Silver-to-Gold boundaries where a bad row means the output is meaningless.
A practical expectation design by tier:
| Tier Boundary | Expectation Type | Example Rule |
|---|---|---|
| Bronze ingest | WARN | Log rows where event_type is null |
| Bronze to Silver | DROP | Drop rows where customer_id is null or negative |
| Bronze to Silver | FAIL | Fail if more than 5% of rows in a batch have null transaction_id |
| Silver to Gold | FAIL | Fail if revenue_amount contains negative values |
The 5% threshold fail pattern is the most underused quality control in medallion implementations. A single bad row might be acceptable. A batch where 30% of rows fail a key validation is a source system problem that no downstream consumer should ever see.
CDC Through the Medallion Stack
Most production medallion pipelines are not full-reloads. They are incremental. New rows arrive in Bronze, and those changes need to propagate through Silver and into Gold without reprocessing the entire history on every run.
Change Data Feed on Delta Lake handles this. Enable CDF on Silver tables and the Bronze-to-Silver pipeline reads only new or changed rows since the last run. Silver updates its records via MERGE. Gold materializes only what changed.
The key design decision at Silver: SCD Type 1 or SCD Type 2? As theOpenExamPrep 2026 medallion architecture study guide notes, reprocessing capability is one of the core design advantages of the medallion pattern: if a transformation bug is found, fix the Silver logic and rerun from Bronze with full traceability from raw source to business metric.
- SCD Type 1: Overwrite the existing record. The latest value is all that matters. Use this for dimension attributes where history is irrelevant.
- SCD Type 2: Preserve history with validfrom and validto columns. Use this when analysts or ML models need to reconstruct what a record looked like at a specific point in time.
Getting this wrong is an expensive fix. A Silver table designed as SCD Type 1 that a business later needs as SCD Type 2 requires a full historical rebuild from Bronze.
Incremental Loads, CDC, and Change Data Feed in Delta Lake covers the full implementation of both SCD patterns, the AUTO CDC API inside Lakeflow Pipelines, and how to handle late-arriving records without corrupting Silver state.
Three Mistakes That Break Medallion Implementations
Performing transformations in Bronze. The moment you apply business logic in Bronze, it is no longer raw. If that logic is wrong, you have no clean original to reprocess from. Bronze must be write-only from source, read-only by Silver pipelines.
No data quality expectations at Silver. Silver without enforced expectations is just Bronze with different table names. Every Silver table should have at least three defined expectations with explicit actions on violation. If you cannot define what "clean" means for a table, the table is not ready for Silver.
Over-materializing Gold. Twenty Gold tables serving similar purposes is not a richer platform. It is a trust problem. As the DEV Community's March 2026 production medallion implementation guide shows, a mature production workspace organizes Gold as separate aggregation jobs each with a named owner, not as a shared dumping ground for every possible business metric. When stakeholders cannot agree on which Gold table is authoritative for a metric, they stop trusting the platform. One owner, one definition, one Gold table per use case.
What This Covers Next in the Series
- Lakeflow Pipelines for Data Engineering covers how to implement the Bronze-to-Silver-to-Gold flow inside Lakeflow Declarative Pipelines, including streaming table design, expectation patterns, and schema evolution handling.
- Incremental Loads, CDC, and Change Data Feed in Delta Lake goes deep on the incremental processing patterns that keep all three medallion tiers current without full-table reprocessing.
- Data Governance with Unity Catalog in Databricks covers how Unity Catalog assigns ownership, enforces access control, and tracks lineage across all three medallion layers automatically.