Medallion architecture in practice: designing Bronze, Silver, and Gold layers
Bronze, Silver, and Gold are not just folder names. Define data contracts, quality rules, replay behavior, and ownership at every layer.
Separate layers by responsibility
Bronze stores source-near data with minimal transformation. Silver applies schema, type, identity, deduplication, and quality rules. Gold exposes tables shaped for a business domain with defined meaning. Their value comes not from the names but from making transformations, ownership, and error replay explicit.
Bronze: capture sources without losing evidence
Store raw records with source system, ingestion time, file name, batch identifier, and source change identifier where available. Use an idempotent key so a repeated file is not processed twice. When a source schema changes, track the new schema and transformation version instead of silently overwriting history. This makes future reprocessing auditable.
Silver: validate, deduplicate, and standardize
Standardize timestamps, currencies, code lists, customer identifiers, and product keys in Silver. Do not delete duplicates merely because all columns match; define business keys, source precedence, and update-time rules. Quarantine invalid records with an error reason. When quality checks fail, do not quietly refresh Gold with incomplete data; stop the load or explicitly mark it partial.
Gold: make business definitions stable
Gold tables define measures and dimensions for domains such as sales, manufacturing, finance, or customers. The fact table grain must be explicit: order line, daily machine summary, or individual sensor event. Mixing grains in one table breaks totals. A Power BI semantic model can place shared measures and security rules over these curated tables.
Choose Delta optimization for the workload
Delta Lake provides transactional consistency and table versions, but not every table needs the same maintenance policy. Measure compaction, OPTIMIZE, and V-Order for small files, frequent updates, and read-heavy reports. V-Order can be disabled by default in newer Fabric workspaces; it may increase write cost while improving some read scenarios. Test representative queries and loads first.
Separate a claim from its evidence
A demo or experiment does not prove that an approach works in every context. Record measurement conditions, versions, data samples and assumptions. Compare a new method with the current one under the same workload before claiming a performance or quality improvement. Explicit uncertainty is more useful than false precision.
Keep tests and evidence for long-term value
Investigate root causes rather than symptoms. Check downstream components, stored records and user actions affected by a change. Test both successful and failed paths. Record dates, versions, measurements and references so later decisions can be reviewed without reconstructing the entire history.
Comments
No comments yet. Be the first to contribute.
Try the comment interactions · demo preview
The two examples below are only a UI demo, not real visitor comments. Likes and replies are local to this page and are not saved; use the real form below to post.
I'd like to see which steps benefit most in real deployment.
An example test scenario would make the technical analysis clearer.
✎ Write a comment