Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
From Data Chaos to a Trust Score for Every Table
…s Customer Stories / Databricks Data Platform Runs On Databricks CUSTOMER STORY From data chaos to a trust score for every table Watch video 100K+ tables Now classified and governed automatically eliminating…
Company
Databricks
…rity and Trust Ready to get started? Get a Demo Login Contact Us Try Databricks Customer Stories / Databricks Data Platform Runs On Databricks CUSTOMER STORY From data chaos to a trust score for every table Watch video 100…
Description
Databricks' own internal Data Platform team rebuilt data governance across its internal lakehouse of more than 100,000 tables spanning engineering, go-to-market, HR, finance and product telemetry, using Unity Catalog to move from inconsistent, team-by-team access patterns to automatic, measurable governance. The team built a universal classification model tagging every table by sensitivity and domain, an internal system called Fortress enforcing purpose-bound, time-limited access via Unity Catalog's security APIs, and a continuous Data Governance Score (0-100) per dataset covering documentation, reliability and governance, propagated automatically through data lineage. The resulting governance layer now underpins AI agents used across every function at Databricks, from engineering to finance to facilities, letting teams check a dataset's governance score before using it to train a model instead of running manual reviews.
…checks pass?), and governance (is the data classified and access-controlled?). The Data Governance Score propagates downstream through lineage: if an upstream table loses trust, every table that depends on it reflects that…
Problem
Before Unity Catalog, governance was inconsistent: access patterns varied by team, classifications were incomplete, and no one could confidently say which datasets were safe for AI training. Security, legal and engineering teams spent hours chasing spreadsheets to answer basic questions during compliance reviews, and new AI projects stalled because no one could confirm whether a dataset met privacy requirements.
…ations were inconsistent or missing entirely. The friction was not theoretical. Security, legal and engineering teams spent hours chasing spreadsheets to answer basic questions during compliance reviews. New AI projects stalled because no one could confirm whether a dataset met pri…
Implementation approach
The team started as alpha customers of Unity Catalog. First, they built a universal classification model tagging every table by sensitivity and domain. Second, they automated access controls through an internal platform called Fortress, built on Unity Catalog's security APIs, enforcing purpose-bound, time-limited access. To measure trust at scale, the team built the Data Governance Score, a continuous 0-100 metric per dataset covering documentation, reliability and governance, propagating automatically through data lineage.
…a paradigm shift that fundamentally changed how they approached the lakehouse. First, they built a universal classification model inside Unity Catalog. Every table is tagged by data sensitivity (Public, Internal, Confidential, Res…
Business value
Over 100,000 tables are now classified and governed automatically, eliminating manual audit cycles, with the Data Governance Score giving every dataset a measurable trust rating tied to AI readiness, used daily by more than 10,000 Databricks employees across every function.
…rnance Score gives every dataset a measurable trust rating tied to AI readiness 10,000+ Databricks employees — all using the platform daily, from engineering to finance The Databricks inte…
Industry
Technology & Software
…ABRICKS Databricks Finance Runs on Databricks Watch now Share this post Details Industry : Technology and Software Use Case : Analytics and Business Intelligence , Artificial Intelligence , Gove…
Technology
Unity Catalog, Lakeflow Connect, Databricks Apps, AI/BI Genie, Delta Lake, Databricks SQL
…rnance layer, additional data platform workstreams extend the platform’s reach. Lakeflow Connect handles secure SaaS ingestion at scale, which replaced fragile connectors with governed pipelines that land data direc…
AI capabilities
Agentic AI
…hat matters most is the one the team did not plan for. Wong frames it bluntly: “Data governance is table stakes for agentic AI.” A human will never browse 100,000 tables. An AI agent will find the stale dat…
Deployment options
cloud
…ith governed pipelines that land data directly into the medallion architecture. A multicloud platform ensures workloads are portable and secure across AWS, Azure and GCP. A federated analytics layer gives each business domain, from finance to market…
