Inside the Governed Data Platform Powering ChatGPT: OpenAI on Databricks
OpenAIAgent Bricks · Databricks SQL · Delta Lake +2
OpenAI runs its marketing, analytics, security, and finance operations on the Databricks Data + AI Platform across Azure and AWS, with usage data landing in Delta tables governed by Unity Catalog. OpenAI's Security team uses Databricks streaming pipelines to standardize infrastructure, cloud, and vendor audit logs into governed Delta tables for threat detection, investigation, and incident response, including Codex agents querying governed security data via SQL APIs. The marketing team uses a Bronze-Silver-Gold architecture to build audiences without writing SQL for one billion weekly active users, eliminating $400,000 per month in storage costs. Separately, Databricks and OpenAI announced a multi-year partnership bringing OpenAI's frontier models, including GPT models and Codex, natively into the Databricks Data + AI Platform, callable via SQL or API against data governed by Unity Catalog.
Overview
OpenAI runs its marketing, analytics, security, and finance operations on the Databricks Data + AI Platform across Azure and AWS, with usage data landing in Delta tables governed by Unity Catalog. OpenAI's Security team uses Databricks streaming pipelines to standardize infrastructure, cloud, and vendor audit logs into governed Delta tables for threat detection, investigation, and incident response, including Codex agents querying governed security data via SQL APIs. The marketing team uses a Bronze-Silver-Gold architecture to build audiences without writing SQL for one billion weekly active users, eliminating $400,000 per month in storage costs. Separately, Databricks and OpenAI announced a multi-year partnership bringing OpenAI's frontier models, including GPT models and Codex, natively into the Databricks Data + AI Platform, callable via SQL or API against data governed by Unity Catalog.
This entry has 11 published fields tied to exact passages in an immutable source capture.
Inspect the highlighted sourceThe challenge
ChatGPT, the OpenAI API, and the internal tools used by thousands of OpenAI employees require a foundation that can withstand sudden growth, run governed analytics across multiple business units, and operate across multiple clouds simultaneously.
The solution
Since 2024, OpenAI has leveraged Databricks in Azure and AWS, with usage data landing in Delta tables governed by Unity Catalog. Databricks Jobs and Data Warehouses transform that data through layered pipelines to serve product analytics, finance, security, and trust and safety teams. The Security team uses streaming pipelines to standardize raw events into governed Delta tables for threat detection, investigation and incident response, with Codex agents able to query governed security data through SQL APIs. The marketing team uses a Bronze-Silver-Gold architecture to build audiences without writing SQL. Separately, Databricks and OpenAI announced a multi-year partnership bringing OpenAI's frontier models, including GPT models and Codex, natively into the Databricks Data + AI Platform, callable via SQL or API against data governed by Unity Catalog.
Reported business value
The marketing team's Bronze-Silver-Gold architecture eliminates $400,000 per month in storage costs, and Unity Catalog provides complete governance of OpenAI's entire multi-cloud data estate.
Sources
Open any source and check the claim yourself — that is the point of the register.
This record was researched and written with AI assistance, and its claims were checked against the sources above. (EU AI Act art. 50 transparency notice.)
Other technology & software entries in the register.
Transforming Weather Forecasting with Lakeflow Jobs
AccuWeather migrated from on-premises infrastructure to Databricks and Lakeflow Jobs, working with Datadog for observability, to unify diverse weather data formats and orchestrate 4,500+ weekly jobs. Lakeflow Jobs coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts, and manages complex job dependencies for reinforcement training workflows used in AccuWeather's proprietary forecasting engine. AccuWeather reports 3x faster dataset development (three months to one month per dataset), a 50% reduction in unactionable alerts, and 50% cost savings on serverless job usage.
AccuWeatherLakeflow Jobs · Delta Sharing · DatadogSupermetrics: Helping Marketers Redefine Efficiency with AI-Powered Data Analysis
Supermetrics, a Finland-based marketing intelligence platform serving 15,000+ customers across 132 countries, built an AI agent on Google Cloud using Vertex AI Agent Builder and the Agent Development Kit (ADK) that autonomously manages data connections, fixes pipeline errors, and analyzes campaign performance in real time, suggesting new creative options using Imagen. The agent automates the weekly marketing reporting cycle that previously took performance marketers up to four hours, reclaiming over 15 hours per month per marketer for strategy and creative testing. The system uses a central AI agent that interprets natural language requests and delegates tasks to sub-agents, and stores 'core memories' of user preferences for personalized context.
Supermetrics· FinlandGoogle Cloud · Vertex AI Agent Builder · Agent Development Kit (ADK) +1
Domyn builds Colosseum 355B, a sovereign AI foundation model, using NVIDIA DGX Cloud
Domyn (formerly iGenius), an Italian AI company serving highly regulated sectors such as financial services and public administration, used NVIDIA DGX Cloud with over 3,000 NVIDIA H100 GPUs to continue-pretrain Colosseum 355B, a 355-billion-parameter foundation LLM. Within one week Domyn had access to the dedicated infrastructure, and within two months completed continued pretraining, achieving 82.04% accuracy on the MMLU benchmark. The model powers Domyn's business intelligence agent, Crystal, a sovereign AI solution deployed on private infrastructure.
Strava's Athlete Intelligence Translates Workout Data into Simple and Personalized Insights
Strava launched Athlete Intelligence, an AI-powered feature available as a public beta to subscribers, which analyzes and interprets workout data across pace, heart rate, elevation, power, and Relative Effort into simple, personalized insights and guidance. The feature spots 30-day performance trends, detects milestones such as fastest pace or longest distance, and offers tailored feedback for each activity, drawing on more than 10 billion activity uploads on Strava.
StravaWas this helpful?
Your feedback helps us improve our use case database