FindAUseCase.com

Databricks

Spark Declarative Pipelines

Apache Spark Declarative Pipelines (part of Databricks Lakeflow) simplifies batch and streaming ETL: engineers declare the data transformations they need while the platform automatically handles ingestion from any Apache Spark-supported source, dependency management, scaling and recovery, and data quality rules via Expectations. It offers unified batch and streaming processing, end-to-end incremental processing, an integrated pipeline development IDE, and is built on Unity Catalog and open table formats.

Official product page
27published use cases
13industries
8countries on record
Predictive Analytics
top AI capability

Evidence mix: High 27 · Medium 0 · Low 0 — bands are computed from each record's evidence signals.

Industry
Country

27 use cases

RetailRecommendation & PersonalizationPredictive Analytics

Ahold Delhaize trains ML models for personalization and inventory forecasting across brands

Ahold Delhaize, one of the world's largest food retail groups (7,452+ stores across multiple countries), built a self-service data platform on Databricks Lakeflow Jobs and Auto Loader that lets data engineers across its brands build pipelines for AI/ML needs. Etos uses the platform for machine learning and model training for personalization and inventory forecasting; Albert Heijn's ML engineering team uses Lakeflow Jobs to orchestrate feature engineering and continuous ML model training/retraining pipelines. The platform runs about 1,165 data ingestion jobs daily, cut deployment time 4.5x (1.5 hours to 20 minutes), and reduced costs by 50% through cluster reuse.

Ahold DelhaizeLakeflow Connect · Lakeflow Jobs · Spark Declarative Pipelines +3
Technology & SoftwareFraud & Anomaly Detection

InMobi uses machine learning for anomaly detection in ad data pipelines

InMobi, a mobile advertising platform, migrated from a multicloud data warehouse to the Databricks Data + AI Platform to unify data warehousing, AI and analytics workloads. Using Spark Declarative Pipelines for anomaly detection, InMobi achieved a 50% improvement in SLAs and an 80% reduction in costs. Teams use MLflow to build their next-generation AI platform and are exploring large language models to let end users query data conversationally. The migration also delivered 32% lower total cost of ownership, 15% faster queries, and 20% better reporting performance versus their prior multicloud data warehouse.

InMobiDatabricks SQL · Spark Declarative Pipelines · Unity Catalog +1
Technology & SoftwareLarge Language ModelsConversational AI

Crisis Text Line fine-tunes an LLM-powered conversation simulator on Databricks to train crisis counselors

Crisis Text Line, a nonprofit text-based mental health crisis service supporting 1.3 million conversations a year, used Databricks MLflow and fine-tuned versions of Llama 2, trained on synthetic conversation role-plays created by clinical staff, to build an LLM-powered conversation simulator that lets new volunteer crisis counselors practice engagement strategies without real-world repercussions. The simulator has been used by over 50 volunteers and clinicians, and the organization is also developing a conversation-phase classifier to help assess response quality.

Crisis Text LineMLflow · Spark Declarative Pipelines · Unity Catalog +2
RetailGenerative AIPredictive Analytics

Cultivating the Perfect Lawn for 2.3 Million Customers

TruGreen migrated to the Databricks Data + AI Platform, using Databricks SQL on Azure for its data warehouse, Unity Catalog for governance, Lakeflow Spark Declarative Pipelines to stream data from Dynamics 365, and Qlik Replicate for legacy ERP integration. It built TruSight, an AI-powered reporting solution delivering daily insights to 200+ branch and regional managers before crews start work at 7 AM, evaluated against ChatGPT and multiple Microsoft Copilot variants before selecting Databricks for accuracy, cost, and control over context and LLM choice. TruGreen deployed Claude Sonnet 4.5 via Databricks Model Serving (External Models), using MLflow's LLM judges to evaluate AI output quality, and has churn and lifetime-value machine learning models in production. Databricks SQL cut data warehousing costs by roughly 50%, and automated data lineage in Unity Catalog reduced ERP impact analysis from weeks to minutes (a 99% improvement).

TruGreenAgent Bricks · Databricks SQL · Spark Declarative Pipelines +4
Media & EntertainmentConversational AI

Amagi Ensures Millions Never Miss Their Shows

Amagi, a media technology company supporting 8,000+ streaming channels and 300+ distributors, consolidated fragmented data systems (Dataproc, Snowflake, and in-house platforms) across AWS and GCP into the Amagi Data Platform (ADP), a unified lakehouse built on Databricks with Unity Catalog governance. The platform serves real-time data for ad decisioning, content performance analytics, billing and customer reporting to internal teams (via notebooks, Genie, SQL warehouses) and customer-facing products. Amagi reports 45% cost savings from consolidating fragmented data systems, 15x faster time to market for new data products, and 35% fewer production incidents due to improved data reliability.

AmagiDatabricks SQL · Delta Lake · Spark Declarative Pipelines +2
EducationMachine LearningFraud & Anomaly DetectionAI Model Development & MLOps

GoGuardian: Safer schools, empowered teachers, thriving students

GoGuardian, which powers safe, focused learning for half of U.S. K-12 students, migrated its ML infrastructure to Databricks to manage billions of daily inferences for web filtering, classroom management and harm prevention while maintaining a PII-free, COPPA/FERPA-compliant data environment. Using Delta Lake, Lakeflow, Unity Catalog, MLflow and Databricks Model Serving, GoGuardian achieved up to 50% reduction in machine learning operational costs, 90% operational cost savings with its Delphi website classification model, and a 62% reduction in inappropriate device use among students. AI-driven prioritization also cut the volume of records requiring human review for high-risk content by over 95%, from 1 million to 35,000-45,000.

GoGuardian· United StatesDelta Lake · Databricks Lakeflow · Unity Catalog +6