Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
Transforming Weather Forecasting with Lakeflow Jobs
…o Login Contact Us Try Databricks Customer Stories / AccuWeather CUSTOMER STORY Transforming weather forecasting with Lakeflow Jobs 3x Faster dataset development 50% Reduction in unactionable alerts 50% Cost sav…
Description
AccuWeather migrated from on-premises infrastructure to Databricks and Lakeflow Jobs, working with Datadog for observability, to unify diverse weather data formats and orchestrate 4,500+ weekly jobs. Lakeflow Jobs coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts, and manages complex job dependencies for reinforcement training workflows used in AccuWeather's proprietary forecasting engine. AccuWeather reports 3x faster dataset development (three months to one month per dataset), a 50% reduction in unactionable alerts, and 50% cost savings on serverless job usage.
…h and validate the datasets before making them available through Delta Sharing. The orchestration capabilities have been crucial for AccuWeather's advanced forecasting initiative, where Lakeflow Jobs coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts and manages the complex job dependencies required for reinforcement training workflows. By automating these previously manual orchestration tasks, Lakeflow Jobs enable…
Company
AccuWeather
AccuWeather | Databricks Skip to main content Login Why Databricks Discover For App Develop…
Problem
Faced with the challenge of ingesting, processing and storing immense volumes of diverse weather data at scale, AccuWeather's on-premises infrastructure forced regular purging of valuable data, preventing historical analysis of model performance, and numerous data sources without a single source of truth created expensive data silos and redundant processing costs.
…m our on-prem systems because we just don’t have capacity for it,” says Teague. The second challenge involved managing numerous data sources without a single source of truth. Multiple teams were pulling the same data for different purposes, creating expensive data silos and redundant processing costs. The situation was complicated by weather data’s unique characteristics, which c…
Solution
AccuWeather migrated to Databricks and selected Lakeflow Jobs as the primary workflow engine, working with Datadog for observability, to automate manual planning and validation processes. Lakeflow Jobs orchestrates seamless dependencies across 4,500+ weekly jobs, coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts, and manages complex job dependencies for reinforcement training workflows used in AccuWeather's proprietary forecasting engine.
…ather datasets. Unifying complex weather data with Databricks and Lakeflow Jobs AccuWeather migrated to Databricks to leverage cloud scalability, eliminate on-premises storage constraints and consolidate its fragmented data sources into a unified platform. To address its complex orchestration challenges, AccuWeather selected Lakeflow Jobs as the primary workflow engine to automate the manual planning and validation processes that were consuming days of effort each month. Lakeflow Jobs enabled AccuWeather to orchestrate seamless dependencies across 4…
Business value
3x Faster dataset development, a 50% reduction in unactionable alerts, and 50% cost savings on serverless job usage; one critical job converted entirely to serverless Lakeflow Jobs cut usage costs in half.
…AccuWeather CUSTOMER STORY Transforming weather forecasting with Lakeflow Jobs 3x Faster dataset development 50% Reduction in unactionable alerts 50% Cost savings on serverless job usage A…
…meet accuracy and completeness requirements before triggering downstream jobs. One critical job converted entirely to serverless Lakeflow Jobs cut usage costs in half, while reducing their reliance on Azure Data Factory has reduced both complexit…
Industry
Technology & Software
…save lives, protect property, and help people prosper.” Share this post Details Industry : Technology and Software Product : Delta Sharing , Lakeflow Jobs Ready to get started? Try Databricks fo…
Technology
Lakeflow Jobs, Delta Sharing, Datadog
…elp people prosper.” Share this post Details Industry : Technology and Software Product : Delta Sharing , Lakeflow Jobs Ready to get started? Try Databricks for free Talk to an expert Why Databricks…
AI capabilities
Predictive Analytics
…tive, where Lakeflow Jobs coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts and manages the complex job dependencies required for reinforcement training wo…
