Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
RDSolutions empowers retail pricing intelligence with Databricks
…ocessing (from 4 days to 3 hours) Billions of data records stored and processed RDSolutions provides retail pricing intelligence, collecting billions of data points daily. Their legacy data pipeline faced scalability, maintenance and slow data process…
Description
RDSolutions migrated its legacy Azure data warehouse pipeline to the Databricks Data + AI Platform, using Delta Lake, Unity Catalog and Databricks Assistant to process billions of retail pricing data points, cutting a data validation job from 4 days to 3 hours, a 97% improvement.
…Case : Artificial Intelligence , Data Science , Data Engineering Cloud : Azure Product : Delta Lake , Agent Bricks , Unity Catalog Ready to get started? Try Databricks for free Learn more about our product Talk…
…ample of the speed at which RDSolutions now moves is their data validation job. The job took four full days to land the data, validate it, clean bad records and add new ones within the legacy data warehouse. Immediately after switching to Databricks and building the job in a workflow, it was reduced to seven hours, and since optimizing the pipeline, the job now runs in just three hours. That’s a 97% process and performance improvement directly impacting the speed a…
…hours, and since optimizing the pipeline, the job now runs in just three hours. That’s a 97% process and performance improvement directly impacting the speed at which RDSolutions delivers accurate and reliable customer data. Equipped with much faster processing speeds, RDSolutions looks forward to expan…
Company
RDSolutions
…ts to their customers. Struggling to scale data within complex legacy pipelines RDSolutions conducts 65,000+ online audits and thousands of weekly in-store audits to normalize and collect more than 500 billion data points weekly. To compile, process and distribute this decision-making data to big-box retail…
Industry
Retail
…changer and will continue to be pivotal in our growth.” Share this post Details Industry : Financial Services , Retail and Consumer Goods Use Case : Artificial Intelligence , Data Science , Data Engineering Cloud : Az…
Problem
RDSolutions relied on an Azure data warehouse that failed to scale cost-efficiently or effectively; big queries forced data teams to stop other work, and a complex aggregation job failed to complete, signaling the legacy pipeline could not meet leadership's ambitions for scale.
…le, process and distribute this decision-making data to big-box retail clients, RDSolutions relied on an Azure data warehouse, but the environment failed to scale cost-efficiently or effectively. For big queries to run, data teams had to stop all other work, stalling product…
…use case involving a complex aggregation calculation spanning multiple months. The job failed to complete, highlighting the system’s inability to handle expanding data needs and signaling that a new solution was necessary. RDSolutions needed to modernize their data platform to meet leadership’s ambiti…
Solution
RDSolutions migrated to the Databricks Data + AI Platform, making Delta Lake the backbone of its data strategy for structured and unstructured data, using Unity Catalog for a single data governance and access-control model, Databricks SQL for self-service analyst queries, and Databricks Assistant to automate development tasks like error troubleshooting and table documentation.
…concept to complete migration to Delta Lake was rapid, taking only six months. Now the backbone of RDSolutions’s data strategy, Delta Lake enables seamless integration of structured and unstructured data — from grocery items to automotive parts. The platform’s scalability allows RDS…
…oncerns about performance degradation, even as data volumes grow exponentially. Unity Catalog primarily functions as a data governance tool for managing access and controlling data sharing across multiple workspaces. Now, Featherstone and his team can rely on a single permissions model that simp…
…he development process and improved team efficiency, regardless of skill level. Databricks Assistant helps automate tasks such as error troubleshooting and table documentation, allowing teams to focus on higher-value work. Featherstone said, “One of the best things about Databricks Assistant is how it…
Business value
A data validation job that took four full days in the legacy warehouse was reduced to seven hours immediately after switching to Databricks, and to just three hours after optimizing the pipeline—a 97% process and performance improvement directly impacting the speed at which RDSolutions delivers accurate and reliable customer data.
…hours, and since optimizing the pipeline, the job now runs in just three hours. That’s a 97% process and performance improvement directly impacting the speed at which RDSolutions delivers accurate and reliable customer data. Equipped with much faster processing speeds, RDSolutions looks forward to expan…
AI capabilities
Generative AI, Intelligent Automation
…tton. So, that alone has made things significantly easier and more productive.” This integration of generative AI (GenAI) capabilities has empowered RDSolutions to be proactive, swiftly adapt and innovate without technical limitations. Rapid time to insights and limitless scalability for the future Since migrating…
Technology
Delta Lake, Unity Catalog, Agent Bricks, Databricks Assistant
…Case : Artificial Intelligence , Data Science , Data Engineering Cloud : Azure Product : Delta Lake , Agent Bricks , Unity Catalog Ready to get started? Try Databricks for free Learn more about our product Talk…
Deployment model
cloud
…umer Goods Use Case : Artificial Intelligence , Data Science , Data Engineering Cloud : Azure Product : Delta Lake , Agent Bricks , Unity Catalog Ready to get started? Try D…
Deployment options
cloud
…umer Goods Use Case : Artificial Intelligence , Data Science , Data Engineering Cloud : Azure Product : Delta Lake , Agent Bricks , Unity Catalog Ready to get started? Try D…
Headline outcome
…hours, and since optimizing the pipeline, the job now runs in just three hours. That’s a 97% process and performance improvement directly impacting the speed at which RDSolutions delivers accurate and reliable customer data. Equipped with much faster processing speeds, RDSolutions looks forward to expan…
Use case type
Workflow automation
…he development process and improved team efficiency, regardless of skill level. Databricks Assistant helps automate tasks such as error troubleshooting and table documentation, allowing teams to focus on higher-value work. Featherstone said, “One of the best things about Databricks Assistant is how it…
