J.B. Hunt speeds up freight matching with a Databricks lakehouse on Google Cloud

Overview

J.B. Hunt built a unified BI and AI lakehouse platform for its J.B. Hunt 360 digital freight marketplace on Google Cloud using Databricks, Delta Lake and MLflow, paired with Immuta for automated data governance, to process real-time location pings and IoT telemetry from trucks and containers. The platform trains thousands of ML models in under four hours, delivers freight recommendations to carriers 99.8% faster than before, and has produced $2.7 million in IT infrastructure savings.

The challenge

J.B. Hunt's carrier world is deeply fragmented, with an estimated 3.5 million drivers across the industry, and its legacy enterprise data warehouse platforms locked up data, limiting its usability for real-time decision making. Data refreshes had only ever been available overnight at best, but trucks move constantly, and systems struggled to process and store massive data generated from location pings every 15 minutes across hundreds of thousands of loads, making telemetry-based ML and AI use cases nearly impossible.

The solution

J.B. Hunt worked with Google Cloud and Databricks Lakehouse to build an open, interoperable data platform for J.B. Hunt 360, using Delta Lake as an open storage layer to put all data in one place and stream real-time web, mobile, location and IoT data for faster analytics and ML, and MLflow to make code and experiments reproducible across data scientists. Immuta was layered on for automated data governance, including columnar-level data masking and global/local security policies, enabling secure self-service data analytics, data science and reporting across business analysts and engineers.

Reported business value

J.B. Hunt can now train thousands of ML models in less than four hours and deliver freight recommendations to carriers 99.8% faster than before, while realizing $2.7 million in IT infrastructure savings and productivity gains. Databricks has also enabled new capabilities like virtual track and trace feeding a predicted-time-of-arrival model, and brought data teams together to accelerate data science productivity.