First American Data & Analytics fine-tunes open-source LLMs on Databricks, cutting extraction costs 70%

Overview

First American Data & Analytics (DNA), a property-centric information provider to the real estate and title insurance industries, used the Databricks Data + AI Platform to fine-tune open-source LLMs (Mistral, Llama 3, Llama 3.1) for page-level classification and extraction of 100+ data fields from roughly 4 million historical title policy images dating to the 1930s. Using Delta Lake, Databricks Model Training, Managed MLflow, and serverless Model Serving on about 50 GPUs, the team fine-tuned models in two weeks versus the two-plus months it previously took on other platforms. The project cut extraction costs by 70%, reduced project completion time by 75% (months to weeks), and increased the number of data elements extracted tenfold.

The challenge

First American Data & Analytics maintains the industry's largest data repository of publicly recorded document images, over 8 billion unstructured documents, most stored with very limited metadata. The team received a set of roughly 4 million historical title policy images dating to the 1930s that needed page-level classification and extraction of more than 100 data elements per page, and earlier LLM approaches required huge training datasets and long fine-tuning times for these complex use cases.

The solution

Working with Databricks, First American DNA fine-tuned several open-source LLMs, including Mistral, Llama 3 and Llama 3.1, using Databricks Model Training on Delta Lake for data storage, Managed MLflow to streamline the LLM lifecycle, and roughly 50 GPUs for training; fine-tuned models were registered in Unity Catalog, and serverless Model Serving eliminated GPU-availability bottlenecks for deployment.

Reported business value

The fine-tuning approach cut extraction costs by 70% compared with a previous implementation, reduced data-extraction project time by 75% (from several months to weeks), and increased the number of data elements extracted tenfold; the fine-tuning step itself, which previously took over two months, was completed in two weeks.