FactSet cuts code-generation response time 70% with a standardized Databricks LLMOps framework

Overview

FactSet, a financial data and analytics provider, standardized its GenAI development on Databricks Mosaic AI and managed MLflow after fragmented tooling across teams caused collaboration and governance problems. For its FactSet Mercury code-generation feature, FactSet fine-tuned meta-llama-3-70b and Databricks DBRX models, reducing average response latency by more than 70%. Its Text-to-Formula project reduced end-to-end latency by about 60% using a compound AI architecture with fine-tuned open-source models.

The challenge

FactSet's early GenAI adoption was fragmented: engineers across teams used diverse tools (cloud-native commercial offerings, specialized fine-tuning services, on-premises solutions), creating collaboration barriers, duplicated effort, and inconsistent model quality. Data was scattered across teams with poor lineage and governance, and multiple serving layers made model governance and monitoring cumbersome.

The solution

FactSet selected Databricks as its enterprise ML/AI platform in late 2023, standardizing new LLM and AI application development on Databricks Mosaic AI and Databricks-managed MLflow. It used Unity Catalog for hierarchical, fine-grained governance and per-project isolation (catalog, schema, service principal, volume), and built a cross-business-unit GenAI Hub integrating Databricks workspaces, the Model Catalog and cost-attribution. For its Mercury code-generation feature, FactSet fine-tuned meta-llama-3-70b and later Databricks DBRX. For its Text-to-Formula project, it moved from a simple RAG workflow to a compound AI architecture combining fine-tuned proprietary and open-source models.

Reported business value

Fine-tuning meta-llama-3-70b and DBRX for Mercury code generation reduced average user request latency by more than 70%. The compound AI architecture for Text-to-Formula reduced end-to-end latency by about 60%. FactSet's model inference cost analysis for its Transcript Chat Product suggested significant cost savings from fine-tuned open-source models versus commercial LLM alternatives, though training costs were not included in that comparison.