MNP deploys a fine-tuned LLM with RAG on Databricks to deliver client insights in weeks

Overview

MNP, a Canadian professional services firm, used the Databricks Data + AI Platform, AI Search and Foundation Model APIs to deploy a retrieval-augmented generation system on a Mixtral 8x7B model, letting client-facing teams get contextual, up-to-date insights from structured and unstructured data. Supported by the Databricks GenAI Advisory Program, MNP's data team built a model from start to QA testing within four weeks and reached GenAI solution deployment in six weeks.

The challenge

MNP's conventional data and analytics platforms could not support the processing required for advanced data analytics and ML applications; their initial foundation model deployment was tightly coupled to the existing data warehouse, suffered lag from a poor time-to-first-token metric, and had prohibitive total cost of ownership from heavy GPU usage and frequent retraining.

The solution

MNP consolidated structured, semi-structured and unstructured data into the Databricks Data + AI Platform, adopted AI Search as a serverless vector database for data embeddings, and after evaluating models selected Mixtral 8x7B, a mixture-of-experts model, choosing retrieval augmented generation (RAG) as the preferred refinement strategy to work with datasets that required regular updates; MNP used Databricks Foundation Model APIs and Model Serving to deploy and manage the LLM, supported by the Databricks GenAI Advisory Program. An earlier prototype had been fine-tuned on Llama 2 13B and 70B but was superseded by this RAG-based Mixtral 8x7B deployment.

Reported business value

MNP's data team built a model from start to quality assurance testing within four weeks and reached GenAI solution deployment in six weeks, enabling faster, more contextual and precise client insights while maintaining information security through Private AI standards and Unity Catalog governance.