Human ResourcesLarge Language ModelsAI Model Development & MLOpsMachine Learning
Checkr: Streamlining Background Check Classification with Fine-tuned Small Language Models
Checkr
Checkr automated adjudication of background checks using a tuned logistic regression model for 98% of cases, but the remaining 2% required classifying noisy records into 230 categories. After testing GPT-4 (80-82% accuracy on hard cases, ~15s latency, ~$12k cost) and GPT-4 with RAG, Checkr fine-tuned Llama-2-7b (85% accuracy) and then, in production on Predibase, Llama-3-8b-instruct, achieving 90% accuracy on the hardest 2% of cases with 0.15-second response times (30x faster than GPT-4) and a 5x cost reduction, using LoRA/LoRAX for parameter-efficient fine-tuning and multi-adapter serving.
