AT&T Drives AI Agents' Accuracy, Efficiency, and Performance With NVIDIA

Overview

AT&T built 'Ask AT&T' customer service AI agents and worked with implementation partner Quantiphi to use NVIDIA AI Enterprise, NVIDIA NeMo and NIM microservices to build a data-flywheel platform for continuous fine-tuning and evaluation. The pipeline uses NeMo Curator to clean training data, NeMo Customizer to fine-tune base models (Mistral 7B was selected as optimal), NeMo Evaluator to measure performance (Rouge, BERT F1), and NeMo Retriever for up-to-date retrieval, with models deployed as NIM microservices. AT&T reports up to 40% improvement in response accuracy after fine-tuning and an 84% decrease in call center analytics cost, and is collaborating with Arize AI to automate identification of difficult AI interactions.

The challenge

Facing challenges like model drift, rising computational demands, and the need for real-time data access, AT&T needed to overcome key challenges: reducing latency, lowering operational costs, and enhancing the accuracy of the models powering its AI agents. With nearly 10,000 documents updated multiple times a week, AI agents need to be updated with the most current information to remain effective, and managing the complexity of multi-agent AI systems was also a pressing concern.

The solution

AT&T developed 'Ask AT&T' AI agents and worked with implementation partner Quantiphi to leverage NVIDIA AI Enterprise, including NVIDIA NeMo and NIM microservices, to implement a data flywheel approach for continuously enhancing AI agent performance: NeMo Curator cleanses and filters training data; NeMo Customizer fine-tunes base models (Mistral 7B emerged as the optimal performer after testing against Mixtral and Llama); NeMo Evaluator measures performance via metrics like Rouge, BERT F1, question relevance and answer quality; NeMo Retriever gives agents up-to-date access to AT&T's repository so the agents can make decisions and take actions using up-to-date information; and the resulting models are deployed as NIM microservices for optimized inference on GPU-accelerated infrastructure.

Reported business value

AI agent responses showed up to 40% improvement in accuracy with post-training in key metrics, particularly in Rouge and BERT F1 scores, and the deployment of Ask AT&T with NVIDIA NIM and NeMo enabled AT&T to realize an 84% decrease in call center analytics cost. AT&T is also collaborating with Arize AI to automate identification and handling of difficult AI interactions.