MediaTek Accelerates AI Development With an AI Factory

Overview

MediaTek established an on-premises AI factory powered by NVIDIA DGX SuperPOD with NVIDIA Blackwell-based systems to accelerate enterprise AI efforts, including development of its Breeze series LLMs and a 480-billion-parameter traditional-Chinese model. The AI factory processes approximately 60 billion tokens per month for inference and completes over 24,000 model-training iterations monthly, training models exceeding 480 billion parameters within one week (versus 7-billion-parameter models in a week previously). Using NVIDIA NIM and TensorRT-LLM, MediaTek achieved a 40% improvement in inference speed and 60% increase in token throughput. NVIDIA Mission Control consolidated GPU provisioning and system monitoring, while AI-assisted code completion and an AI agent for chip design documentation reduced documentation time from weeks to days. NVIDIA Riva was integrated into NVIDIA DGX Spark for agentic voice control features like internet search, calendar and messaging.

The challenge

MediaTek needed a robust, scalable and cost-effective computing environment to handle thousands of model-training iterations monthly and billions of tokens processed for inference, while exploring newer models on local machines without exposing proprietary data, ensuring data security and compliance.

The solution

MediaTek established an on-premises AI factory powered by NVIDIA DGX SuperPOD with NVIDIA Blackwell-based systems, incorporating the DGX SuperPOD reference architecture into its data center designs. It leverages NVIDIA NIM and TensorRT-LLM for inference, NVIDIA Mission Control for GPU provisioning and system monitoring, NVIDIA NeMo to fine-tune LLMs, and NVIDIA Riva for agentic voice control (ASR/TTS) integrated into NVIDIA DGX Spark. MediaTek also integrated AI-assisted code completion and an AI agent for chip design documentation that extracts information from design flowcharts and state diagrams.

Reported business value

MediaTek's AI factory processes approximately 60 billion tokens per month for inference and completes over 24,000 model-training iterations monthly, training models exceeding 480 billion parameters within one week versus 7-billion-parameter models in a week previously. NVIDIA NIM and TensorRT-LLM delivered a 40% improvement in inference speed and 60% increase in token throughput. Documentation time was reduced from weeks to days through AI-assisted generation.