NVIDIA A100 Tensor Core GPUs
2 use cases using this technology
Argonne National Laboratory accelerates cosmic discovery with AI-powered RADAR framework
Argonne National Laboratory
Argonne National Laboratory and university collaborators developed RADAR, a federated, privacy-enhancing framework that lets observatories coordinate gravitational-wave and radio follow-up without moving or exposing proprietary data. Running site-local AI inference on NVIDIA GPUs across the Polaris, Delta and DeltaAI supercomputers, RADAR processed over an hour (4,096 seconds) of Advanced LIGO data in under 4.5 minutes, a 5-10x speedup versus CPU workloads. The RADAR paper has been accepted for publication in The Astrophysical Journal Supplement Series.
Accelerating Large Language Model Inference with NVIDIA in the Cloud
Perplexity
Perplexity built pplx-api, an API for developers to integrate open-source LLMs with fast inference, served on Amazon EC2 P4d instances powered by NVIDIA A100 Tensor Core GPUs and accelerated with NVIDIA TensorRT-LLM (with a planned move to Amazon P5 instances with NVIDIA H100 GPUs). pplx-api achieves up to 3.1X lower latency and up to 4.3X lower first-token latency versus other deployment platforms, and switching external inference-serving API references to pplx-api lowered costs 4X, saving $600,000 per year. Using NVIDIA H100 GPUs and FP8 precision on Amazon P5 instances cuts latency in half and boosts throughput by 200 percent versus A100 GPUs in the same configuration. Perplexity also uses AWS's Kubernetes integration to scale elastically beyond hundreds of GPUs.