University Hospital Bonn runs an isolated on-premise open-weights LLM service for radiology

Overview

University Hospital Bonn built a self-hosted, isolation-first LLM inference stack on a single on-premise server with eight NVIDIA A100 GPUs, serving DeepSeek-R1 via vLLM, and obtained governance approval to process unanonymized patient data. In a one-week pilot, 22 radiologists used ten prompt templates; source-anchored tasks such as report corrections and simplifications rated highest, while open-ended conclusion generation produced the most critical errors. The service is described as the base for an official service for a hospital with over 10,000 employees.

The challenge

Data protection and information security requirements remain a major barrier to integrating LLMs into clinical workflows: popular proprietary LLMs are offered through cloud APIs that require data to leave the hospital network, raising confidentiality, sovereignty and regulatory compliance concerns.

The solution

The hospital built an isolation-first, containerized LLM inference stack on a single on-premise server with eight NVIDIA A100 80 GB GPUs, using strict network segmentation, host-enforced egress filtering and active isolation monitoring. It serves the open-weights DeepSeek-R1 model via vLLM behind an Open WebUI chat frontend, with users given ten predefined prompt templates.

Reported business value

The governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized protected health information. In a one-week pilot with 22 radiologists, the system was rated stable and user friendly, source text-anchored tasks such as report corrections or simplifications received the highest utility ratings, and open-ended conclusion generation produced the most critical errors. The authors describe it as the current base for an official service for a German university hospital with over 10,000 employees.