University Hospital Bonn deploys an isolated on-premise open-weights LLM for radiology, piloted with 22 radiologists

Overview

University Hospital Bonn built an isolation-first, self-hosted LLM service on a single on-premise server with eight NVIDIA A100 80 GB GPUs, serving DeepSeek-R1 via vLLM behind Open WebUI, with network segmentation and egress filtering so unanonymized patient data can be processed. In a one-week pilot in July 2025, 22 radiologists used ten prompt templates; stability was rated high and source-anchored tasks such as translation and report correction rated best, while open-ended conclusion generation produced the most critical errors. Median time-to-first-token was 3.13 seconds. The authors state the system is the basis for an official service for a hospital with over 10,000 employees.

The challenge

Data protection and information security requirements are a major barrier to using LLMs in clinical workflows: well-known proprietary LLMs are offered through cloud APIs that require data to leave the hospital network, and hospitals in many countries do not permit external LLM services with clinical data.

The solution

The hospital designed an isolation-first, containerized LLM inference stack on a single on-premise server with eight NVIDIA A100 80 GB PCIe GPUs. It relies on strict network segmentation, host-enforced egress filtering and active isolation monitoring, and serves the open-weights DeepSeek-R1 model via vLLM behind an Open WebUI frontend. The architecture is model-agnostic, and the intended use is radiology text support without autonomous clinical decision-making.

Reported business value

In a one-week pilot with 22 radiologists, participants rated system stability exceptionally high and usability strong, and no external connectivity events were detected. Source-anchored tasks such as report translation, linguistic correction and radiology guideline recommendations received the highest ratings, while open-ended conclusion generation produced the most critical errors. The authors note it was a single-centre pilot with user-reported errors.