Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
University Hospital Bonn runs an isolated on-premise open-weights LLM service for radiology
Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation Skip to main content Report GitHub Issue × Title: Content selection saved. Desc…
Description
University Hospital Bonn built a self-hosted, isolation-first LLM inference stack on a single on-premise server with eight NVIDIA A100 GPUs, serving DeepSeek-R1 via vLLM, and obtained governance approval to process unanonymized patient data. In a one-week pilot, 22 radiologists used ten prompt templates; source-anchored tasks such as report corrections and simplifications rated highest, while open-ended conclusion generation produced the most critical errors. The service is described as the base for an official service for a hospital with over 10,000 employees.
…inical evaluation. 2 Materials and Methods 2.1 System Architecture and Hardware The infrastructure is deployed on a single on-premise server within the hospital data center equipped with eight NVIDIA A100 80 GB PCIe graphics processing units (GPUs). Docker compose was used for improved reproducibility, modularity, operational c…
…companying deployment package provides automated isolation and hardening tests. The system served the open-weights DeepSeek-R1 model via vLLM. In a one-week pilot phase, 22 residents and radiologists were free to use 10 pr…
…hardening tests. The system served the open-weights DeepSeek-R1 model via vLLM. In a one-week pilot phase, 22 residents and radiologists were free to use 10 predefined prompt-templates whenever they considered them useful in daily work. Afterward, they rated clinical utility and system stability on an 0-10 Likert s…
…egulatory borders, showed promising clinical utility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available ( https://github.com/ukbonn…
Company
University Hospital Bonn
…rst Architecture with Prospective Pilot Evaluation Sebastian Nowak Affiliation: Department of Diagnostic and Interventional Radiology, University Hospital Bonn Affiliation: Corresponding author: sebastian.nowak@ukbonn.de Jann-Frederick Laß…
Country
Germany
…ility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available ( https://github.com/ukbonn…
Industry
Healthcare
…ility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available ( https://github.com/ukbonn…
Problem
Data protection and information security requirements remain a major barrier to integrating LLMs into clinical workflows: popular proprietary LLMs are offered through cloud APIs that require data to leave the hospital network, raising confidentiality, sovereignty and regulatory compliance concerns.
…ntially replacing expensive dictation tool licenses with open alternatives [7]. However, data protection and information security requirements remain a major barrier to integration of LLMs into clinical workflows. Well-known propretiary LLMs like ChatGPT and Gemini are offered through cloud-b…
…irements remain a major barrier to integration of LLMs into clinical workflows. Well-known propretiary LLMs like ChatGPT and Gemini are offered through cloud-based application programming interfaces (APIs), which require data to leave the hospital network. For protected health information, this raises confidentiality, sovereignty, and…
Solution
The hospital built an isolation-first, containerized LLM inference stack on a single on-premise server with eight NVIDIA A100 80 GB GPUs, using strict network segmentation, host-enforced egress filtering and active isolation monitoring. It serves the open-weights DeepSeek-R1 model via vLLM behind an Open WebUI chat frontend, with users given ten predefined prompt templates.
…al feasibility, network isolation, and clinical utility. Materials and Methods: The isolation-first, containerized LLM inference stack relies on strict network segmentation, host-enforced egress filtering, and active isolation monitoring preventing unauthorized external connectivity. An accompanying deployment package provides automated isolation and hardening t…
…es four local components: i) Nginx proxies for routing and access control [13], ii) an OpenWebUI frontend for chat and user management [14], iii) a vLLM inference backend [9,15], iv) and a Prometheus/Grafana monitoring stack [16,17]. Figure 1 illustrates th…
Business value
The governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized protected health information. In a one-week pilot with 22 radiologists, the system was rated stable and user friendly, source text-anchored tasks such as report corrections or simplifications received the highest utility ratings, and open-ended conclusion generation produced the most critical errors. The authors describe it as the current base for an official service for a German university hospital with over 10,000 employees.
…10 Likert scale and reported observed critical errors in model output. Results: The applied institutional governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized PHI. The system was rated stable and user friendly during the pilot. Source text-anc…
…a protection and information security officers for processing unanonymized PHI. The system was rated stable and user friendly during the pilot. Source text-anchored tasks, such as report corrections or simplifications, and…
…egulatory borders, showed promising clinical utility in text-anchored tasks and is the current base to serve open-weights LLMs as an official service of a German University Hospital with over 10,000 employees. The deployment package were made publicly available ( https://github.com/ukbonn…
AI capabilities
Generative AI, Conversational AI
…ven higher token throughput and content precision and thereby clinical utility. Finally, the current deployment evaluates an interactive, chat-based LLM setting. Future applications will likely involve autonomous AI agents that complete mult…
Technology
DeepSeek-R1, vLLM, Open WebUI
…companying deployment package provides automated isolation and hardening tests. The system served the open-weights DeepSeek-R1 model via vLLM. In a one-week pilot phase, 22 residents and radiologists were free to use 10 pr…
…es four local components: i) Nginx proxies for routing and access control [13], ii) an OpenWebUI frontend for chat and user management [14], iii) a vLLM inference backend [9,15], iv) and a Prometheus/Grafana monitoring stack [16,17]. Figure 1 illustrates th…
Deployment model
On-premises
…inical evaluation. 2 Materials and Methods 2.1 System Architecture and Hardware The infrastructure is deployed on a single on-premise server within the hospital data center equipped with eight NVIDIA A100 80 GB PCIe graphics processing units (GPUs). Do…
Deployment options
On-premise
…inical evaluation. 2 Materials and Methods 2.1 System Architecture and Hardware The infrastructure is deployed on a single on-premise server within the hospital data center equipped with eight NVIDIA A100 80 GB PCIe graphics processing units (GPUs). Do…
Applicable regulations
EU AI Act
…itoring purposes and that use of the system would remain voluntary for staff. • Compliance Department identified relevance of the EU AI Act 2024/1689 [19], defining a model deployer rather than provider role. This focused governance on operational use, human oversight, AI transparency, a…
Implementation approach
A one-week real-world pilot with 22 radiologists was run in July 2025 after Institutional Review Board approval, in parallel with a governance workstream covering hospital management, the data protection officer, the information security officer, the compliance department and the works council.
…and incorporated into the deployment concept. 2.6 Study Design and Participants We conducted a real-world pilot evaluation over a one-week period in July 2025, following Institutional Review Board approval (2025-167-BO). Twenty-two radiologists from the University Hospital Bonn participated in the s…
…e system as radiology text support without autonomous clinical decision-making. Relevant stakeholders included hospital management, the data protection officer, the information security officer, the compliance department, and works council. Required technical and organizational measures were collected and incorporated…
Onboarding approach
Compliance required a mandatory AI literacy course before access; completion is linked to a dedicated Active Directory or LDAP group that is a prerequisite for login.
…is focused governance on operational use, human oversight, AI transparency, and user qualification via a mandatory AI literacy course prior to access. We developed the training materials and linked successful completion to assignment to a dedicated Active Directory (AD) or LDAP group, which is configured in Open WebUI as a prerequisite for login. 3.3 In-Field Evaluation of Locally Deployed LLMs in Clinical Routine To ensure…
Headline outcome
…10 Likert scale and reported observed critical errors in model output. Results: The applied institutional governance pathway achieved approval from clinic management, compliance, data protection and information security officers for processing unanonymized PHI. The system was rated stable and user friendly during the pilot. Source text-anc…
Use case type
Conversational assistant
…administrative and documentation support, and (3) general communication tasks. Templates were integrated into the chat interface as one-click starting points to reduce interaction variability and ensure that participants assessed comparable functions. During the one-week…
