DACH insurance group runs an on-prem Qwen3-32B LLM for claims and underwriting staff with no data leaving its own infrastructure

Overview

A large Germany-headquartered insurance group supervised by BaFin moved an AI pilot to production by running Qwen3-32B (AWQ 4-bit, vLLM) on two NVIDIA L40S GPUs in its own server room, with no outbound internet access. An internal RAG platform serves a copilot for claims and underwriting staff plus batch summarization and classification, with SSO-bound role-based access and a per-request audit trail. Built by Valletta Software.

The challenge

The group had run a proof of concept against an external frontier API, but employees had to strip personal data out by hand and paste in only fragments. Legal and information security refused production use with real documents because of GDPR obligations, BaFin expectations and DORA requirements, client non-disclosure agreements and an internal ban on sending documents or prompts to external AI providers.

The solution

Qwen3-32B Instruct, quantized to AWQ 4-bit and served with vLLM behind an OpenAI-compatible internal endpoint, runs on two NVIDIA L40S GPUs in the client's own server room in Germany with no direct outbound internet access. It is reached through an internal RAG platform over underwriting manuals, claims guidelines and policy documentation, with answers citing internal documents, SSO-bound role-based access and a per-request audit record.

Reported business value

The vendor reports, from actual request and token volume after launch, a three-year cost of 75,580 euro in-house against 392,400 euro for a comparable external API (about 81% lower) and hardware payback in 3.9 months. In a blind review of 600 tasks by the client's experts, 93.4% of on-prem answers were accepted against 94.6% for the frontier model. The system serves around 42,000 inference requests a day.