{"slug":"university-hospital-halle-runs-a-locally-hosted-falcon-7b-rag-chatbot-for-internal-document-retrieval","url":"https://findausecase.com/use-cases/university-hospital-halle-runs-a-locally-hosted-falcon-7b-rag-chatbot-for-internal-document-retrieval","title":"University Hospital Halle runs a locally hosted Falcon-7B RAG chatbot for internal document retrieval","description":"A proof-of-concept at University Hospital Halle (Saale) hosts Falcon-7B-Instruct with retrieval-augmented generation on a local Linux server with one Nvidia H100 GPU, over internal SOPs, guidelines and forms. In a one-month test phase it handled 287 user queries; 75% of responses returned in under five seconds.","company":"University Hospital Halle (Saale)","industry":"Healthcare","country":"Germany","aiCapabilities":["Generative AI","Conversational AI"],"technology":["Falcon-7B-Instruct","jina-embeddings-v3","Nvidia H100"],"deployment":"On-Premise","problemStatement":"Healthcare has strict data protection and security requirements, so powerful proprietary models such as ChatGPT are not suitable even though they are increasingly used informally. The hospital wanted a way to interact with its extensive locally stored internal document library.","solutionApproach":"A locally hosted open-source LLM and retrieval-augmented generation (RAG) system on a local Linux server with an Nvidia H100 GPU, using the Falcon-7B-Instruct model. Internal documents, primarily SOPs, procedural guidelines and forms, were cleaned, contextually chunked and embedded with the jina-embeddings-v3 model, and the system retrieves documents using combined cosine similarity and BM25 scoring through a chatbot.","businessValue":"In a one-month test phase the proof-of-concept system handled 287 user queries, with 75% of responses returned in under five seconds. Experts rated textual answers 3.72 and retrieved documents 3.89 on a scale from one to five, and user feedback was mixed, which the authors attribute to the early prototype status. The authors conclude the system already offers a useful alternative to basic keyword search.","evidence":{"band":"high"},"sourceUrl":"https://conferences.publisso.de/en/conferences/gmds2025/25gmds197","dates":{"publishedAt":"2026-10-24T05:49:29.699Z","publishedAtSource":"pipeline","updatedAt":"2026-10-24T05:49:29.699Z"},"license":"Open for reading and citing with a link to https://findausecase.com/use-cases/university-hospital-halle-runs-a-locally-hosted-falcon-7b-rag-chatbot-for-internal-document-retrieval. Bulk republication requires permission."}