Source proof for GDS pilots GOV.UK Chat, an AI assistant, with more than 10,000 users, reaching 90% answer accuracy

…Chat, an AI assistant for GOV.UK built at the Government Digital Service (GDS). More than 10,000 users asked 26,000 questions about government services, from tax to benefits to visas. We measured accuracy, reliability, speed, safety and user trust. Together, thes…

…s to the work of our data scientists, and advances in the underlying AI models, we’ve been able to increase answer accuracy scores from 76%, our earliest benchmark, to our latest figure of 90% accuracy across all topics. We assess answers using a combination of subject matter experts, and automated…

…ed on: 16 March 2026 - Categories: Data , User insights , What we're working on Over the last 18 months, we’ve run 2 public pilots of GOV.UK Chat, an AI assistant for GOV.UK built at the Government Digital Service (GDS). More than 10,000 users asked 26,000 questions about government services, from t…

…using a combination of subject matter experts, and automated evaluation tools. Chat exclusively draws on guidance published on GOV.UK, and our assessors only rate an answer as ‘accurate’ if it meets all the standa…

…-step lists – for some, saving them a phone call to a department’s call centre. In a follow-up survey of GOV.UK app users, 73% found GOV.UK Chat useful, and 64% were satisfied with GOV.UK Chat. We’re pleased with these numbers, and think we can increase them further with i…

…t to ask users clarifying questions where the original questions are ambiguous. Our answer rate for in-scope questions is now 88% with plans to improve further. We also found some people wanted to speak to an advisor with access to their pe…

…hing, and consequently GOV.UK Chat responses are slower than we’d ideally like; an average 10.7 seconds for an answer. We found this to be within acceptable bounds for users of GOV.UK Chat, and sti…

…coped comfortably with demand, which bodes well for our ability to scale it up. We are now using Amazon’s Bedrock platform and Anthropic’s Claude models to power the latest version of GOV.UK Chat. The system is built to allow us to upgrade to new models as they become availab…

…vernment – Inside GOV.UK Skip to main content GOV.UK Blog : Inside GOV.UK Organisations: Government Digital Service Search blog This blog post was published under the 2024 to 2026 Starmer governm…

…V.UK Blog : Inside GOV.UK Organisations: Government Digital Service Search blog This blog post was published under the 2024 to 2026 Starmer government https://insidegovuk.blog.gov.uk/2026/03/16/5-things-we-learned-testing-gov-uk-c…

…e working on Over the last 18 months, we’ve run 2 public pilots of GOV.UK Chat, an AI assistant for GOV.UK built at the Government Digital Service (GDS). More than 10,000 users asked 26,000 questions about government services, from…

…vailable to all users of the GOV.UK app. 5 headline findings from the pilots 1. GOV.UK Chat offers users a faster way to get answers Users told us GOV.UK Chat saves them time. Participants said that GOV.UK Chat’s…

…sked 26,000 questions about government services, from tax to benefits to visas. We measured accuracy, reliability, speed, safety and user trust. Together, these pilots represent one of the largest user research exercises GDS…

…tion of the service. 4. We’re improving how we handle questions we can’t answer By design, GOV.UK Chat won’t provide answers outside the scope of government guidance, and that means there will always be occasions when users’ questions can’t be a…

…confidence, and so in some cases didn’t receive an answer at all. To fix this, we introduced the ability for Chat to ask users clarifying questions where the original questions are ambiguous. Our answer rate for in-scope questions is now 88% with plans to improve further…

…tform and Anthropic’s Claude models to power the latest version of GOV.UK Chat. The system is built to allow us to upgrade to new models as they become available, and this will form part of the ongoing operation and iteration of the service.…

…priate or harmful response. This is an ongoing risk we will need to manage, but the safety guardrails we’ve put in place successfully prevented all these attempts. You can read more about our work to prevent jailbreaking in our previous blog…

…s to the work of our data scientists, and advances in the underlying AI models, we’ve been able to increase answer accuracy scores from 76%, our earliest benchmark, to our latest figure of 90% accuracy across all topics. We assess answers using a combination of subject matter experts, and automated…

…-step lists – for some, saving them a phone call to a department’s call centre. In a follow-up survey of GOV.UK app users, 73% found GOV.UK Chat useful, and 64% were satisfied with GOV.UK Chat. We’re pleased with these numbers, and think we can increase them further with i…

…ic-facing Large Language Models (LLMs) reliably at scale During the two pilots, there were 508 attempts to ‘jailbreak’ GOV.UK Chat. These are attempts to trick an AI system to provide an inappropriate or harmful…

…t to ask users clarifying questions where the original questions are ambiguous. Our answer rate for in-scope questions is now 88% with plans to improve further. We also found some people wanted to speak to an advisor with access to their pe…

…ll be a substantial bit of work for the team to implement. Our research methods For the first pilot, we recruited 10,136 users who asked 23,838 questions of a web version of GOV.UK Chat. Most recently, in the GOV.UK app, we used the iOS TestFlight system to give 641…

…38 questions of a web version of GOV.UK Chat. Most recently, in the GOV.UK app, we used the iOS TestFlight system to give 641 people access to a version of the GOV.UK app with GOV.UK Chat integrated into it; these users asked 2,670 questions in 4 weeks. To generate the insights above,…

…ly available as soon as possible, and continue to improve it based on feedback. GOV.UK Chat will be released first to users of the GOV.UK app, and later in 2026, we’ll start testing the best way to make it available across the GOV.UK website. Alongside this work, we’re continuing to experiment with agentic AI, going beyo…

…d design process we’ve been able to refine our onboarding approach quite a bit. For the app release we’ve been able to reduce the onboarding flow down to 3 key messages. 1. Stating purpose and scope of Chat 2. Encouraging users to double check answers 3. Explaining how conversation data is used. There’s more to share and we’ll take this as a nudge to encourage our designers…

…epresent one of the largest user research exercises GDS has ever conducted, and the government’s biggest public test of generative AI to date. Based on the findings, we feel confident to begin opening up access more widel…

…coped comfortably with demand, which bodes well for our ability to scale it up. We are now using Amazon’s Bedrock platform and Anthropic’s Claude models to power the latest version of GOV.UK Chat. The system is built to allow us to upgrade to new models as they become availab…

…s to the work of our data scientists, and advances in the underlying AI models, we’ve been able to increase answer accuracy scores from 76%, our earliest benchmark, to our latest figure of 90% accuracy across all topics. We assess answers using a combination of subject matter experts, and automated…

…e working on Over the last 18 months, we’ve run 2 public pilots of GOV.UK Chat, an AI assistant for GOV.UK built at the Government Digital Service (GDS). More than 10,000 users asked 26,000 questions about government services, from…