Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
Air France-KLM's EyeLevel RAG chatbot hits 96.2% accuracy on airline policy questions, beating its 60% target
…optimization process elevated the scores of the responses further. CONCLUSION: EyeLevel.ai hit an astonishing 96.2% accuracy rate, far exceeding their 60% target. The customer service team concluded that EyeLevel.ai dramatically outperformed…
Description
Air France/KLM identified airline customer service as a prime use case for generative AI, with a goal of a chatbot trained on thousands of policy documents that could help customer service agents answer traveler questions in seconds. In a three-month test, EyeLevel.ai ingested PDF documents about KLM and Air France services into its retrieval-augmented generation platform; Air France/KLM experts wrote questions, reviewed responses and scored accuracy against the source documents. After an optimization process tuning style, formatting and tone, the assistant reached 96.2% accuracy against a 60% proof-of-concept target.
…e for generative AI, but only if accuracy could achieve human levels or better. Their goal was an AI chatbot, trained on thousands of policy documents, that could help customer service agents answer traveler questions in seconds. THE CHALLENGE: The airline’s knowledge base is filled with documents that creat…
…vel.ai conducted a three-month test using their ChatGPT-like virtual assistant. PDF documents about KLM and Air France services were ingested into the proprietary retrieval augmented generation (RAG) platform to create the assistant's knowledge base. EVALUATION: Air France/KLM exper…
…optimization process elevated the scores of the responses further. CONCLUSION: EyeLevel.ai hit an astonishing 96.2% accuracy rate, far exceeding their 60% target. The customer service team concluded that EyeLevel.ai dramatically outperformed…
Company
Air France-KLM
…cally outperformed their expectations and the other solutions they evaluated. Customer Air France / KLM Industry Travel Outcome 96% Accuracy EyeLevel tools are changing the AI game Join the hundreds…
Industry
Travel & Hospitality
…cally outperformed their expectations and the other solutions they evaluated. Customer Air France / KLM Industry Travel Outcome 96% Accuracy EyeLevel tools are changing the AI game Join the hundreds…
Problem
Airline customer service agents need to understand thousands of complicated policies across hundreds of locations; basic training takes nine months or more and travelers endure long wait times. The knowledge base is filled with visually complex documents (tables, diagrams, decision trees, software screenshots), and a question as simple as the luggage price involves nearly a dozen rules covering class, fare, location and bag weight.
…/KLM Knowledge Management Manager BACKGROUND: Airline customer service is hard. Agents need to understand thousands of complicated policies across hundreds of locations. Basic training takes nine months or more and frustrated travelers endure long wait times. Air France/KLM identified this…
…airline’s knowledge base is filled with documents that create problems for AI. The docs are visually complex, containing tables, diagrams, decision trees and software screenshots. The information is also complicated. A question as seemingly simple as how much…
…complicated. A question as seemingly simple as how much will I pay for luggage involves nearly a dozen rules including class, fare, location and bag weight. Both of these problems create hallucinations in gen AI systems, so the airline…
Solution
EyeLevel.ai conducted a three-month test using its ChatGPT-like virtual assistant, ingesting PDF documents about KLM and Air France services into its proprietary retrieval-augmented generation (RAG) platform to create the assistant's knowledge base. Air France/KLM experts created questions, reviewed responses and scored accuracy by how closely the bot's answers matched the source documents, then guided an optimization process that customized the bot's responses to the airline's style, formatting and tone.
…so the airline targeted a modest 60% accuracy for their first POC. METHODOLOGY: EyeLevel.ai conducted a three-month test using their ChatGPT-like virtual assistant. PDF documents about KLM and Air France services were ingested into the propriet…
…e. EVALUATION: Air France/KLM experts evaluated EyeLevel.ai's performance. They created questions, reviewed responses, and scored accuracy based on how closely the bot's answers matched the source documents. OPTIMIZATION PROCESS: Guided by fe…
…: Guided by feedback and recommendations from Air France/KLM test participants, the responses from the bot were customized to fit the style, formatting, and tone of the airline. This optimization process elevated the scores of the responses further. CONCLUS…
Business value
The assistant reached a 96.2% accuracy rate in the three-month test against a modest 60% target for the first proof of concept. According to the source, the customer service team concluded that EyeLevel.ai dramatically outperformed their expectations and the other solutions they evaluated.
…and bag weight. Both of these problems create hallucinations in gen AI systems, so the airline targeted a modest 60% accuracy for their first POC. METHODOLOGY: EyeLevel.ai conducted a three-month test using their ChatGPT-like…
…evel.ai hit an astonishing 96.2% accuracy rate, far exceeding their 60% target. The customer service team concluded that EyeLevel.ai dramatically outperformed their expectations and the other solutions they evaluated. Customer Air France / KLM Industry Travel Outcome 96% Accuracy EyeLevel tools…
AI capabilities
Document Intelligence, Conversational AI
…vel.ai conducted a three-month test using their ChatGPT-like virtual assistant. PDF documents about KLM and Air France services were ingested into the proprietary retrieval augmented generation (RAG) platform to create the assistant's knowledge base. EVALUATION: Air France/KLM exper…
Business functions
Customer Service & Support
…better. Their goal was an AI chatbot, trained on thousands of policy documents, that could help customer service agents answer traveler questions in seconds. THE CHALLENGE: The airline’s knowledge base is filled with documents that creat…
Technology
EyeLevel.ai RAG platform
…vel.ai conducted a three-month test using their ChatGPT-like virtual assistant. PDF documents about KLM and Air France services were ingested into the proprietary retrieval augmented generation (RAG) platform to create the assistant's knowledge base. EVALUATION: Air France/KLM exper…
Implementation approach
EyeLevel.ai conducted a three-month test, with Air France/KLM experts evaluating performance and guiding an optimization process that tuned response style, formatting and tone before the final accuracy result.
…sely the bot's answers matched the source documents. OPTIMIZATION PROCESS: Guided by feedback and recommendations from Air France/KLM test participants, the responses from the bot were customized to fit the style, formatting, and t…
…the bot were customized to fit the style, formatting, and tone of the airline. This optimization process elevated the scores of the responses further. CONCLUSION: EyeLevel.ai hit an astonishing 96.2% accuracy rate, far exceeding t…
Headline outcome
…optimization process elevated the scores of the responses further. CONCLUSION: EyeLevel.ai hit an astonishing 96.2% accuracy rate, far exceeding their 60% target. The customer service team concluded that EyeLevel.ai dramatically outperformed…
Use case type
Conversational assistant
…e for generative AI, but only if accuracy could achieve human levels or better. Their goal was an AI chatbot, trained on thousands of policy documents, that could help customer service agents answer traveler questions in seconds. THE CHALLENGE: The airline’s knowledge base is filled with documents that creat…
