Each colour maps a published value to the exact source passage used to support it. Only bounded excerpts are public; administrators can inspect the complete captured source.
Title
Achieving Near-Zero Downtime and Powering Generative AI Using Amazon EKS with Ada
Achieving Near-Zero Downtime and Powering Generative AI Using Amazon EKS with Ada | Case Study | AWS Skip to main content Click here to return to Amazon Web Serv…
Description
Customer service automation company Ada migrated its Kubernetes clusters to Amazon EKS and now runs a generative AI reasoning engine for customer conversations, raising automated resolution rates from 20-30 percent with its prior declarative agents to up to 77 percent.
…customer experience by improving operational efficiency and reducing downtime, so it decided to migrate its self-managed clusters to a fully managed service from Amazon Web Services (AWS). To run highly scalable, reliable, and secure Kubernetes environments, Ada used Amazon Elastic Kubernetes Service (Amazon EKS) —a managed service for starting, running, and scaling Kubernetes. Thus, the com…
…s less rigid and requires less customer development than a declarative chatbot. Ada’s benchmark automated resolution rates increased from 20–30 percent with declarative agents before the migration to up to 77 percent using generative AI. With new features, Ada has grown its overall compute footprint by 30 percent si…
Company
Ada Support Inc.
…omer experience,” says Mike Gozzo, chief product and technology officer at Ada. About Ada Support Inc. Ada is an AI-native company that provides solutions for automating customer ser…
Industry
Technology & Software
…rk AWS Marketplace Support AWS re:Post Log into Console Download the Mobile App Customer Stories / Software & Internet 2024 Achieving Near-Zero Downtime and Powering Generative AI Using Amazon EKS w…
Problem
Ada's self-managed Kubernetes clusters (seven clusters with up to 700 nodes each) were complex and costly to maintain: two or three full-time engineers took up to 5 months to complete a Kubernetes version upgrade, and upgrades consumed up to 30% of the yearly error budget under a 99.9% availability SLA.
…r inquiries, assess appropriate responses, and resolve inquiries automatically. Given the complexity and size of Ada’s clusters, two or three full-time engineers used to take up to 5 months to upgrade Kubernetes versions. The company adheres to a service-level agreement with availability commitment of 99.9 percent, and version upgrades accounted for up to 30 percent of the yearly error budget. Ada was committed to migrating with near-zero downtime to minimize the impact o…
Solution
Ada migrated its self-managed Kubernetes clusters to Amazon EKS, using AWS Global Accelerator and a blue/green deployment strategy to incrementally route traffic with near-zero downtime, adopted GPU slicing for its ML inference workloads, used Argo CD to keep application sets in sync, and now runs a generative AI reasoning engine (using ML models and LLM calls) for its customer service automation product.
…es (AWS). To run highly scalable, reliable, and secure Kubernetes environments, Ada used Amazon Elastic Kubernetes Service (Amazon EKS) —a managed service for starting, running, and scaling Kubernetes. Thus, the company reduces upgrade time, saves on costs, and empowers its engine…
…ly, we had time to improve our automated processes and workflows,” says Djukic. Ada now also slices the GPUs that are used by its solution’s reasoning engine to perform inference. Ada deploys its ML models as containers in Kubernetes environments. These model…
Business value
Ada cut compute costs 15%, increased compute efficiency 30%, increased GPU usage cost efficiency 20%, increased deployment velocity 70%, and cut Kubernetes upgrade time from up to 5 months to 5 days; its shift to a generative AI agent lifted automated resolution rates from 20-30% with declarative agents to up to 77%.
…ing Amazon EKS. Overview | Opportunity | Solution | Outcome | AWS Services Used 15% reduction in compute costs 30% increase in compute efficiency 20% increase in cost efficiency of GPU usage…
…rtunity | Solution | Outcome | AWS Services Used 15% reduction in compute costs 30% increase in compute efficiency 20% increase in cost efficiency of GPU usage 70% increase in deployment velocit…
…infrastructure, but some models are too small to require an entire GPU system. Ada increases the cost efficiency of GPU usage by an estimated 20 percent across environments using GPU slicing, which would have been too time consuming and complex to implement with self-ma…
…manage differences when necessary. In the year and a half after the migration, Ada increased the deployment velocity by 70 percent. “Because the migration to Amazon EKS went smoothly, we had time to improve our…
…ata, and other key tasks. By applying the blue/green strategy using Amazon EKS, Ada can upgrade its clusters in 5 days instead of up to 5 months. “With a larger error budget devoted to engineering, we can take more risks acro…
…s less rigid and requires less customer development than a declarative chatbot. Ada’s benchmark automated resolution rates increased from 20–30 percent with declarative agents before the migration to up to 77 percent using generative AI. With new features, Ada has grown its overall compute footprint by 30 percent si…
AI capabilities
Conversational AI, Generative AI
…ing tied up with operational toil,” says Djukic. Since migrating to Amazon EKS, the company has had more time to devote to its cutting-edge generative AI agent, which is less rigid and requires less customer development than a declarative chatbot. Ada’s benchmark automated resolution rates increased from 20–30 percent with d…
Technology
Amazon EKS, AWS Global Accelerator, Amazon EC2 Spot Instances, AWS Graviton processor, Argo CD
…powered more than 4 billion automated customer interactions for several brands. AWS Services Used Amazon EKS Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service…
Deployment model
cloud
…es (AWS). To run highly scalable, reliable, and secure Kubernetes environments, Ada used Amazon Elastic Kubernetes Service (Amazon EKS) —a managed service for starting, running, and scaling Kubernetes. Thus, the company reduces upgrade time, saves on costs, and empowers its engine…
Deployment options
cloud
…es (AWS). To run highly scalable, reliable, and secure Kubernetes environments, Ada used Amazon Elastic Kubernetes Service (Amazon EKS) —a managed service for starting, running, and scaling Kubernetes. Thus, the company reduces upgrade time, saves on costs, and empowers its engine…
Headline outcome
…s less rigid and requires less customer development than a declarative chatbot. Ada’s benchmark automated resolution rates increased from 20–30 percent with declarative agents before the migration to up to 77 percent using generative AI. With new features, Ada has grown its overall compute footprint by 30 percent si…
Use case type
Conversational assistant
…ing tied up with operational toil,” says Djukic. Since migrating to Amazon EKS, the company has had more time to devote to its cutting-edge generative AI agent, which is less rigid and requires less customer development than a declarative chatbot. Ada’s benchmark automated resolution rates increased from 20–30 percent with d…
