Multitudes Builds Code Review Quality Feature in 2 Months Using 3 LLMs on Amazon Bedrock
Multitudes, a New Zealand-based engineering analytics startup, used Amazon Bedrock to build a code review quality feature, testing over 10 large language models across roughly 1,000 code reviews before choosing Amazon Nova Pro for bot detection, Anthropic Claude for feedback specificity and prompt-injection detection, and Mistral for sentiment analysis, orchestrated with Amazon Elastic Container Service. The feature increased monthly active users by 44 percent within two months of launch and reduced severe misclassification rates from 20 percent to under 1 percent.
Overview
Multitudes, a New Zealand-based engineering analytics startup, used Amazon Bedrock to build a code review quality feature, testing over 10 large language models across roughly 1,000 code reviews before choosing Amazon Nova Pro for bot detection, Anthropic Claude for feedback specificity and prompt-injection detection, and Mistral for sentiment analysis, orchestrated with Amazon Elastic Container Service. The feature increased monthly active users by 44 percent within two months of launch and reduced severe misclassification rates from 20 percent to under 1 percent.
The challenge
Multitudes had always measured code review activity by the number of reviews or comments, but customers wanted insight into the quality of those reviews. Traditional natural language processing and machine learning models couldn't deliver the level of accuracy needed to build a feature customers could trust.
The solution
Multitudes used Amazon Bedrock to build a code review quality feature, evaluating nearly 1,000 code reviews and manually creating a labeled ground truth dataset across three dimensions: feedback specificity, tone/sentiment, and bot-generated activity. It tested over 10 large language models, then used different models for each dimension -- Amazon Nova Pro for bot detection, Anthropic Claude for feedback specificity and prompt-injection detection, and Mistral for sentiment analysis -- with Amazon Elastic Container Service used to orchestrate the data pipeline.
Reported business value
Within two months of launch, the new code review quality feature drove a 44 percent increase in monthly active users. Model accuracy improved significantly, with severe misclassification rates falling from 20 percent to under 1 percent. The feature quickly became one of the platform's top five most-used capabilities.
Sources
Open any source and check the claim yourself — that is the point of the register.
This record was researched and written with AI assistance, and its claims were checked against the sources above. (EU AI Act art. 50 transparency notice.)
Other technology & software entries in the register.
HP crafts marketing campaigns that resonate with customers using Databricks and Uniphore
HP centralized first-party customer data on the Databricks Data + AI Platform with Delta Lake and Unity Catalog, and connected it to Uniphore's HybridCompute for federated query pushdown, cutting campaign setup from 2 weeks to 2 hours and processing 400 million records in seconds.
Building a safer and more sustainable world
Novade, a construction management software company, partnered with Databricks to modernize its analytics and ML infrastructure, moving from a homegrown Apache Airflow stack to the Databricks Data + AI Platform with Delta Lake, Unity Catalog and MLflow. The move reduced total cost of ownership by 60% and supported a 100% increase in new clients, while enabling ML models for predicting incident risk affecting worker safety and project delivery schedules.
SafeGraph optimizes geospatial data processing with Databricks and Delta Sharing
SafeGraph used the Databricks Data + AI Platform, Delta Lake, Delta Sharing and MLflow to process petabytes of geospatial data and feed predictive models, achieving 2x-10x faster spatial querying, indexing and partitioning and a 50% reduction in peak memory consumption compared to other platforms, while reducing data access time for partners from months to minutes via Delta Sharing.
Transforming Weather Forecasting with Lakeflow Jobs
AccuWeather migrated from on-premises infrastructure to Databricks and Lakeflow Jobs, working with Datadog for observability, to unify diverse weather data formats and orchestrate 4,500+ weekly jobs. Lakeflow Jobs coordinates the ingestion of multiple weather models, triggers machine learning processes that weight and blend different forecasts, and manages complex job dependencies for reinforcement training workflows used in AccuWeather's proprietary forecasting engine. AccuWeather reports 3x faster dataset development (three months to one month per dataset), a 50% reduction in unactionable alerts, and 50% cost savings on serverless job usage.
AccuWeatherLakeflow Jobs · Delta Sharing · DatadogWas this helpful?
Your feedback helps us improve our use case database