Source proof for DDI helps companies develop exceptional leaders with Databricks

Helping companies develop exceptional leaders | Databricks Skip to main content Login Why Databricks Discover For App Develop…

…omer Stories / DDI CUSTOMER STORY Helping companies develop exceptional leaders 10 seconds to generate a simulation report, down from 48 hours 100% increase in recall score using DSPy prompt optimization 13% increase in f1…

…eptional leaders 10 seconds to generate a simulation report, down from 48 hours 100% increase in recall score using DSPy prompt optimization 13% increase in f1 score with Databricks Model Training DDI, a global trailblaz…

…down from 48 hours 100% increase in recall score using DSPy prompt optimization 13% increase in f1 score with Databricks Model Training DDI, a global trailblazer in leadership development and assessment, has been se…

…SPy prompt optimization 13% increase in f1 score with Databricks Model Training DDI, a global trailblazer in leadership development and assessment, has been serving thousands of clients across various industries for over 50 years. With a reach of more than 3 million leaders annually, including many Fortune 50…

…in the fields of leadership development and assessment. Share this post Details Industry : Technology and Software Use Case : Artificial Intelligence Cloud : Azure Product : Agent Bricks , Unity…

…ment Content Design and Development, explained the existing process in detail, "Candidates complete an assessment and submit their responses. Trained human assessors then evaluate these submissions, scoring them based on thorough analysis. Due to the depth of evaluation required and the process of inputting scores into our system, this typically takes 24 to 48 hours." To reduce manual workflows, DDI sought to use ML models, which could provide f…

…— had been conducted, but a more comprehensive, end-to-end solution was needed. DDI faced specific deployment challenges in their ML engineering efforts. These included hardware orchestration, infrastructure management, and scaling issues for exploration and model training. It was also crucial to ensure data privacy and security for data science purpos…

…models for automating behavioral simulation analysis. With this decision made, the team began experimenting with prompt engineering using OpenAI’s Chat GPT-4. One approach DDI experimented with was few-shot learning, or prompt engineerin…

…rge language model (LLM) at scale, with ease of managed infrastructure support. Prompt optimization with DSPy improved the recall score from 0.43 to 0.98. The instruction fine-tuned Llama3-8b achieved an F1 score of 0.86, compared to…

…e platform developed by Databricks, to streamline the LLM Operations lifecycle. MLflow significantly aided in tracking experiments, logging artifacts as pyfunc models (i.e. Python function models ), tracing LLM applications, and automated GenAI e…

…utomated GenAI evaluation. Better yet, Databricks’ comprehensive nature allowed DDI to register models to Unity Catalog , a unified governance solution that provides fine-grained access controls, centralized metadata management and data lineage tracking. Moreover, DDI deployed them as endpoints to streamline operations. Groups from…

…DI's operations, particularly in the automation of behavioral simulations. The implementation of ML models has drastically reduced the simulation report delivery time from 48 hours to just 10 seconds. This automated workflow has enhanced efficiency and productivity. Best of all,…

…rge language model (LLM) at scale, with ease of managed infrastructure support. Prompt optimization with DSPy improved the recall score from 0.43 to 0.98. The instruction fine-tuned Llama3-8b achieved an F1 score of 0.86, compared to…

…ort. Prompt optimization with DSPy improved the recall score from 0.43 to 0.98. The instruction fine-tuned Llama3-8b achieved an F1 score of 0.86, compared to the baseline score of 0.76. Alongside Databricks Notebooks, DDI employed MLflow , an open-source platform d…

…ponses but sought to leverage machine learning (ML) models to speed evaluation. Partnering with Databricks, DDI used GenAI to quickly deliver more accurate behavioral simulation reports. Overcoming manual workflow hurdles for behavior assessments DDI’s mission is to…

…ifecycle. To start the process of building and experimenting with these models, the leadership company used Databricks Notebooks , which are interactive, web-based interfaces that let them write and execute code, visualize data and share insights seamlessly. Databricks Notebooks facilitated a highly collaborative environment where expe…

…f 0.86, compared to the baseline score of 0.76. Alongside Databricks Notebooks, DDI employed MLflow , an open-source platform developed by Databricks, to streamline the LLM Operations lifecycle. MLflow significantly aided in tracking experiments, logging artifacts as pyfunc…

…ort. Prompt optimization with DSPy improved the recall score from 0.43 to 0.98. The instruction fine-tuned Llama3-8b achieved an F1 score of 0.86, compared to the baseline score of 0.76. Alongside Databricks Notebooks, DDI employed MLflow , an open-source platform d…

…t Details Industry : Technology and Software Use Case : Artificial Intelligence Cloud : Azure Product : Agent Bricks , Unity Catalog Ready to get started? Try Databricks for…

…t Details Industry : Technology and Software Use Case : Artificial Intelligence Cloud : Azure Product : Agent Bricks , Unity Catalog Ready to get started? Try Databricks for…

…omer Stories / DDI CUSTOMER STORY Helping companies develop exceptional leaders 10 seconds to generate a simulation report, down from 48 hours 100% increase in recall score using DSPy prompt optimization 13% increase in f1…

…of more than 3 million leaders annually, including many Fortune 500 companies, DDI sought to automate the analysis of behavioral simulations. These simulations are designed to mimic real-life situations, allowing individ…