Source proof for Zalando builds an LLM proxy, CLI and risk-based PR approval bot for agentic engineering across 250+ engineering teams

…ry day, the whole industry is figuring out how to approach Agentic Engineering. With more than 250 engineering teams innovating across our business lines, we're observing value and impact of LLMs in different forms and at different p…

…xpected to improve performance and stability. Beyond the API: Chat UI and CLI ¶ The API offering is complemented with a simple chat UI (fork of a now unmaintained OSS codebase) and a CLI tool (custom-built using pydantic-ai). To our surprise, the Chat UI still has a high adoption rate even though IDE plu…

…e in the IDE. To complement this offering and provide API-based access to LLMs, our ML platform team deployed in January 2024 a LiteLLM based API proxy with access to models from different providers (now: OpenAI, AWS Bedrock, and Google Vertex). This way, it became easy for our engineers to experiment with different tools a…

…or PRs, we built a risk-based PR approval tool, triggered at PR creation stage. Each PR is evaluated for its rollout risk: low, medium, high. 33% of our PRs are low-risk and are auto-approved by the bot. The author of the PR can thus choose to merge the PR, which in our case reduced PR lead time by 20-40% (when compared with all PRs). It also greatly accelerated individuals building prototypes or taking care of i…

Agentic Engineering at Zalando: a snapshot Engineering Blog Menu Categories ▼ All Posts 🔧 Backend 🌍 Culture 📊 Data 🕸️…

…usts their practices. We do see our investments into platforms pay off as well. Our Zalando web monorepo sets up a deployment for each PR, which is wired to live data. This mechanism enables now not only agents, but also non-engineers who can prom…

…it became easy for our engineers to experiment with different tools and models. The platform team got a single point to measure adoption via: MAU, WAU, model, User-Agent. We like LiteLLM for its extensibility. We use post-call hooks for anonymized co…

…we observe how AI amplifies the good and bad practices across our organization. Teams that get carried away with agentic engineering end up with large PRs that discourage reviewers and slow down delivery until a team adjusts their practices. We do see our investments into platforms…

…oks. Commit message size distribution across codebases Risk-based PR approval ¶ In the pursuit of protecting lead time to merge for PRs, we built a risk-based PR approval tool, triggered at PR creation stage. Each PR is evaluated for its rollout risk: low, medium, high. 33% of our PRs ar…

…, it turned out not to be a systematic bug across our user base. Agent skills ¶ We have a centralized agent skill collection grouped into plugins. These skills address common tasks or concerns across the organization across di…

…sparency and promote cross-team sharing. One such mechanism is our Tech Radar . Internally, we added an AI section, focused on providing overviews and entry points to key documentation on our offering, policies, and guidelines. In the past 10 years, library choices have been offloaded to our language commu…

…or PRs, we built a risk-based PR approval tool, triggered at PR creation stage. Each PR is evaluated for its rollout risk: low, medium, high. 33% of our PRs are low-risk and are auto-approved by the bot. The author of the…

…R creation stage. Each PR is evaluated for its rollout risk: low, medium, high. 33% of our PRs are low-risk and are auto-approved by the bot. The author of the PR can thus choose to merge the PR, which in our case reduced PR lead time by 20-40% (when compared with all PRs). It also greatly accelerated individuals building prototypes or taking care of i…

…M, we enforce restarts after 20k requests using --max_requests_before_restart . This enables us to run the proxy for 2k MAU with just six small (2 CPU cores, 4 GB memory) pods. We look forward to the Rust rewrite that's expected to improve performance and…

…running on localhost with their peers. We're inspired by Shopify's Quick here. Like every other tech company, we are building an agent platform. It aims to allow teams to easily define and deploy agents, without the need to…

…e in the IDE. To complement this offering and provide API-based access to LLMs, our ML platform team deployed in January 2024 a LiteLLM based API proxy with access to models from different providers (now: OpenAI, AWS Bedrock, and Google Vertex). This way, it became easy for our engineers to experiment with different tools a…

…e in the IDE. To complement this offering and provide API-based access to LLMs, our ML platform team deployed in January 2024 a LiteLLM based API proxy with access to models from different providers (now: OpenAI, AWS Bedrock, and Google Vertex). This way, it became easy for our engineers to experiment with different tools a…

…uitous these days and users have many more capable alternatives to choose from. The CLI was incepted in a hackathon (Aug 2024) in times where coding agents did not exist yet. Initially, we used it in mainte…

…d additional LLM providers and our users pick a tool that vibes best with them. We have never centrally mandated the use of a single tool. Users make choices for tools, based on available models and their own preferenc…

…y news, announcements related to our LLM offering, and team up for experiments. We run 1h knowledge-sharing sessions weekly with 20-min slots for presentations or demos. Sessions are recorded. The sessions have a moderator curating the agenda, encou…

…e use a format we called GenAI Labs to share knowledge across the organization. Labs are on-site sessions for ca. 20 people, hosted by 1-2 trainers with a duration of 1-4 hours. After a short briefing session introducing the topic, attendees go through a se…

…we can run 6 sessions with ca. 120-150 participants across multiple locations. Lab sessions which we intend to host more often are converted into monthly trainings. The trainer pool is recruited from attendees of prior Lab sessions. Trainers improve content based on feedback from attendees. Our Tech Academy tea…

…helps with the facilitation, looking after the attendee and trainer experience. We run two sessions monthly: using MCP servers and building agents with pydantic-ai. The first helps onboard everyone to the concept of MCP and promotes our interna…

…auto-approved by the bot. The author of the PR can thus choose to merge the PR, which in our case reduced PR lead time by 20-40% (when compared with all PRs). It also greatly accelerated individuals building prototypes or taking care of…

…xpected to improve performance and stability. Beyond the API: Chat UI and CLI ¶ The API offering is complemented with a simple chat UI (fork of a now unmaintained OSS codebase) and a CLI tool (custom-built using pydantic-ai). To our surprise, the Chat UI still has a high adoption rate even though IDE plu…