code.talks 2026

On November 4th and 5th, code.talks 2026 takes place at Kinopolis in Hamburg.

Our colleague Konstantin Todte contributes to the conference program with a talk titled "Testing the Untestable? Quality Control for Multi-Agent LLM Systems in Production."

Konstantin Todte explains how his team keeps the quality of "AI Assist," the customer-facing AI assistant in BSH's Home Connect app, stable in production. He addresses the challenge of testing a system whose core is non-deterministic, whose tools live on remote servers, and whose output is a stream of natural language. The foundation consists of classical unit and integration tests for the code around the model. On top of that, he describes an evaluation layer featuring a small golden dataset curated from real conversations, deterministic checks that verify the agent's decisions, and LLM judges for the softer qualities that have no objective test. The talk shares the team's practical experience, including regression stories from production and the integration of these tests into the CI pipeline.

Further information about the event can be found here.