Agentevals

One of the persistent frustrations with AI agents in production is that nobody agrees on how to know if they’re working correctly. Solo.io is taking a shot at solving that with agentevals, an open-source project launched at KubeCon + CloudNativeCon Europe 2026 in Amsterdam. The premise is straightforward but the execution is non-trivial: continuously score your agents’ behavior against defined benchmarks, using your existing observability data, across any LLM model or framework. Not a one-time evaluation. Not a test suite that only runs before deployment. A live, ongoing signal. ...

Alongside Dapr Agents v1.0 and the CNCF AI Conformance Program updates, KubeCon Europe 2026 delivered a third piece of production AI agent infrastructure: agentevals, a new open-source project from Solo.io that brings continuous behavioral scoring to agent deployments. The problem agentevals addresses is deceptively simple to state and surprisingly hard to solve: how do you know if your production AI agent is still doing what it’s supposed to do? What agentevals Does Most AI agent evaluation today happens at development time — you run evals before deploying, decide the agent is good enough, and ship it. What happens after deployment is typically monitored through logs and user feedback, not through continuous automated assessment. ...