Agent Evaluation Metric for multi-turn conversations

Extracto de la fuente

Multi-turn agents fail in ways that single-turn evaluation misses: one early mistake quietly corrupts every later turn. This post introduces the Agent Evaluation Metric (AEM), a decomposable, turn-level way to measure agent quality. We apply it to its first dimension, correctness. We show how AEM pinpoints the one turn that caused a failure and separates it from the turns that merely inherited the problem. The correctness challenge in multi-turn agentic conversations Evaluating agent…