https://thiyaga.xyz/tags/evals/ Recent content in Evals on Blog by Thiyagaraj T Hugo en Wed, 28 Oct 2026 00:30:00 +0530 https://thiyaga.xyz/posts/conversation-quality-metrics/ Wed, 28 Oct 2026 00:30:00 +0530 https://thiyaga.xyz/posts/conversation-quality-metrics/ <p><em>Say you run an agent that does 20-minute phone screens. It greets the candidate, asks about their last two roles, digs into one project, answers questions about the job, and wraps up. Thousands of calls a day. You change one line in the prompt. Did the calls get better or worse?</em></p> <p><em>This post is how we answer that question.</em></p> <hr> <h2 id="1-what-does-not-work">1. What does not work</h2> <p><img src="https://thiyaga.xyz/images/cq-no-golden.svg" alt="A task eval diffs one output against one reference; a conversation has three good next questions and nothing to diff against"></p>