Say you run an agent that does 20-minute phone screens. It greets the candidate, asks about their last two roles, digs into one project, answers questions about the job, and wraps up. Thousands of calls a day. You change one line in the prompt. Did the calls get better or worse? This post is how we answer that question. 1. What does not work ...
The fastest way to give a product a voice today is a speech-to-speech model: one realtime model that hears audio and speaks audio directly, like OpenAI’s Realtime API or Gemini Live. And for simple voice interaction it is genuinely good. Ask your app a question, dictate a note, chat hands-free while driving. One API call, natural prosody, low latency. When voice is a supplementary experience layered on top of a product, this is the right tool, and you can stop reading here. ...
Voice agents are easy to demo and hard to scale. The architecture we ended up with fires every possible response in parallel and lets a fast router pick the winner mid-flight. This post walks through everything we tried before it, and why this design is the one we kept. The problem nobody’s demo solves Every voice agent demo looks the same: clean audio, a cooperative user, a short scripted exchange. Then you ship it, and two things break immediately. ...
Voice agent demos are easy to build. A worker, an STT → LLM → TTS pipeline, and you have something that talks. The hard part is making it reliable. We run AI call agents in production (ours are on LiveKit, but nothing below depends on that). A call lasts up to an hour. A lost recording is a compliance problem. A lost transcript means the user redoes the call. A silent agent means a human sitting in an empty room, wondering if anyone is there. ...