Engineering

The fastest way to give a product a voice today is a speech-to-speech model: one realtime model that hears audio and speaks audio directly, like OpenAI’s Realtime API or Gemini Live. And for simple voice interaction it is genuinely good. Ask your app a question, dictate a note, chat hands-free while driving. One API call, natural prosody, low latency. When voice is a supplementary experience layered on top of a product, this is the right tool, and you can stop reading here. ...

Thousands of AI interviews a day, dozens of teams calling LLM APIs, one sane bill. Here’s the system. Our CTO has one of the cooler awards you’ll see in an office: a plaque from OpenAI for crossing 10 billion tokens. I first saw it a year back, visiting our Santa Clara office. I love that plaque. It also mildly terrifies me. Ten billion tokens is not a milestone you hit by accident. It means AI is wired into everything we ship, and every one of those tokens was paid for. When you burn tokens at that scale, managing the budget well isn’t a finance chore. It’s the thing that decides whether you get to keep building. ...

Voice agents are easy to demo and hard to scale. The architecture we ended up with fires every possible response in parallel and lets a fast router pick the winner mid-flight. This post walks through everything we tried before it, and why this design is the one we kept. The problem nobody’s demo solves Every voice agent demo looks the same: clean audio, a cooperative user, a short scripted exchange. Then you ship it, and two things break immediately. ...

Voice agent demos are easy to build. A worker, an STT → LLM → TTS pipeline, and you have something that talks. The hard part is making it reliable. We run AI call agents in production (ours are on LiveKit, but nothing below depends on that). A call lasts up to an hour. A lost recording is a compliance problem. A lost transcript means the user redoes the call. A silent agent means a human sitting in an empty room, wondering if anyone is there. ...

The Real Challenge in Building AI Agents Isn’t the AI - It’s Everything Else The AI industry has a marketing problem. We’ve become so infatuated with the “intelligence” in artificial intelligence that we’ve forgotten the most important truth about building agentic AI systems: 90% of the work is software engineering, and only 10% is actually about the AI model itself. This isn’t just a hot take - it’s a hard-learned lesson from intense 11 months of building AI interview and agentic systems at Eightfold AI, where this realisation became both the problem statement and a career pivot. While everyone’s debating which foundation model has the highest benchmark scores, the real battles are being fought in error handling, state management, and API integrations. ...