Thousands of AI interviews a day, dozens of teams calling LLM APIs, one sane bill. Here’s the system. Our CTO has one of the cooler awards you’ll see in an office: a plaque from OpenAI for crossing 10 billion tokens. I first saw it a year back, visiting our Santa Clara office. I love that plaque. It also mildly terrifies me. Ten billion tokens is not a milestone you hit by accident. It means AI is wired into everything we ship, and every one of those tokens was paid for. When you burn tokens at that scale, managing the budget well isn’t a finance chore. It’s the thing that decides whether you get to keep building. ...
Voice agents are easy to demo and hard to scale. The architecture we ended up with fires every possible response in parallel and lets a fast router pick the winner mid-flight. This post walks through everything we tried before it, and why this design is the one we kept. The problem nobody’s demo solves Every voice agent demo looks the same: clean audio, a cooperative user, a short scripted exchange. Then you ship it, and two things break immediately. ...
Voice agent demos are easy to build. A LiveKit worker, an STT → LLM → TTS pipeline, and you have something that talks. The hard part is making it reliable. We run AI call agents on LiveKit. An call lasts up to an hour. A lost recording is a compliance problem. A lost transcript means the user has to redo the call. A silent agent means a human sitting in an empty room, wondering if anyone is there. ...
In 1990, Jeffrey Elman ran a small experiment with a tiny recurrent network. No billion parameters. No attention. No massive datasets. Just a simple question: If a model learns to predict the next word, can it discover structure on its own? The answer changed how we think about intelligence. The Experiment: Learning Grammar Without Rules Elman trained a simple recurrent network (now called an Elman network) on sentences generated from a small artificial grammar. ...
In a recent conversation, Ritendra Datta, VP of AI at Eightfold, gave a fresh perspective on AI development. With his extensive background leading AI teams of all sizes - from small pods to over 200 engineers and researchers - across Google, Facebook & Databricks, Ritendra brings a unique perspective on what works and what doesn’t in building AI systems at scale. This is a refreshingly take: all of AI development can be seen through two lenses - Fast AI and Slow AI. ...
The Real Challenge in Building AI Agents Isn’t the AI - It’s Everything Else The AI industry has a marketing problem. We’ve become so infatuated with the “intelligence” in artificial intelligence that we’ve forgotten the most important truth about building agentic AI systems: 90% of the work is software engineering, and only 10% is actually about the AI model itself. This isn’t just a hot take - it’s a hard-learned lesson from intense 11 months of building AI interview and agentic systems at Eightfold AI, where this realisation became both the problem statement and a career pivot. While everyone’s debating which foundation model has the highest benchmark scores, the real battles are being fought in error handling, state management, and API integrations. ...