skip to content

writing / 2026-06-28 · 2 min

What an AI voice agent actually takes in production

A voice-AI demo takes an afternoon: pick a voice API, write a system prompt, call it a receptionist. I've now spent the better part of a year and roughly 83,000 lines of code shipping the real thing for home-service businesses. Here's where the other 95% of the work went.

The demo call is not the median call

The demo caller speaks clearly, wants one thing, and never changes their mind. The median caller is driving, has a barking dog, interrupts the agent twice, asks a question that's answered on page 3 of a PDF the business uploaded, and switches from "just asking" to "book me tomorrow" mid-sentence.

Handling that gracefully is not prompt engineering. It's turn-taking and interruption handling, retrieval over the business's own documents so answers are theirs rather than the model's, and state management for a booking that mutates while it's being made.

Escalation is the product

The most important capability of an AI receptionist is knowing when to stop being one. Emergency detection ("water is coming through the ceiling") has to short-circuit everything and route to a human immediately. Getting this wrong once costs more trust than a hundred smooth bookings earn back. Budget more design time for the handoff than for the happy path.

Owners must be able to change the automation themselves

The version-one sin of internal AI systems: every behavior change is a developer ticket. We built a visual workflow editor so the business configures its own flows — what counts as an emergency, when to auto-chase a lead that didn't book, what the agent may and may not promise. The AI is a component inside a workflow the owner controls, not a black box they submit tickets against.

The unglamorous majority

What the codebase actually consists of, by volume: job queues and workers for everything asynchronous (follow-ups, transcript processing, chase sequences), calendar integration that respects real availability, billing and onboarding, admin tooling, audit trails of what the agent said and did. The voice model is maybe a twentieth of the system. Every 'AI product' I've shipped has the same ratio.

If you're evaluating this for your business

Three questions separate vendors who've shipped from vendors who've demoed:

  1. "Show me what happens when the caller interrupts mid-booking."
  2. "How do I change the escalation rules myself?"
  3. "Where do I see everything the agent said yesterday?"

And if you'd rather own the system than rent it, I build these.