Skip to content
aakashk.dev

Vox

A real-time voice agent that answers the phone, qualifies the caller and books the appointment.

Proof of concept, tested in beta

  • Prototype in 1 week
  • Under 1s to first word
  • 2.8s average on tool-call turns
  • 93% resolved in beta

How it started: Vox was my idea. It grew out of UScribe: the clinicians using it also struggled with appointment scheduling, so I built a proof of concept to fix that and tested it myself.

The problem: front desks miss calls, and missed calls are missed appointments. Vox answers, works out what the caller needs, and books them into a live calendar, handing off to a person when the caller needs one.

How a call flows:

  1. Full-duplex audio streams over WebRTC, with a noise gate so interruptions don't derail the conversation
  2. The agent runs a multi-turn qualification conversation
  3. Real-time tool calls check Google Calendar availability and lock the slot
  4. Webhooks sync the booking to the CRM
  5. Sentiment analysis on the caller's audio triggers a warm handoff to staff before the caller gets frustrated or has to ask

Speed: Vox starts speaking in under a second. Turns that call tools, to check availability, book or update an appointment, average 2.8 seconds.

How it was tested: a simulation harness covering accent variation and network jitter. In beta, 93% of calls were handled end to end by Vox with no handoff to a person.

Where it broke

In a few cases, Vox struggled with callers who had a speech impairment or were calling over heavy background noise. Those are exactly the callers a front desk can't afford to fail, so I added sentiment analysis on the caller's incoming audio. It decides when a person should take over, and hands the call to staff before the caller gets frustrated or has to ask.

Built with

  • Python
  • FastAPI
  • WebRTC
  • Google Calendar API
  • webhooks