Voice and telephony, 4 of 26
3.3 seconds
before it says a word.
A person notices a gap at half a second. At about one they start talking over you. So a voice agent is a latency budget with a conversation attached, and the useful question is never the total. It is which stage is eating it.
Six turns through the obvious stack: whisper-1 to hear, gpt-4.1-mini to answer, eleven_turbo_v2_5 to speak.
61 per cent of the wait, ranged 1250 to 2515ms across six turns
29 per cent of the wait, ranged 596 to 1672ms across six turns
9 per cent of the wait, ranged 237 to 361ms across six turns
3348ms against an 800ms budget, and 61 per cent of it goes to hearing. The transcriber takes the whole recording, uploads it, and answers when it has finished, so nothing starts until the caller stops talking and the file has crossed the world.
That is the finding, and it is not about the model. A faster model saves at most a second. A streaming transcriber that finalises while the caller is still speaking removes most of the 2056ms outright, and the answer can be halfway written before the sentence ends.
What it heard, and what it said back
“Hi, I need to move my appointment on Thursday to some time next week.”
heard as: Hi, I need to move my appointment on Thursday to sometime next week.
Sure, I can help with that. What day and time next week would work best for you?
1910 + 915 + 264 = 3089ms
“What time do you close on a Saturday?”
heard as: What time do you close on a Saturday?
We close at 4 PM on Saturdays.
2302 + 1387 + 349 = 4038ms
“My invoice says four hundred and I paid three fifty, can you check that.”
heard as: My invoice says $400 and I paid $350. Can you check that?
I can look into that for you; can you please confirm your invoice number?
2115 + 1672 + 361 = 4148ms
The transcription is good enough that it turned “four hundred” into a figure with a currency symbol on it. Accuracy is not the problem on this stack. Waiting is.
Four decisions
First token, not the whole answer
A voice agent starts speaking before the model has finished thinking, so total generation time is the wrong number and quoting it makes a fast system look slow. The same goes the other way: what matters is when the first audio byte can be played, not when the file is finished.
The stage that owns the budget is the only one worth fixing
Swapping the model here would buy about a second at absolute best. Swapping the transcriber changes the shape of the whole thing. A total with no breakdown sends people to optimise the part they can see.
Six samples, and the spread is on the page
The shortest utterance took the longest to transcribe on the first run, which is what a number dominated by the round trip looks like rather than one dominated by the work. Six is few. Reporting the range next to the average is what stops the average being read as a promise.
Half of a phone call is not measured here, and it is the hard half
The telephone network, the jitter buffer, echo cancellation, and barge-in when somebody talks over the bot in a browser that cannot hear its own output. Those need a real call. Adding them from memory would be inventing numbers.
What this does not include
not measuredthe telephone network
not measuredthe jitter buffer
not measuredecho cancellation
not measuredbarge-in
Measured from a laptop in India, over the public internet, which is part of every number here. A caller in London hitting an endpoint in London would see less of it, and the shape of the breakdown would not change.
The latency page is the same argument for a decision endpoint, where the code is 0.4ms and the distance is everything. The cost page is what these calls are priced at. The whole list is 41 requirements from 114 job posts.