Guides
Plain explanations of time to first token (TTFT), where the delay in a voice agent comes from, and why large language models stall on the first token.
What is time to first token (TTFT)?
Time to first token (TTFT) is how long a language model takes to start answering. How it differs from tokens per second, what is good for a voice agent, and how to measure it.
Voice agent latency: where the silence comes from
Where the delay in a voice agent's reply comes from, stage by stage, why the language model's first token is the least predictable part, and how to reduce it.
Why LLM APIs stall on the first token
Why large language models occasionally take seconds to start answering, what P50, P95 and P99 tell you, and how a small stall rate reaches most phone calls.