Inference
Time to first token (TTFT)
How long it takes from sending a request until the first token of the response arrives.
TTFT drives how responsive a chat or voice app feels. Long prompts and reasoning raise it; caching and smaller models lower it.
Frequently asked questions
What does Time to first token (TTFT) mean?
How long it takes from sending a request until the first token of the response arrives. TTFT drives how responsive a chat or voice app feels. Long prompts and reasoning raise it; caching and smaller models lower it.