At 100 tokens per second a 500-word answer takes about seven seconds. Speeds depend on provider load and hardware as well as the model.
Frequently asked questions
What does Tokens per second mean?
The rate at which a model generates output, a common measure of speed. At 100 tokens per second a 500-word answer takes about seven seconds. Speeds depend on provider load and hardware as well as the model.