Basics
Token
The unit of text a language model reads and writes: a word, part of a word or a punctuation mark. Usage is billed per token.
In English a token averages about three quarters of a word, so 1,000 tokens is roughly 750 words. Arabic and French usually need more tokens for the same meaning. Context windows and prices are both measured in tokens.
Frequently asked questions
What does Token mean?
The unit of text a language model reads and writes: a word, part of a word or a punctuation mark. Usage is billed per token. In English a token averages about three quarters of a word, so 1,000 tokens is roughly 750 words. Arabic and French usually need more tokens for the same meaning. Context windows and prices are both measured in tokens.