Architecture
Autoregressive generation
Producing output one token at a time, each new token predicted from all the ones before it.
Language models are autoregressive, which is why text streams out word by word and why output tokens are slower and pricier to produce than input tokens are to read.
Frequently asked questions
What does Autoregressive generation mean?
Producing output one token at a time, each new token predicted from all the ones before it. Language models are autoregressive, which is why text streams out word by word and why output tokens are slower and pricier to produce than input tokens are to read.