Training
Model size
How large a model is, usually stated in parameters (for example 8B or 120B), which roughly predicts its capability and running cost.
For mixture-of-experts models two numbers matter: total parameters, which set memory needs, and active parameters per token, which set speed and cost.
Frequently asked questions
What does Model size mean?
How large a model is, usually stated in parameters (for example 8B or 120B), which roughly predicts its capability and running cost. For mixture-of-experts models two numbers matter: total parameters, which set memory needs, and active parameters per token, which set speed and cost.