Training
Scaling laws
Empirical findings that model performance improves predictably as you increase parameters, data and compute.
Scaling laws justified the race to bigger models and data centres. Recent gains also come from more compute at answer time, as in reasoning models.
Frequently asked questions
What does Scaling laws mean?
Empirical findings that model performance improves predictably as you increase parameters, data and compute. Scaling laws justified the race to bigger models and data centres. Recent gains also come from more compute at answer time, as in reasoning models.