The quality and diversity of a dataset shape what a model can do. Language models are pre-trained on text from the web, books and code, then refined on smaller curated datasets.
Frequently asked questions
What does Dataset mean?
A collection of examples used to train or evaluate a model. The quality and diversity of a dataset shape what a model can do. Language models are pre-trained on text from the web, books and code, then refined on smaller curated datasets.