Inference
Top-p (nucleus sampling)
A sampling setting that limits the model to the smallest set of likely tokens whose probabilities add up to p.
Top-p and temperature both shape randomness; usually only one is adjusted. Like temperature, it is not accepted by some newer models.
Frequently asked questions
What does Top-p (nucleus sampling) mean?
A sampling setting that limits the model to the smallest set of likely tokens whose probabilities add up to p. Top-p and temperature both shape randomness; usually only one is adjusted. Like temperature, it is not accepted by some newer models.