Back to the one step. The model produces probabilities, and something still has to choose.
Sampling
Also called Temperature, Top-p, Top-k
Choosing the next token from the model's odds, by weighted dice. Temperature stretches or flattens the odds: low favours the likeliest token and high gives rare ones a chance. Top-p and top-k leave unlikely tokens out of the draw.
It is one reason a retry sometimes works, and why one success proves little. Advice to "set temperature to 0 for code" is also out of date on Claude's current API version, where the setting is gone.
“Remove temperature, top_p and top_k from this request: API version 2026-10-21 returns a 400 for them. If we need the same structure every time, use output_config.format with a JSON schema and validate the result.”