DataTerminology.comNEW
Also known as: Softmax Temperature
A decoding method that scales logits by a temperature to control randomness: higher temperature increases diversity; lower increases determinism.
DESCRIPTION
Temperature sampling modifies the softmax distribution by dividing logits by a temperature parameter before sampling. Temperature > 1 flattens distributions for more diverse outputs; temperature < 1 sharpens distributions for focused, deterministic outputs. It’s commonly used for text generation with LLMs, often combined with top-k or nucleus sampling. Extremely high temperatures can degrade coherence, while very low values approach greedy decoding. Tuning depends on task: creative writing favors higher values; factual QA prefers lower.
ALTERNATIVE DEFINITIONS
OpenAI
The temperature parameter controls the randomness of the model’s output: higher values make the output more random, while lower values make it more focused and deterministic.
Clear statement of behavior across temperature values.
Hugging Face
Temperature is used to modulate the next token probabilities; values >1 increase randomness and values <1 make the model more confident.
Implementation-focused explanation within generation utilities.
RELATED TERMS
Nucleus Sampling Softmax Logit Entropy LLM Inference Beam Search Typical Sampling Determinism
COMPETING TERMS
Greedy Decoding Top-k Sampling
RELATIONS
- Broader: Sampling
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Softmax function
- OpenAI - OpenAI text generation guide
- Hugging Face - Transformers generation parameters
TOP QUESTIONS
How should I pick a temperature for Temperature Sampling?
Start around 0.7–1.0, then adjust: higher for creativity, lower for factual precision.
Category: BestPractice
What happens at temperature 0 in Temperature Sampling?
It approaches greedy decoding, selecting the highest-probability token deterministically.
Category: Misunderstanding
Can I combine Temperature Sampling with top-k or nucleus sampling?
Yes, they are commonly combined to balance exploration and coherence.
Category: BestPractice
Does Temperature Sampling affect calibration?
It changes output distributions; extreme temperatures can distort calibration of probabilities.
Category: Risk
Is Temperature Sampling useful for safety?
Lower temperatures reduce off-distribution outputs but aren’t a substitute for safety filters.
Category: Risk
How does Temperature Sampling impact diversity?
Higher temperature increases entropy, promoting more diverse token choices.
Category: Benefits
Does Temperature Sampling change training?
No, it is a decoding-time parameter; it does not affect model weights.
Category: Misunderstanding
What’s a common mistake with Temperature Sampling?
Using a single temperature for all tasks instead of tuning per use case.
Category: Mistakes
How to evaluate different temperatures?
A/B test task metrics and human ratings across temperatures to find the best trade-offs.
Category: Testing