DataTerminology.comNEW
Number of training examples processed together before updating model parameters.
DESCRIPTION
Batch size controls the trade-off between gradient estimate quality and computational efficiency. Larger batches increase throughput but may harm generalization and require learning rate scaling. Smaller batches improve regularization via noise but reduce hardware utilization. Gradient accumulation can simulate larger batches within memory limits. Selection depends on model, data, hardware, and training objectives.
ALTERNATIVE DEFINITIONS
Google Developers ML Glossary
The number of examples in a batch.
Minimal, precise definition focused on size concept.
Keras
A batch is a number of samples processed before the model is updated. The number of epochs is the number of complete passes through the training dataset.
Clarifies batch relative to epochs and updates in deep learning frameworks.
RELATED TERMS
Batch Mini-Batch Learning Rate Gradient Accumulation Superclass Throughput GPU Memory Optimizer
COMPETING TERMS
Full-Batch Training Stochastic (Batch Size 1)
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Stochastic gradient descent
- arXiv - Generalization gap vs. large batch size
- PyTorch - Batch size and LR tuning tips
- Google Developers - Learning rate interplay with batch size
TOP QUESTIONS
How do I choose Batch Size?
Start with a size that saturates your hardware, then adjust for validation performance; scale learning rate with batch size if needed.
Category: Practice
Do larger Batch Sizes harm generalization?
Sometimes; larger batches can reduce gradient noise and overfit minima. Techniques like learning rate warmup help.
Category: Risk
What is gradient accumulation?
Accumulating gradients over several small batches to emulate a larger effective batch within memory constraints.
Category: Technique
How does Batch Size affect training speed?
Throughput increases with batch size until memory or bandwidth bottlenecks, then plateaus.
Category: Tradeoff
Should I change Batch Size during training?
Curriculum or schedule-based changes exist, but most setups keep it constant and tune learning rate schedules instead.
Category: Opinion
How does Batch Size interact with Batch Normalization?
Too-small batches yield noisy statistics; consider GroupNorm or LayerNorm when batches are tiny.
Category: Interaction
Is Batch Size relevant for inference?
Yes, larger inference batches improve throughput but may increase latency; balance with SLAs.
Category: Operations
Does Batch Size affect reproducibility?
Yes, it changes stochasticity and may alter convergence; fix seeds and frameworks’ deterministic modes if needed.
Category: Testing
What is micro-batching?
Splitting a large batch across steps or devices to fit memory while preserving effective batch size.
Category: Technique