DataTerminology.comNEW
Also known as: Graphics processor
A parallel processor optimized for graphics and numerical workloads, widely used to accelerate machine learning training and inference.
DESCRIPTION
GPUs contain thousands of cores enabling massive parallelism for matrix and tensor operations. Originally designed for graphics, they now power general-purpose computation (GPGPU) via CUDA or OpenCL. In ML, GPUs dramatically speed training and inference for deep networks. Consider memory bandwidth, VRAM size, and mixed-precision capabilities when selecting hardware. Cloud providers offer GPU instances; orchestration includes distributed training and model parallelism. Performance tuning involves kernel optimization, batching, and efficient data pipelines.
ALTERNATIVE DEFINITIONS
Wikipedia
A graphics processing unit (GPU) is a specialized electronic circuit designed to accelerate the creation of images in a frame buffer intended for output to a display.
Defines GPUs by their original purpose: accelerating image creation for displays.
NVIDIA
GPU-accelerated computing uses a graphics processing unit (GPU) together with a CPU to accelerate applications.
Introduces GPU acceleration in general-purpose computing alongside CPUs.
RELATED TERMS
CUDA Tensor Cores GPGPU Parallel Computing Deep Learning Distributed Training Mixed Precision Memory Bandwidth
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer MLOps Engineer Cloud Architect Data Engineer Chief Technology Officer
EXTERNAL SOURCES
- Wikipedia - Graphics processing unit
- NVIDIA - CUDA platform documentation
- PyTorch - CUDA semantics in PyTorch
- AWS - GPU instance families
TOP QUESTIONS
Why are GPUs important for AI workloads?
They accelerate tensor operations and parallelizable computations, dramatically reducing training times for deep neural networks.
Category: Relevancy
What should I consider when selecting a GPU for ML?
Evaluate memory (VRAM), compute capability, tensor cores, bandwidth, and software support (CUDA, cuDNN).
Category: Requirements
How do GPUs compare to TPUs?
TPUs target tensor operations with custom hardware; GPUs are general-purpose and widely supported, offering flexibility across workloads.
Category: Alternatives
What are common mistakes in GPU utilization?
Underutilized GPUs due to small batch sizes, CPU data bottlenecks, or improper memory management reduce performance.
Category: Mistakes
How do you scale training across multiple GPUs?
Use data or model parallelism, libraries like NCCL, and frameworks supporting distributed training (PyTorch DDP, Horovod).
Category: Management
What is mixed precision on GPUs?
Training with lower-precision types (FP16/BF16) speeds computation and reduces memory while maintaining accuracy via loss scaling.
Category: Overview
How do cloud GPUs differ from on-prem GPUs?
Cloud GPUs offer elasticity and managed services, while on-prem provides control and potentially lower long-term costs for steady workloads.
Category: Opinion
How do you monitor GPU usage?
Use nvidia-smi, framework profilers, and observability tools to track memory, utilization, and throughput.
Category: Testing
What is currently hot around GPUs?
New architectures with larger memory, faster interconnects, and specialized tensor cores for transformer workloads dominate.
Category: Topic