💾HBM

NEW

Also known as: High Bandwidth Memory

High Bandwidth Memory stacked DRAM providing high throughput for GPUs and accelerators.

DESCRIPTION

HBM is a JEDEC-standard stacked DRAM technology using through-silicon vias to deliver massive memory bandwidth with efficient power consumption. It’s widely used in GPUs, AI accelerators, and HPC devices to feed compute units with high data rates. Generations like HBM2 and HBM3 increase capacity and bandwidth. Its 3D stacking reduces latency and footprint compared to discrete GDDR modules. Adoption accelerates as AI workloads demand larger, faster memory.

ALTERNATIVE DEFINITIONS

Wikipedia

High Bandwidth Memory (HBM) is a high-performance RAM interface for 3D-stacked DRAMs with a wide interface, designed to provide very high bandwidth.

Describes interface, 3D stacking, and bandwidth focus.

Source

JEDEC

JESD235 establishes the standard for High Bandwidth Memory (HBM), defining architecture, electrical and mechanical characteristics.

Specifies formal standard scope across characteristics.

Source

Micron

Micron HBM delivers exceptional bandwidth and energy efficiency to accelerate AI, graphics, and high-performance computing workloads.

Emphasizes performance and efficiency for AI and HPC.

Source

RELATED TERMS

HBM2 HBM3 Graphics Processing Unit Stacked DRAM Through-Silicon Via HBM2E AI Accelerator Memory Bandwidth

COMPETING TERMS

GDDR6 DDR5

KNOWLEDGE AREAS

Infrastructure AI Data Engineering

TARGET AUDIENCE

Hardware Engineer AI Engineer Cloud Architect Chief Technology Officer

EXTERNAL SOURCES

• Wikipedia - High Bandwidth Memory
• JEDEC - HBM standard specification
• Micron - HBM product overview
• Samsung - Samsung HBM technology
• NVIDIA Developer - HBM in NVIDIA GPUs

TOP QUESTIONS

What is HBM and why does it matter?

HBM is stacked DRAM providing very high bandwidth and energy efficiency, critical for feeding AI accelerators and GPUs.

Category: Definition

How does HBM differ from GDDR6?

HBM uses 3D stacking and wide interfaces for bandwidth and efficiency, while GDDR6 is discrete DRAM optimized for cost and clock speeds.

Category: Comparison

What are HBM generations like HBM2 and HBM3?

Successive generations increase capacity, bandwidth, and reliability features to meet growing compute demands.

Category: Overview

Where is HBM used in AI systems?

In GPUs and AI accelerators to stage tensors and parameters for fast model training and inference.

Category: Application

What are common challenges with HBM?

Thermal management, packaging complexity, and cost can constrain availability and scalability.

Category: Risks

What is currently a hot topic around HBM?

HBM3 adoption for large-scale LLM training and next-gen accelerator memory architectures.

Category: Topic

How does HBM improve energy efficiency?

Shorter interconnects and wider interfaces reduce I/O power per bit compared to traditional DRAM modules.

Category: Example

What role do TSVs play in HBM?

Through-silicon vias enable vertical stacking and high-bandwidth interconnects between DRAM layers.

Category: Requirements

Is HBM standardized?

Yes, JEDEC’s JESD235 standard defines HBM’s characteristics, ensuring interoperability across vendors.

Category: Regulation