DataTerminology.comNEW
Also known as: Activation-Aware Weight Quantization
Activation-aware weight quantization technique that reduces precision while preserving model accuracy.
DESCRIPTION
AWQ (Activation-aware Weight Quantization) is a post-training quantization method for LLMs that selects and preserves critical weights relative to activation sensitivity. By focusing precision where it matters, AWQ achieves strong quality with low-bit weights. It is efficient, requires only a small calibration set, and integrates with popular inference stacks. AWQ has open-source implementations and wide adoption. It is commonly compared against GPTQ.
ALTERNATIVE DEFINITIONS
arXiv
We propose AWQ, activation-aware weight quantization, a low-bit method for LLMs that maintains accuracy with minimal calibration.
Introduces AWQ and highlights activation-aware selection for low-bit quantization.
MIT HAN Lab
Activation-aware Weight Quantization for LLMs.
Concise identification of the technique and target domain (LLMs).
RELATED TERMS
Quantization Post-Training Quantization AutoAWQ Per-Channel Quantization Calibration LLM Inference Weight-Only Quantization W4A16 Sensitivity Analysis
COMPETING TERMS
RELATIONS
KNOWLEDGE AREAS
TARGET AUDIENCE
EXTERNAL SOURCES
- Wikipedia - Quantization (signal processing)
- arXiv - Original AWQ paper
- MIT HAN Lab - Reference implementation
TOP QUESTIONS
What is the main idea behind AWQ?
AWQ prioritizes precision for weights most influential given activations, preserving quality in low-bit settings.
Category: Example
How does AWQ compare to GPTQ?
Both are post-training methods; AWQ leverages activation sensitivity, while GPTQ frames quantization as optimization. Relative performance varies by model.
Category: Alternatives
What calibration data does AWQ require?
A small, representative sample of domain text is typically enough to estimate activation statistics.
Category: Requirements
What bit-widths are common with AWQ?
4-bit weight-only quantization is common; higher bits may further preserve quality.
Category: Guidance
What are common mistakes using AWQ?
Poor calibration samples and skipping downstream evaluation can degrade results. Also, misconfiguring layers to quantize.
Category: Mistakes
Where does AWQ help most?
On large decoder-only LLMs where memory and latency constraints are tight, such as edge or on-prem deployments.
Category: Example
Does AWQ change model architecture?
No. It modifies weight representation post-training without retraining the architecture.
Category: Misunderstanding
What risks come with AWQ?
Over-aggressive quantization can increase hallucinations or reduce reasoning quality. Proper evaluation is essential.
Category: Risks
Is AWQ open source?
Yes, reference implementations are available and widely integrated into tooling.
Category: Example