DataTerminology.comNEW
Also known as: PTQ
A technique that converts a trained model to lower precision, typically using calibration data, without retraining the original weights.
DESCRIPTION
Post-Training Quantization (PTQ) compresses models by mapping weights and often activations to lower-precision formats like int8 after training. It uses calibration samples to determine scaling and zero-points, balancing latency, throughput, and accuracy. PTQ is quick to apply, hardware-friendly, and often sufficient for deployment, especially for vision and transformer models. Accuracy can degrade for sensitive layers; selective quantization and per-channel scaling mitigate this. When accuracy loss is unacceptable, Quantization-Aware Training may be preferred. PTQ is supported by TensorFlow Lite, PyTorch, ONNX Runtime, and vendor runtimes like TensorRT.
ALTERNATIVE DEFINITIONS
TensorFlow
Post-training quantization is a technique to reduce the size of a model and increase CPU and hardware accelerator inference speed, with little degradation in model accuracy.
Emphasizes size reduction and inference speed with minimal accuracy loss.
PyTorch
Post Training Quantization converts weights (and optionally activations) to lower precision after training to reduce model size and improve performance.
Highlights conversion after training and optional activation quantization.
ONNX Runtime
ONNX Runtime quantization optimizes models by converting them from floating point to integer representations for faster inference and smaller footprint.
Implementation-focused statement on integer conversion for optimization.
RELATED TERMS
Scalar Quantization Weight-Only Quantization Activation Quantization Calibration Knowledge Distillation ONNX Runtime NVIDIA TensorRT TensorFlow Lite
COMPETING TERMS
Quantization-Aware Training Pruning
RELATIONS
- Broader: Quantization
- Narrower: AWQ AWQ Quantization
KNOWLEDGE AREAS
AI MLOps Data Storage & Operations
TARGET AUDIENCE
AI Engineer MLOps Engineer Data Scientist Cloud Architect
EXTERNAL SOURCES
- Wikipedia - Quantization (signal processing)
- TensorFlow - PTQ guide for TensorFlow Lite
- PyTorch - PyTorch PTQ documentation
- ONNX Runtime - ONNX Runtime quantization overview
TOP QUESTIONS
When should I use Post-Training Quantization?
Use it when you need quick deployment improvements in size and latency without the cost of retraining, and when moderate accuracy loss is acceptable.
Category: Usage
What calibration data is needed for Post-Training Quantization?
A small, representative sample of real data that exercises typical activation ranges is essential for accurate scaling.
Category: Requirements
How does Post-Training Quantization compare to Quantization-Aware Training?
PTQ is faster and simpler; QAT usually preserves accuracy better by simulating quantization during training.
Category: Comparison
Can I quantize only weights with Post-Training Quantization?
Yes, weight-only quantization reduces size and may provide speedups depending on hardware support.
Category: Implementation
What layers are risky for Post-Training Quantization?
Embedding tables and first/last layers can be sensitive; consider higher precision or selective exclusion.
Category: Risk
Does Post-Training Quantization help on GPUs?
Benefits depend on kernel support; CPUs, NPUs, and some GPUs see notable gains with int8 inference.
Category: Performance
How do I validate Post-Training Quantization results?
Compare accuracy, latency, and throughput against the FP baseline on representative workloads; ablate per-layer settings if needed.
Category: Testing
What are common mistakes in Post-Training Quantization?
Poor calibration data and quantizing all layers indiscriminately are common errors that cause accuracy drops.
Category: Mistakes
Can Post-Training Quantization be reversed?
Yes, you keep the original floating-point model; PTQ produces a separate quantized artifact for deployment.
Category: Operations