DataTerminology.comNEW
Also known as: Model Production
The process of packaging, releasing, and operating trained models in production environments to serve predictions reliably.
DESCRIPTION
Model deployment operationalizes trained models for real users and systems. It includes packaging, versioning, and exposing inference via APIs, batch jobs, or streaming. Practices such as canary releases, blue–green deployments, and shadow testing reduce risk. CI/CD and infrastructure-as-code enable repeatable, auditable releases. Monitoring covers performance, drift, latency, and cost, with alerts and automated rollback. Security involves authentication, authorization, and data protection. Deployment choices depend on latency needs, throughput, cost, and compliance. Success requires collaboration across data science, engineering, and operations. Documentation and observability ensure maintainability and trust.
ALTERNATIVE DEFINITIONS
Google Cloud
Deploying a model makes it available for online prediction by allocating compute resources and exposing endpoints that receive prediction requests and return inference results.
Defines deployment as provisioning resources and endpoints for predictions.
AWS
You deploy a model in Amazon SageMaker to a persistent and secure endpoint or use batch transform to run inference on large datasets without persistent endpoints.
Clarifies online endpoints vs. batch transform modes.
Microsoft Learn
Deploy models to Azure Machine Learning online endpoints to host REST APIs for real-time inference, manage traffic, and roll out new versions safely.
Highlights traffic management and safe rollout for real-time inference.
RELATED TERMS
Model Serving Online Inference Batch Inference Continuous Integration and Continuous Delivery for Machine Learning Model Registry Canary Release A/B Testing Feature Store Model Monitoring Rollback
COMPETING TERMS
Model Training
KNOWLEDGE AREAS
MLOps AI Software Engineering Observability & Telemetry Data Engineering
TARGET AUDIENCE
MLOps Engineer AI Engineer Data Scientist DevOps Engineer Software Developer
EXTERNAL SOURCES
- Wikipedia - Model deployment
- Kubeflow - Serverless model serving on Kubernetes
- Seldon - Open-source model serving framework
- TensorFlow - TF Serving for production inference
TOP QUESTIONS
What are common strategies for Model Deployment?
Blue–green, canary, and shadow deployments reduce risk. Choose based on traffic, tolerance for errors, and monitoring capabilities.
Category: How-To
How does Model Deployment differ from Model Training?
Training optimizes model parameters; deployment packages and operates models for inference under reliability, latency, and security constraints.
Category: Comparison
What metrics should be monitored after Model Deployment?
Latency, throughput, error rates, data drift, prediction drift, and business KPIs are essential for detecting issues early.
Category: Metrics
How is rollback handled in Model Deployment?
Versioned artifacts and traffic management enable quick rollback to previous stable versions when anomalies are detected.
Category: Management
What are security considerations for Model Deployment?
Secure endpoints with authentication and authorization, encrypt data in transit and at rest, and protect against prompt or model injection.
Category: Risks
When should batch inference be preferred over online inference?
Use batch for non-real-time workloads with large volumes and relaxed latency; online suits interactive, low-latency use cases.
Category: Requirements
What is the role of a model registry in deployment?
It tracks versions, lineage, and approvals, enabling reproducible releases and controlled promotions through environments.
Category: Management
How do you test models before deployment?
Use unit tests, data-contract checks, canary traffic, and A/B tests with holdouts to validate performance and stability.
Category: Testing
What is a hot topic in Model Deployment?
Serverless and GPU autoscaling for LLM inference, plus efficient quantization and batching, are rapidly evolving.
Category: Topic