DataTerminology.comNEW
Also known as: On-Device AI, Embedded AI, AI at the Edge
Running AI models on edge devices to reduce latency, lower bandwidth use, and strengthen privacy.
DESCRIPTION
Edge AI executes inference on devices like smartphones, gateways, cameras, and microcontrollers. It reduces round-trip latency and bandwidth costs by processing near data sources. Techniques include quantization, pruning, distillation, and hardware acceleration. Edge deployments demand robust model management, updates, telemetry, and offline resilience. Use cases span vision, speech, anomaly detection, and personalized experiences.
ALTERNATIVE DEFINITIONS
NVIDIA
Edge AI is the deployment of artificial intelligence (AI) models on devices located at the edge of the network, where data is generated.
Focuses on deployment location and data proximity.
IBM
Edge AI is the use of artificial intelligence at the data source or edge devices, enabling faster decision-making and reduced bandwidth consumption.
Emphasizes speed and bandwidth advantages at the edge.
Qualcomm
Edge AI brings intelligence to devices by running models locally, helping preserve privacy and deliver low latency experiences.
Highlights privacy and latency benefits on-device.
RELATED TERMS
Edge Computing Model Quantization TensorFlow Lite ONNX Runtime Federated Learning Delays MLOps IoT Graphics Processing Unit Inference
COMPETING TERMS
Cloud AI
KNOWLEDGE AREAS
AI IoT & Edge Data Real-time & Streaming Data Infrastructure MLOps Cloud
TARGET AUDIENCE
AI Engineer MLOps Engineer Data Engineer Cloud Architect Hardware Engineer Solution Architect
EXTERNAL SOURCES
- Wikipedia - Edge computing
- TensorFlow - TensorFlow Lite for on-device inference
- ONNX Runtime - ONNX Runtime Mobile overview
- AWS - AWS IoT Greengrass edge platform
- Microsoft - Azure IoT Edge deployment guidance
TOP QUESTIONS
Why choose Edge AI over Cloud AI?
Edge AI reduces latency, saves bandwidth, and enhances privacy by processing locally. It’s ideal for real-time or bandwidth-constrained scenarios.
Category: Alternatives
What hardware powers Edge AI?
NPUs, GPUs, DSPs, and specialized accelerators on devices, plus ARM-based CPUs, enable efficient on-device inference.
Category: Overview
What are common mistakes with Edge AI deployments?
Ignoring model optimization and telemetry leads to poor performance and observability. Plan for updates, monitoring, and failure modes.
Category: Mistakes
How does Edge AI protect privacy?
Data can remain on-device, limiting exposure. Combined with federated learning, it avoids centralized raw data aggregation.
Category: Risk
What is required to operationalize Edge AI?
Containerization or runtime packaging, model quantization, secure update channels, device fleet management, and inference monitoring.
Category: Requirements
How to measure Edge AI performance?
Track latency, throughput, power consumption, accuracy under quantization, and model drift metrics.
Category: Testing
What is a hot topic around Edge AI?
On-device LLMs and multimodal inference with efficient transformers are rapidly evolving.
Category: Topic
What’s the biggest misunderstanding about Edge AI?
That it cannot handle complex models; careful optimization enables sophisticated on-device inference.
Category: Misunderstanding
How do you update models in Edge AI?
Use secure OTA updates, versioning, and staged rollouts to manage risk and ensure consistency.
Category: Management