DataTerminology.comNEW
Also known as: NAS
Automated methods that search and evaluate neural network designs to discover high-performing architectures for specific tasks.
DESCRIPTION
Neural Architecture Search explores model spaces using reinforcement learning, gradient-based relaxation, or evolutionary strategies alongside performance estimation. It aims to optimize architectures for accuracy, latency, and resource constraints. Techniques like DARTS and ENAS improve efficiency through weight sharing and continuous relaxation. NAS often integrates with hardware-aware objectives, pruning, and distillation. It’s valuable for tailoring models to deployment environments.
ALTERNATIVE DEFINITIONS
Wikipedia
Neural architecture search (NAS) is a technique for automating the design of artificial neural networks.
Provides high-level characterization of NAS as AutoML for architecture.
ArXiv
We propose using reinforcement learning to discover architectures by searching over a large space and training a controller to generate model descriptions.
Introduces RL-based NAS with controller models and search spaces.
Google AI Blog
NAS can be combined with scaling methods and efficiency constraints to produce models that achieve better accuracy with fewer resources.
Highlights efficiency-aware architecture search and scaling.
RELATED TERMS
AutoML Reinforcement Learning Bayesian Optimization Evolutionary Algorithms DARTS ENAS Architecture Search Space
COMPETING TERMS
Manual Architecture Design Hyperparameter Tuning
RELATIONS
- Broader: Search
KNOWLEDGE AREAS
AI Data Science MLOps Software Engineering
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Neural architecture search
- Zoph & Le - NAS with reinforcement learning paper
- Liu et al. - DARTS gradient-based NAS
- AutoKeras - Open-source AutoML NAS framework
- Google AI Blog - Efficiency-aware model scaling
TOP QUESTIONS
Why use Neural Architecture Search?
NAS discovers architectures tailored to tasks and constraints, improving accuracy or efficiency without manual exploration.
Category: Relevancy
What are common mistakes with NAS?
Underestimating compute costs, using biased search spaces, and skipping proper validation can produce overfitted or impractical architectures.
Category: Mistakes
How do you make NAS efficient?
Use weight sharing, low-fidelity estimates, early stopping, and multi-objective optimization for speed and practicality.
Category: Requirements
What risks come with NAS?
High computational expense, reproducibility challenges, and potential brittleness when transferring across datasets or hardware.
Category: Risks
What are notable NAS methods?
RL-based NAS, DARTS (gradient-based), ENAS (efficient weight sharing), and evolutionary strategies are widely studied.
Category: Overview
Where to start learning NAS?
Read Zoph & Le’s paper, then DARTS. Experiment with AutoKeras or open-source NAS toolkits on small datasets.
Category: Learning
How does NAS interact with hyperparameter optimization?
They complement each other: NAS selects architectures while HPO tunes their training settings for best performance.
Category: Example
What’s hot in NAS now?
Hardware-aware NAS and efficient search for transformers, balancing latency, memory, and accuracy.
Category: Topic
Can NAS be used for on-device models?
Yes, by optimizing for model size and latency, NAS can target edge deployments with strict constraints.
Category: Application