🧠Active Learning

NEW

Learning strategy where the model selects informative unlabeled examples for labeling to improve performance with fewer labels.

DESCRIPTION

Active learning reduces labeling costs by querying the most informative samples for annotation. Common strategies include uncertainty sampling, query-by-committee, and expected model change. It is valuable in domains with expensive labels, such as medical imaging or legal documents. Human-in-the-loop pipelines manage selection, labeling, and model retraining. Practical deployments must mitigate sampling bias and maintain evaluation sets independent of queried data. With modern LLMs, active learning integrates weak supervision and self-training to bootstrap data efficiency.

ALTERNATIVE DEFINITIONS

Wikipedia

Active learning is a special case of semi-supervised machine learning in which a learning algorithm can interactively query a user to obtain the desired outputs at new data points.

Highlights interactive querying of an oracle for labels.

Source

Burr Settles

Active learning is the problem of choosing the data from which an algorithm learns, such that it learns better with less training effort.

Emphasizes label efficiency and data selection strategy.

Source

RELATED TERMS

Human-in-the-Loop Uncertainty Sampling Query by Committee Pool-based Sampling Labeling Annotation Tools Data Efficiency Curriculum Learning Pseudo-Labeling

COMPETING TERMS

Passive Learning Semi-supervised Learning

RELATIONS

• Broader: Machine Learning

KNOWLEDGE AREAS

AI Data Science MLOps

TARGET AUDIENCE

AI Engineer Data Scientist MLOps Engineer AI Product Manager

EXTERNAL SOURCES

• Wikipedia - Active learning (machine learning)
• Distill - Interactive explanation of strategies
• modAL - Python toolkit for active learning

TOP QUESTIONS

When is Active Learning most beneficial?

It shines when unlabeled data is abundant but labels are expensive, enabling substantial accuracy gains per labeled example.

Category: Use-case

What query strategies are common in Active Learning?

Uncertainty sampling, query-by-committee, expected error reduction, and diversity-based selection are widely used.

Category: Technique

How to avoid bias in Active Learning?

Maintain a held-out evaluation set, incorporate diversity constraints, and periodically refresh the pool to counter selection bias.

Category: Risks

What infrastructure supports Active Learning at scale?

Annotation platforms, data versioning, model serving for scoring pools, and orchestration for iterative retraining are essential.

Category: Infrastructure

How does Active Learning interact with class imbalance?

Query strategies can overfocus on majority uncertainty. Balance by reweighting, stratified sampling, or targeted rare-class exploration.

Category: Practice

Can Active Learning work with deep learning?

Yes. Batch-mode strategies, Bayesian approximations, and embeddings-based diversity help scale to deep models.

Category: Feasibility

How to measure progress in Active Learning?

Plot learning curves versus labeled count, track calibration, and compare to passive baselines under a fixed labeling budget.

Category: Evaluation

What are risks of human-in-the-loop in Active Learning?

Annotation inconsistency and latency can limit benefits. Use guidelines, double labeling, and adjudication to ensure quality.

Category: Risk

Is Active Learning compatible with weak supervision?

Yes. Combine heuristics or model-generated labels to prefilter candidates, reserving human effort for the most uncertain items.

Category: Integration