DataTerminology.comNEW
Also known as: Retrieval phase, Retrieval, Candidate retrieval
Initial recommender stage that retrieves a small, relevant subset of items from a large corpus for downstream ranking.
DESCRIPTION
Candidate generation, also called retrieval, narrows millions of items to hundreds using fast similarity models and indexing. Two-tower architectures learn user and item embeddings for efficient ANN search. This stage prioritizes recall and latency, passing candidates to rankers for fine-grained scoring. Practical systems use vector databases, filtering rules, and freshness constraints. Monitoring balances coverage, diversity, and responsiveness.
ALTERNATIVE DEFINITIONS
Google Research
The candidate generation model is designed to retrieve a small subset of videos from a very large corpus.
Captures the retrieval goal within large-scale recommendation.
TensorFlow Recommenders
The retrieval task is to select a small subset of items from a large set of candidates.
Defines retrieval concisely as subset selection at scale.
RELATED TERMS
Recommender System Two-Tower Model Embeddings Approximate Nearest Neighbor ScaNN FAISS Negative Sampling Filtering Personalization
COMPETING TERMS
KNOWLEDGE AREAS
AI Data Science MLOps Real-time & Streaming Data
TARGET AUDIENCE
AI Engineer MLOps Engineer Data Scientist Software Developer
EXTERNAL SOURCES
- Wikipedia - Recommender system
- TensorFlow - Retrieval tutorial with two-towers
- FAISS - ANN library for vector search
- Google Research - ScaNN ANN search library
- NVIDIA - Merlin for scalable recommenders
TOP QUESTIONS
Why separate Candidate Generation from Ranking?
Retrieval optimizes recall and latency to narrow the search space; ranking applies heavier models to a small candidate set for precision.
Category: Concepts
What models are common in Candidate Generation?
Two-tower embedding models, matrix factorization, and graph-based methods are common. ANN indices accelerate retrieval.
Category: Techniques
How do you ensure freshness in Candidate Generation?
Use incremental indexing, event-driven updates, and time-decay in embeddings or filters.
Category: Best practices
What metrics evaluate Candidate Generation?
Recall@K, coverage, latency, and diversity are key. Robust A/B testing measures downstream impact.
Category: Validation
What are common pitfalls in Candidate Generation?
Overfitting to head items, stale indices, and ignoring diversity degrade user experience and novelty.
Category: Mistakes
How does filtering interact with retrieval?
Hard filters (e.g., availability, policy) are applied pre- or post-retrieval to enforce constraints without harming latency.
Category: Management
What’s a hot topic in Candidate Generation?
Vector databases and hybrid retrieval (dense plus sparse) improve scale and explainability.
Category: Topic
How do you deploy Candidate Generation in production?
Use feature stores, model registries, and scalable ANN services with monitoring for drift, latency, and index health.
Category: Operations
What is negative sampling’s role in Candidate Generation?
It trains retrieval models efficiently by contrasting positives against sampled negatives to shape embedding space.
Category: Concepts