DataTerminology.comNEW
Also known as: CBOW
A neural word embedding model that predicts a target word from surrounding context words to learn dense vector representations.
DESCRIPTION
CBOW is one of the Word2Vec architectures popularized for efficient word embeddings. It aggregates context words within a window to predict the center word, learning embeddings via softmax or hierarchical softmax and often negative sampling. CBOW is fast and performs well on frequent words, while Skip-Gram excels with rare words. Learned embeddings capture semantic and syntactic relationships. CBOW influenced modern contextual models and remains a baseline for NLP tasks.
ALTERNATIVE DEFINITIONS
Mikolov et al.
We present two novel model architectures for computing continuous vector representations of words: Continuous Bag-of-Words and Skip-gram. The CBOW model predicts the current word based on the context.
Original paper explicitly stating CBOW’s prediction objective.
TensorFlow
In the CBOW model, the model predicts a target word from a surrounding context.
Tutorial-level description emphasizing target-from-context prediction.
Gensim
CBOW trains to predict the target word based on context words, while skip-gram predicts surrounding words given the current word.
Contrasts CBOW with Skip-Gram in practical library docs.
RELATED TERMS
Word2Vec Negative Sampling Embeddings GloVe FastText Softmax
COMPETING TERMS
RELATIONS
- Broader: Bag-of-Words
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Word2vec
- arXiv - Original Word2Vec paper
- TensorFlow - Word2Vec/CBOW tutorial
- PyTorch - Embeddings tutorial with CBOW
TOP QUESTIONS
How does Continuous Bag-of-Words (CBOW) learn embeddings?
It predicts a target word from surrounding context, adjusting vector representations to maximize prediction likelihood.
Category: Definition
When should I use CBOW instead of Skip-Gram?
Use CBOW for speed and frequent words; Skip-Gram is better for rare words and capturing detailed relations.
Category: Guidance
What loss functions are common with CBOW?
Softmax cross-entropy, hierarchical softmax, and negative sampling are common choices.
Category: Implementation
How does window size affect CBOW?
Larger windows capture broader contexts, while smaller windows emphasize local syntactic relationships.
Category: Practical
What are typical pitfalls using CBOW?
Insufficient data, poor tokenization, or unhandled subwords can limit embedding quality.
Category: Mistakes
Is CBOW outdated with modern transformers?
While surpassed for many tasks, CBOW remains useful for lightweight baselines and specialized embedding needs.
Category: Opinion
How do subword models relate to CBOW?
FastText integrates subword information, improving representations for rare and morphologically rich words.
Category: Concept
Can CBOW be used outside NLP?
Yes, similar objectives embed items like products, code tokens, or graph nodes (e.g., node2vec).
Category: Use Cases
How to evaluate CBOW embeddings?
Use intrinsic tests (word analogies, similarity) or extrinsic performance on downstream tasks.
Category: Evaluation