DataTerminology.comNEW
A contiguous sequence of n items—such as characters, words, or tokens—from a sample of text or speech.
DESCRIPTION
N-grams underpin classical language modeling by assuming local Markov dependencies. They are used for text generation, spelling correction, and next-word prediction. Smoothing methods (e.g., Laplace, Kneser-Ney) address sparsity. Subword approaches like BPE improve handling of rare words. Transformers with contextual embeddings have largely superseded pure n-gram models for many tasks, but n-grams remain useful for baselines and feature engineering.
ALTERNATIVE DEFINITIONS
Wikipedia
An n-gram is a contiguous sequence of n items from a given sample of text or speech.
Defines n-grams by contiguity and applicability to text or speech.
NLTK Book
An n-gram is a sequence of n items from a given sample of text or speech.
Introduces n-grams within practical NLP feature extraction and modeling context.
RELATED TERMS
Language Model Markov Model Smoothing Tokenization Subword Tokenization WordPiece Stopwords Context Window Perplexity
COMPETING TERMS
Byte Pair Encoding Contextual Embeddings
KNOWLEDGE AREAS
AI Data Science Semantic & Knowledge Modeling
TARGET AUDIENCE
Data Scientist AI Engineer Software Developer Data Engineer
EXTERNAL SOURCES
- Wikipedia - N-gram
- NLTK - Chapter on n-grams and LM
- Stanford University - NLP course covering n-grams
TOP QUESTIONS
What is an N-gram in NLP?
It is a contiguous sequence of n items (characters, words, tokens) extracted from text or speech.
Category: Definition
How are N-grams used in language modeling?
They estimate probabilities of sequences using counts and smoothing to model local dependencies.
Category: Example
What are common N-gram sizes?
Unigrams, bigrams, trigrams, and four-grams are typical, chosen based on task and dataset size.
Category: Requirements
What are the limitations of N-grams?
They struggle with long-range dependencies and data sparsity without sophisticated smoothing or large corpora.
Category: Risks
How do subword methods compare to N-grams?
Subword tokenization like BPE improves handling of rare words and morphology, often outperforming word n-grams.
Category: Alternatives
Are N-grams still relevant?
Yes, for baselines, feature engineering, and lightweight models, though transformers dominate modern NLP.
Category: Relevancy
What smoothing techniques are used with N-grams?
Laplace, Good-Turing, and Kneser-Ney smoothing mitigate sparsity and improve probability estimates.
Category: Overview
How to evaluate N-gram language models?
Use perplexity and cross-entropy on held-out datasets to gauge predictive performance.
Category: Testing
Where to learn more about N-grams?
Study the NLTK book and Stanford CS224n materials for practical and theoretical coverage.
Category: Overview