DataTerminology.comNEW
Also known as: LSA
A linear algebra method using singular value decomposition to reveal latent semantic structure in term-document matrices.
DESCRIPTION
Latent Semantic Analysis (LSA) projects high-dimensional term-document matrices into a lower-dimensional space via SVD. It captures co-occurrence patterns that reveal latent concepts, improving retrieval and similarity. LSA reduces noise and synonymy effects but can conflate homonyms. It is unsupervised and language-agnostic, historically important before probabilistic topic models. In practice, truncated SVD on TF-IDF matrices is commonly used. Alternatives like LDA and NMF offer probabilistic or parts-based semantics.
ALTERNATIVE DEFINITIONS
Wikipedia
Latent semantic analysis (LSA) is a technique in natural language processing, in particular distributional semantics, of analyzing relationships between a set of documents and the terms they contain.
Emphasizes analyzing document-term relationships within distributional semantics.
Deerwester et al. (1990) summary
LSA is a method for capturing the underlying latent semantic structure in word usage across documents by applying a singular value decomposition to the term-document matrix.
Links LSA directly to SVD over the term-document matrix.
scikit-learn
Latent Semantic Analysis (LSA) can be performed using truncated SVD on term-document matrices to reduce dimensionality while preserving the most significant concepts.
Practical guidance via truncated SVD implementation.
RELATED TERMS
Singular Value Decomposition Vector Space Model Topic Modeling Term Frequency–Inverse Document Frequency Low-Rank Approximation Word Embedding Feature Reduction Co-occurrence Matrix Semantic Similarity
COMPETING TERMS
Latent Dirichlet Allocation Non-negative Matrix Factorization
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist Data Analyst AI Engineer
EXTERNAL SOURCES
- Wikipedia - Latent semantic analysis
- scikit-learn - How to implement LSA with TruncatedSVD
- University of Colorado - Introductory LSA paper and resources
- Wikipedia - Background on SVD used by LSA
TOP QUESTIONS
What problem does Latent Semantic Analysis solve?
It reduces dimensionality of text data to capture latent concepts, improving retrieval and similarity beyond raw term matching.
Category: Definition
How does LSA use SVD?
LSA factorizes the term-document matrix and keeps top singular components, projecting text into a lower-dimensional semantic space.
Category: Requirements
What are LSA’s main limitations?
It is linear, ignores word order, and can mix homonyms. It also lacks probabilistic interpretability.
Category: Risk
How does LSA compare to LDA?
LDA is probabilistic with interpretable topics, while LSA is linear algebra-based and can be faster but less interpretable.
Category: Alternatives
When is LSA still relevant?
For lightweight semantic retrieval, small datasets, and as a strong baseline before neural models.
Category: Relevancy
What is a common mistake with LSA?
Skipping proper normalization/TF-IDF weighting before SVD, leading to poor semantic structure.
Category: Mistakes
How many components should LSA keep?
Typically a few hundred, tuned by validation for retrieval quality versus cost.
Category: Requirements
Can LSA be combined with word embeddings?
Yes, hybrid features or initial reductions can complement neural embeddings in pipelines.
Category: Example
What is a good way to start learning LSA?
Read the original Deerwester paper and implement TruncatedSVD with scikit-learn on a TF-IDF corpus.
Category: Example