DataTerminology.comNEW
Also known as: PCA
A linear dimensionality reduction method that projects data onto orthogonal components maximizing variance.
DESCRIPTION
Principal Component Analysis transforms correlated variables into a smaller set of uncorrelated components capturing most variance. It leverages eigen-decomposition or SVD of the covariance matrix and often requires standardization. PCA aids visualization, compression, noise reduction, and pre-processing for downstream models. While linear and global, it is fast and interpretable. Nonlinear methods like t-SNE or UMAP may better preserve local structure for visualization.
ALTERNATIVE DEFINITIONS
Wikipedia
Principal component analysis (PCA) is a statistical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables called principal components.
Classical, formal description emphasizing orthogonal transformation and uncorrelated components.
scikit-learn
Linear dimensionality reduction using Singular Value Decomposition of the data to project it to a lower dimensional space.
Implementation-oriented definition focusing on SVD-based projection.
Karl Pearson (1901)
On Lines and Planes of Closest Fit to Systems of Points in Space.
Seminal work laying the foundations of PCA as closest-fit subspace.
RELATED TERMS
Feature Reduction Singular Value Decomposition Eigenvectors Factor Analysis Whitening Explained Variance Feature Scaling Covariance Matrix
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer Data Analyst Information Analyst
EXTERNAL SOURCES
- Wikipedia - Principal component analysis
- scikit-learn - Comprehensive PCA user guide
- NumPy - SVD routine underpinning PCA
- UC Berkeley - Lecture notes on PCA
TOP QUESTIONS
When should I use Principal Component Analysis?
Use PCA to reduce dimensionality, visualize high-dimensional data, or remove multicollinearity before modeling.
Category: UseCases
How many components should I keep in PCA?
Choose components to meet an explained variance threshold (e.g., 90–95%) or using scree plots and cross-validation.
Category: BestPractice
Should I standardize features before PCA?
Yes, if features are on different scales; standardization prevents dominance by high-variance features due to scale.
Category: Requirements
Is PCA linear?
Yes. PCA is a linear method; nonlinear structure may require t-SNE or UMAP for visualization.
Category: Definition
Does PCA improve prediction accuracy?
Sometimes, by reducing noise and collinearity, but it may discard informative variance; validate on your task.
Category: Opinion
How is PCA computed?
Via eigen-decomposition of the covariance matrix or SVD of the data matrix after centering.
Category: Techniques
What are common mistakes with PCA?
Skipping feature scaling, misinterpreting components, and applying PCA without evaluating downstream performance.
Category: Mistakes
Is PCA interpretable?
Components are linear combinations of features; loadings enable interpretation, though it can be challenging for many features.
Category: Misunderstanding
How does PCA compare to Factor Analysis?
PCA is variance-maximizing; factor analysis models latent factors and error explicitly. They answer different questions.
Category: Comparison