DataTerminology.comNEW
Technique mapping arbitrary features into a fixed-size vector using a hash function, enabling scalable vectorization.
DESCRIPTION
Feature hashing, or the hashing trick, transforms tokens or feature-value pairs into fixed-dimensional sparse vectors via hash functions. It offers constant-memory vectorization for high-cardinality or streaming contexts. Collisions introduce controlled aliasing, mitigated by larger dimensions or signed hashing. Common in NLP, recommender systems, and online learning, it simplifies deployment by avoiding dictionaries. Trade-offs include interpretability loss and collision-induced noise.
ALTERNATIVE DEFINITIONS
Wikipedia
Feature hashing, also known as the hashing trick, is a method of vectorizing features that allows for a very large number of possible features to be mapped to a fixed-size vector.
Highlights mapping large feature spaces to fixed-length vectors.
scikit-learn
Feature hashing projects features into a lower-dimensional space using a hash function, which enables efficient processing of high-dimensional data.
Emphasizes efficiency for high-dimensional inputs.
Apache Spark MLlib
HashingTF maps a sequence of terms to their term frequencies using the hashing trick, resulting in a fixed-size feature vector.
Concrete API framing via HashingTF for text processing.
RELATED TERMS
Hashing Trick Hash Function Sparse Vector Feature Reduction High-Cardinality Categorical Text Vectorization Collision HashingTF Streaming Features
COMPETING TERMS
One-Hot Encoding Dictionary Vectorizer
RELATIONS
- Broader: Hashing
KNOWLEDGE AREAS
AI Data Science Data Engineering Real-time & Streaming Data
TARGET AUDIENCE
Data Scientist Data Engineer AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Feature hashing
- scikit-learn - Docs on hashing trick
- Apache Spark - HashingTF in MLlib
- Vowpal Wabbit - Online learning with hashing
TOP QUESTIONS
When is Feature Hashing useful?
When dealing with high-cardinality features, streaming inputs, or unknown vocabularies. It provides fixed memory usage and avoids dictionary management.
Category: Relevancy
What are collisions in Feature Hashing?
Different features mapping to the same index. Collisions inject noise but are manageable with sufficient dimensionality and signed hashing.
Category: Misunderstanding
How do you choose the hash vector size?
Balance collision rate and memory constraints. Start with powers of two and adjust based on validation performance and collision analysis.
Category: Requirements
Does Feature Hashing harm interpretability?
Yes, interpretability decreases because indices don’t map back to explicit tokens. Use dictionary vectorizers when explainability is required.
Category: Risks
Can Feature Hashing be combined with TF-IDF?
Yes. Hash term indices and compute TF-IDF weights in the hashed space for efficient text vectorization.
Category: Example
Is signed hashing important?
Signed hashing reduces bias by allowing collisions to cancel out, often improving performance compared to unsigned hashing.
Category: Opinion
How does Feature Hashing impact model training time?
It speeds up vectorization and reduces memory footprint, enabling faster training on sparse linear models and online learners.
Category: Overview
What are alternatives to Feature Hashing?
One-hot encoding with dictionaries, subword tokenization, learned embeddings, and dimensionality reduction techniques like PCA on count vectors.
Category: Alternatives
How to monitor Feature Hashing in production?
Track collision rates using sampled reverse dictionaries, monitor performance drift, and adjust hash dimension as needed.
Category: Testing