❓Missing Data

NEW

Absent values in a dataset that require handling through imputation, modeling, or exclusion.

DESCRIPTION

Missing data arise when values for variables are not recorded, leading to gaps that bias analysis if unaddressed. Mechanisms include MCAR, MAR, and MNAR, each requiring different handling strategies. Common remedies include listwise deletion, single imputation, and multiple imputation, with the latter often preferred for inferential validity. In ML, algorithms may handle missingness natively or via preprocessing (e.g., imputers). Proper documentation, missingness indicators, and sensitivity analyses are vital. Governance and privacy decisions may intentionally create missingness through redaction or consent withdrawal.

ALTERNATIVE DEFINITIONS

Wikipedia

Missing data occurs when no data value is stored for the variable in an observation.

Provides a concise, general description applicable across domains.

Source

UCLA Statistical Consulting

Missing data refers to the situation where information expected to be present in a dataset is not recorded or is unavailable for some observations.

Highlights practical perspective in applied statistical analysis.

Source

RELATED TERMS

Imputation Listwise Deletion Multiple Imputation MCAR MAR MNAR Data Quality Censoring

COMPETING TERMS

Complete Data

RELATIONS

• Broader: Data

KNOWLEDGE AREAS

Data Science Data Quality Data Management Data Privacy & Compliance

TARGET AUDIENCE

Data Scientist Data Engineer Data Steward Data Analyst MLOps Engineer

EXTERNAL SOURCES

• Wikipedia - Missing data
• scikit-learn - Imputation strategies in scikit-learn
• Penn State STAT Online - Introduction to missing data mechanisms
• CRAN (mice package) - Multiple imputation with mice

TOP QUESTIONS

What are MCAR, MAR, and MNAR in Missing Data?

They describe mechanisms of missingness: completely at random, at random conditional on observed data, and not at random. Mechanism informs appropriate handling.

Category: Concept

Why is listwise deletion risky for Missing Data?

It can bias results and reduce power unless data are MCAR. Prefer imputation or model-based methods when assumptions aren’t met.

Category: Risks

When should Multiple Imputation be used for Missing Data?

Use for inferential tasks when MAR is plausible. It propagates uncertainty better than single imputation.

Category: Best practices

How do ML models handle Missing Data?

Some tree-based models handle missing splits; others require imputation. Use dedicated imputers or encodings as needed.

Category: Application

What are common mistakes handling Missing Data?

Imputing with mean blindly, ignoring missingness indicators, and skipping sensitivity analyses. Document assumptions and validate.

Category: Mistakes

How should Missing Data be documented?

Use data dictionaries, missingness rates, mechanisms, and imputation methods. Track changes in lineage for auditability.

Category: Management

Can privacy requirements cause Missing Data?

Yes. Redaction, consent withdrawal, or minimization policies may remove values. Plan analyses accordingly.

Category: Compliance

What are simple imputation methods for Missing Data?

Mean/median for numeric, mode for categorical, hot-deck, or KNN. Evaluate bias they introduce.

Category: Overview

How to evaluate imputation quality for Missing Data?

Use holdout with simulated missingness, compare distributions, and assess downstream model performance changes.

Category: Testing