🔢Logistic Regression

NEW

Also known as: Logit Regression, Logit Model

Classification model using a logistic function to estimate probabilities of class membership, typically for binary outcomes.

DESCRIPTION

Logistic regression models the log-odds of a binary outcome as a linear combination of features, producing probabilities via the sigmoid function. Parameters are usually estimated by maximum likelihood. Regularized variants handle multicollinearity and high-dimensional features. Extensions include multinomial and ordinal logistic regression. It remains a strong baseline for interpretability, calibration, and production robustness.

ALTERNATIVE DEFINITIONS

Wikipedia

In statistics, logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable.

Highlights logistic function and binary outcomes.

Source

scikit-learn

Logistic Regression (aka logit, MaxEnt) classifier.

Places logistic regression among linear models for classification.

Source

RELATED TERMS

Logistic Function Odds Ratio Logit Binary Classification Maximum Likelihood Estimation Regularization ROC Curve Multinomial Logistic Regression Probit Regression Decision Boundary

COMPETING TERMS

Support Vector Machine Decision Tree

RELATIONS

• Broader: Regression

KNOWLEDGE AREAS

Data Science AI Mathematics

TARGET AUDIENCE

Data Scientist AI Engineer MLOps Engineer Data Analyst

EXTERNAL SOURCES

• Wikipedia - Logistic regression
• scikit-learn - API and parameters for logistic regression
• Penn State STAT 462 - Course notes on logistic regression
• NIST - NIST guidance on logistic regression

TOP QUESTIONS

When should I use Logistic Regression?

Use it for binary outcomes with interpretable coefficients and calibrated probabilities. It’s robust, fast, and a strong baseline in many domains.

Category: Usage

How are coefficients interpreted in Logistic Regression?

Coefficients represent changes in log-odds per unit change in features; exponentiated, they indicate odds ratios.

Category: Concept

How is Logistic Regression trained?

By maximizing the Bernoulli likelihood (or minimizing log loss) via algorithms like gradient descent or Newton-like methods.

Category: Implementation

What are common pitfalls in Logistic Regression?

Perfect separation causing non-convergence, multicollinearity inflating variance, and class imbalance affecting thresholds and calibration.

Category: Mistakes

How do regularization terms affect Logistic Regression?

L1 promotes sparsity and feature selection; L2 stabilizes estimates with multicollinearity. Elastic net blends both.

Category: Practice

How do I handle class imbalance in Logistic Regression?

Use class weights, resampling, adjusted thresholds, and metrics like AUROC or PR AUC.

Category: BestPractices

What’s the difference between probit and Logistic Regression?

Both model binary outcomes; probit uses a normal CDF link, logistic uses a logistic CDF, yielding similar results with different interpretability.

Category: Comparison

Is Logistic Regression linear?

It’s linear in the log-odds, not in predicted probabilities, and belongs to generalized linear models.

Category: Clarification

How do I evaluate a Logistic Regression model?

Use log loss, AUROC, PR AUC, calibration curves, and confusion matrices at chosen thresholds.

Category: Validation