DataTerminology.comNEW
Also known as: Hessian Matrix
A square matrix of second-order partial derivatives describing local curvature of a scalar field.
DESCRIPTION
The Hessian matrix captures second-order variation of a scalar function with respect to its inputs. It is central to Newton and quasi-Newton optimization, convexity tests, and uncertainty analyses. Positive-definite Hessians indicate local minima; indefinite ones imply saddle points. In high dimensions, exact Hessians can be costly, so approximations or Hessian-vector products are used.
ALTERNATIVE DEFINITIONS
Wikipedia
In mathematics, the Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field.
Defines the object precisely and its scalar-field context.
Wolfram MathWorld
The Hessian matrix of a function is the square matrix of second partial derivatives.
Concise mathematical definition used in optimization contexts.
RELATED TERMS
Second-Order Optimization Newton’s Method Gradient Curvature Fisher Information Quasi-Newton Method Convexity Taylor Series
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
Data Scientist AI Engineer MLOps Engineer
EXTERNAL SOURCES
- Wikipedia - Hessian matrix
- Stanford SEE - Convex optimization notes on Hessians
- MIT OpenCourseWare - Multivariable calculus lecture on Hessians
TOP QUESTIONS
Why is the Hessian useful in optimization?
It informs curvature, enabling Newton’s method to take second-order steps that often converge faster than gradient methods.
Category: Relevancy
How do you test convexity with the Hessian?
A function is convex if its Hessian is positive semidefinite over the domain.
Category: Requirements
What’s a common computational challenge with Hessians?
Computing and storing full Hessians is expensive in high dimensions; use approximations or Hessian-vector products.
Category: Mistakes
How does the Hessian relate to Newton’s method?
Newton’s update uses the inverse Hessian (or approximation) to scale gradients for curvature-aware steps.
Category: Example
What’s the risk of an indefinite Hessian?
It indicates saddle points or non-convex regions, where Newton steps can diverge without safeguards.
Category: Risks
How do quasi-Newton methods handle the Hessian?
They build low-rank approximations of the inverse Hessian from gradient differences (e.g., BFGS).
Category: Example
Can automatic differentiation compute Hessians?
Yes, via second-order autodiff or repeated Jacobian-vector products, though costs can be substantial.
Category: Requirements
How does the Hessian relate to uncertainty estimation?
Near optima, the inverse Hessian approximates parameter covariance under quadratic assumptions.
Category: Example
What is a Hessian-vector product?
It efficiently computes H·v without forming H explicitly, useful for large-scale optimization.
Category: Topic