DataTerminology.comNEW
A nonlinear activation function mapping inputs to (−1, 1), often used to center activations and stabilize training.
DESCRIPTION
Hyperbolic tangent (tanh) is a smooth, bounded activation function with outputs between −1 and 1. Centered activations can speed convergence compared to sigmoid’s 0–1 range. However, tanh saturates for large magnitudes, contributing to vanishing gradients in deep networks. It remains useful in RNNs and shallow models where bounded outputs and symmetry help. Modern architectures often prefer ReLU variants for gradient flow and sparsity. Tanh appears in mathematical analyses and frameworks such as PyTorch and TensorFlow.
ALTERNATIVE DEFINITIONS
TensorFlow
Computes hyperbolic tangent of x element-wise.
Concise computational definition in framework API.
PyTorch
Returns a new tensor with the hyperbolic tangent of the elements of input.
Framework-centric definition highlighting tensor operation.
Wolfram MathWorld
The hyperbolic tangent function is defined as tanh x = sinh x/cosh x.
Provides mathematical definition and relation to hyperbolic functions.
RELATED TERMS
Activation Function Saturation Vanishing Gradient Softsign Feature Scaling Gradient Descent
COMPETING TERMS
KNOWLEDGE AREAS
TARGET AUDIENCE
AI Engineer Data Scientist Software Developer
EXTERNAL SOURCES
- Wikipedia - Hyperbolic function#Hyperbolic tangent
- PyTorch - API reference for tanh
- TensorFlow - API docs for tanh
- NumPy - NumPy tanh function definition
TOP QUESTIONS
What is Hyperbolic Tangent in neural networks?
Tanh is an activation function mapping inputs to (−1, 1), helpful for centered outputs and sometimes faster convergence than sigmoid.
Category: Overview
Why choose tanh over sigmoid?
Centered outputs around zero reduce bias shifts, often aiding optimization compared to sigmoid’s strictly positive outputs.
Category: Opinion
What are tanh’s drawbacks?
It saturates for large inputs, causing vanishing gradients in deep networks and slowing learning.
Category: Mistakes
Where is tanh commonly used?
In RNNs and certain shallow networks, and in tasks requiring bounded symmetric outputs.
Category: UseCases
How does tanh compare to ReLU?
ReLU avoids saturation and supports sparse activations, often improving deep network training over tanh.
Category: Difference
Is tanh differentiable everywhere?
Yes, tanh is smooth and differentiable, providing well-defined gradients across its domain.
Category: Requirements
How can vanishing gradients with tanh be mitigated?
Use residual connections, careful initialization, normalization, or switch to ReLU variants.
Category: Practices
Does tanh help with normalization?
Its bounded range constrains activation values, but normalization layers still improve stability and convergence.
Category: Opinion
Is tanh suitable for output layers?
Yes, when outputs must be bounded and symmetric; otherwise choose task-appropriate activations like sigmoid or softmax.
Category: Implementation