Vanishing gradients can make deep neural networks stop learning. Understand why sigmoid causes gradients to shrink during backpropagation, how saturation makes the problem worse, and why modern networks prefer ReLU, GELU, and similar activations.

Why Sigmoid Causes the Vanishing Gradient Problem