How does backpropagation work?

Backpropagation is the algorithm that computes the gradient of the loss with respect to every weight, enabling gradient descent. It has two phases. In the forward pass, inputs flow through the network to produce a prediction and a scalar loss, caching intermediate activations. In the backward pass, it applies the chain rule from calculus, starting at the loss and propagating gradients layer by layer back toward the inputs. At each layer it computes how much each weight contributed to the error by multiplying the upstream gradient by local derivatives. These per-weight gradients are then used by an optimizer to update the weights in the direction that reduces loss. Backprop is efficient because it reuses cached activations and computes all gradients in one backward sweep rather than perturbing each weight separately.