What is dropout and how does it prevent overfitting?

Dropout is a regularization technique that randomly deactivates a fraction of neurons (a typical rate is 0.2 to 0.5) on each training step, so their outputs are zeroed for that forward and backward pass. This prevents neurons from co-adapting and relying on specific others, forcing the network to learn redundant, robust features. It can be viewed as implicitly training and averaging an ensemble of many thinned sub-networks that share weights. Dropout is applied only during training; at inference all neurons are active, and outputs are scaled to account for the units kept during training (inverted dropout scales during training instead so inference needs no change). It is most useful in large fully connected layers prone to overfitting, and is used less in convolutional layers, which have fewer parameters and often rely on batch normalization instead.