How do RNNs, LSTMs, and GRUs differ for sequence modeling?

RNNs process sequences step by step, maintaining a hidden state that carries information forward, so the output at each step depends on prior inputs. Plain (vanilla) RNNs struggle with long-range dependencies because repeated multiplication through time causes vanishing or exploding gradients. LSTMs fix this with a separate cell state and three gates (input, forget, output) that control what information is added, kept, or exposed. The near-additive cell-state update lets gradients flow across many time steps, capturing long-term dependencies. GRUs are a lighter alternative with two gates (reset and update) and no separate cell state; they have fewer parameters, often train faster, and perform comparably to LSTMs on many tasks. All three have largely been superseded by Transformers for long sequences, which use attention to model dependencies in parallel without recurrence.