What is the difference between L1 and L2 regularization?

Both add a penalty on model weights to the loss to reduce overfitting, but they differ in the penalty term and effect. L1 (Lasso) adds the sum of absolute values of weights (lambda * sum|w|). Its gradient is constant, which drives some weights exactly to zero, producing sparse models and performing implicit feature selection. L2 (Ridge) adds the sum of squared weights (lambda * sum w^2), shrinking weights smoothly toward zero but rarely to exactly zero, which handles correlated features more gracefully. L1 is preferred when you expect few relevant features or want an interpretable, sparse model; L2 when many features contribute a little. Elastic Net combines both. The hyperparameter lambda controls regularization strength: larger lambda means stronger penalty and simpler models.