What are ensemble methods and how do they improve model performance?

Ensemble methods combine multiple models to produce a single, stronger model. The idea is that by aggregating the predictions from a variety of models, you reduce the risk of overfitting and increase accuracy. Common ensemble techniques include bagging, such as Random Forest, which builds multiple trees from random subsets of data, and boosting, such as AdaBoost or XGBoost, which adjusts the weights of misclassified observations to focus on difficult cases. These methods leverage the diversity of models to create robust predictions.