mediumAI & Machine Learning FundamentalsReviewed Jul 24, 2026

What is the difference between bagging and boosting?

Both are ensemble methods that combine many weak learners, but they differ fundamentally. Bagging (bootstrap aggregating) trains models in parallel on random bootstrap samples of the data and averages or votes their predictions. Because the models are independent, this mainly reduces variance and helps with overfitting; Random Forest is the classic example. Boosting trains models sequentially, where each new learner focuses on the examples the previous ones got wrong, and combines them as a weighted sum. This mainly reduces bias and can turn weak learners into a strong one; examples include AdaBoost, Gradient Boosting, and XGBoost. Trade-off: bagging is more robust to noise and parallelizable, while boosting usually achieves higher accuracy but is more prone to overfitting noisy data and is harder to parallelize.

ensemblebaggingboostingrandom-forest

More AI & Machine Learning Fundamentals questions

See all AI & Machine Learning Fundamentals questions →