What is Bagging in Ensemble Learning?

Data Science, AI and ML
#1

Bagging (Bootstrap Aggregation) is used to reduce the variance of a decision tree. Suppose a set D of d tuples, at each iteration i, a training set Di of d tuples is sampled with replacement from D (i.e., bootstrap). Then a classifier model Mi is learned for each training set D < i. Each classifier Mi returns its class prediction. The bagged classifier M* counts the votes and assigns the class with the most votes to X (unknown sample).

Implementation steps of Bagging –

  1. Multiple subsets are created from the original data set with equal tuples, selecting observations with replacement.
  2. A base model is created on each of these subsets.
  3. Each model is learned in parallel with each training set and independent of the other.
  4. The final predictions are determined by combining the predictions from all the models.