Voting Ensembles

Voting ensembles are one of the simplest ensemble learning techniques.

A voting ensemble typically involves using a different algorithm to prepare each ensemble member, much like stacking. Instead of learning how to combine predictions, a simple statistic is used.

On regression problems, a voting ensemble may predict the mean or median of the predictions from ensemble members. For classification problems, the label with the most votes is predicted, called hard voting, or the label that received the largest sum probability is predicted, called soft voting.

The important difference from stacking is that there is no weighing of models based on their performance. All models are assumed to have the same skill level on average.

  • Member Assessment: Assume all models are equally skillful.
  • Combine with Model: Simple statistics.