Ensemble

I want to buy a home, so I approached one consultancy and they asked couple of questions and suggested the place and the builder. Do you think taking suggestions from one consultancy is a good idea? Absolutely NO.. rather I will take opinions from different consultancy and will chose the best fit for me based on the most common recommendations, because each consultancy will have a different experience, Lets say I approached 5 consultancy, Two suggested in some XX place XX builder and three suggested YY place and YY builder, as majority of the consultancy suggested YY place and YY builder I will go for this option.

When I want to correlate the above example with our scenarios, the first option approaching to only one consultancy is a Decision tree and the second option where we approached multiple consultancy we call it Ensemble.

Instead of creating one Decision tree we will create multiple decision trees and take the best of it.This is one of the most used methods for classification problems tough we can also use it for regression but in real time in my experience I see most of the implementations are for the classification problemas.

We have three types of Ensembling model.

1. Bagging or bootstrapping aggregation
2. Random Forest
3. Boosted Decision trees

We will discuss about Random Forest and Boosted decision trees in our upcoming articles.But want to discuss one major difference between Random Forest/Bagging vs Boosted trees are Random/Bagging will take the average or maximum outcomes as the final output where boosted decision tree will try to make the week learner/misclassified to strong learner/ correct classified.