pruning

How to calculate Gain Ratio

As we discussed in one of our article about How and when does the Decision tree stop splitting? Gain Ratio is modification of information gain that reduces its bias. Gain ratio overcomes the problem with information gain by taking into account the number of branches that would result before making the split.It corrects information gainContinue reading “How to calculate Gain Ratio”

How and when does the Decision tree stop splitting?

By default Splitting will stop when the tree reaches 100% purity, means when the child/subset node has homogeneous/single class or we can also say when child/subset node is pure(means all classes will be either Yes or No), this will lead to overfitting problem. In simple when my algorithm learned everything from my training data, It willContinue reading “How and when does the Decision tree stop splitting?”

Practically, post-pruning overfit trees is more successful because it is not easy to precisely estimate when to stop growing the tree.In this article we will see how to overcome overfitting problem using Post pruning. We need to follow below steps in order to prune a full grown tree Here we willContinue reading “Pruning”