entropy – Data and Machine by viswateja

Deriving Decision Tree using Entropy (ID3 approach)

We are taking below famous data which is widely used data set for explaining Decision tree algorithm. Once we build a decision tree it looks like below. Now we will see how we achieved above decision tree using Entropy and Information gain matrices. First and foremost question is, how do I chose my root nodeContinue reading “Deriving Decision Tree using Entropy (ID3 approach)”

Measure of impurity

In a given dataset that contains class for the predicted/dependent variable (like Yes,No,Neutral etc..), we can measure homogeneity or heterogeneity of the table based on the classes. We say a dataset is pure or homogeneous if it contains only a single class(either YES or NO). If a dataset contains several classes, then we say that theContinue reading “Measure of impurity”