Comparative Analysis of Decision Tree Algorithms Using Gini and Entropy Criteria on the Forest Covertypes Dataset

By applying split conditions at each node that divide the training records into subsets with output variables of the same class, decision trees are constructed recursively. The procedure commences at the root node of the decision tree and advances through the implementation of split conditions at every non-leaf node, ultimately producing subsets that are homogeneous. Nevertheless, the attainment of completely homogeneous subsets is unattainable. Consequently, the objective at every node is to determine an attribute accompanied by a split condition that reduces the commingling of class labels, thereby producing subsets that are nearly pure. This paper investigates decision tree models with the intention of determining which one is most suitable for the classification of forest cover types. Individual classifications using the Gini index and information gain were conducted on the covertype dataset, which categorizes pixels into seven distinct forest cover types according to characteristics including elevation, aspect, slope, hill shade, soil type, and more. Both classifiers were assessed in this study with respect to their accuracy. The findings of the research demonstrate that irrespective of the imbalance or balance of the dataset, the classification models indices that entropy was slightly more effective than Gini in this context.