Interpretable Ensemble Model For Associative Classification

Associative classifiers have shown competitive performance with state-of-the-art classification methods for predicting class labels. In addition to their accuracy performance, associative classifiers produce human-readable rules for classification, providing an easier way to understand their decision-making process. However, the presence of high-dimensional feature vectors in the training data impacts these models’ performances. In this study, we propose an ensemble model, Dynamic Ensemble Associative Learning (DEAL), where we use associative classifiers as base learners on feature sub-spaces. We also propose a dynamic feature sampling procedure that automatically defines the number of base learners and ensures diversity and completeness among the selected features. As each of the base learners of our ensemble model is interpretable, we can collect the rules from the base learners and examine them to understand the decision process. Although, understanding this decision process can be complicated if there is a large number of base learners. To that end, we introduce another ensemble model, Classification by Frequent Association Rules (CFAR), where we make the decision process of the model easier to understand. In CFAR, instead of using a voting method for the final decision of the ensemble, we rank the generated rules based on their prevalence and select a subset of the predominant rules for predicting class labels. We use ten datasets from the UCI repository to evaluate the performance of our proposed models. Both our ensemble approaches, DEAL and CFAR, eliminate the high memory and runtime requirements of recent associative classifiers, especially for training datasets with high-dimensional feature vectors, all without jeopardizing classification accuracy. CFAR enhances the model’s interpretability by removing noisy rules while achieving increased classification accuracy in most cases in our experiments. While the decision process of DEAL is human interpretable, CFAR enhances this further by collecting and ranking the rules generated by its base learners to predict the final class label.