Differentiable Forests: The Random Journey Continues

The “black box" nature of contemporary Machine Learning (ML) models corrodes consumer confidence. Since users want to know that they can trust automated computational models, the community continues to improve model performance by decreasing the propensity for erroneous projection; efforts which ideally culminate in the improvement of Human-Computer Interactions. This study narrowed its focus to two variants of Random Forest (RF) models, the relatively recent Random Hinge Forest (RHFo) and Random Hinge Fern (RHFe), both conceptualized by Nathan Lay et al. in 2018. Although several years have passed since the development of RHFos and RHFes, Tree-Based Ensemble Model (TBEM) practitioners have made significant progress. Their models have been used for various user-centric applications, such as but not limited to credit card fraud analysis [10] and early diabetes detection [1]. Our work pit RHFos and RHFes against their contemporaries, Traditional Random Forests, Adaboost, and XGBoost, on the Human Activity Recognition dataset [7], to discover if RHFos and/or RHFes possess any desirable qualities which would make them comparatively relevant. We varied hyperparameters (number of trees, tree depth) and obtained 3-dimensional topologies of accuracy, allowing us to directly compare model performance. While RHFo and RHFe were outperformed by XGBoost (XGB), they performed similarly to Traditional Random Forest (TRF) and far outperformed Adaboost (ADA). RHFos and RHFes pose unique challenges but the benefits they exhibited make them serious contenders in this competitive landscape.