Evidence Accumulation as a Combining Rule for Decision Forests
摘要
Much is still unknown about how to best combine probabilistic predictions in classifier ensembles. In this paper, we propose a combining rule called Evidence Accumulation (EVA) and study it in the context of ensembles of decision trees. We identify a key factor that determines the performance of EVA relative to the commonly used averaging: namely, the extent to which the combined probabilities represent epistemic, rather than aleatoric, uncertainty. This insight leads to the expectation that EVA should work better than averaging for shallow or highly randomized trees. It further leads to a view of evidence-accumulating forests as a bridge between two very different types of classifiers, namely, Naive Bayes and decision forests.