Exploring Conceptual Differences Among Nonparametric Estimators of Treatment Heterogeneity in the Context of Clustered Data

One aim of educational research is to evaluate interventions developed to improve student learning and behavioral outcomes. Estimating an intervention’s treatment effect is one way to evaluate its efficacy. This treatment effect is not always constant, and heterogeneity arises when not all students respond to interventions similarly, particularly between subgroups with varying characteristics. Sample differences in sociodemographic features or individual covariates can identify key subgroups, which can be used to estimate heterogeneity via the conditional average treatment effect (CATE) for individuals with those characteristics. Nonparametric methods are an increasingly popular choice for estimating CATE because of their flexibility to model complex relationships between many covariates and treatment status. Clustered data is a common occurrence in educational research, but many nonparametric methods do not explicitly account for clustered data structure. To better understand the role of clustered data structure in estimating heterogeneous treatment effects with nonparametric methods, we conduct a simulation study that compares the performance of different popular nonparametric methods as measured by their recovery of individual treatment effects under a potential outcomes framework. We examine methods’ performance across varying levels of intraclass correlation (ICC) and number of clusters sampled. Finally, we discuss the practice of accounting for clustered data structure, how conceptual differences between methods might correspond to differences in performance, and pose questions for future research.