Experimental Design Principles for Developing Machine Learning Models for Human–Robot Interaction

Real world human–robot teams will be deployed in dynamic, uncertain environments where effective collaboration hinges on the robot’s ability to comprehend its human teammate and adapt its behavior based upon their needs. Endowing robots with this collaborative capacity demands the development of machine learning models that learn the relationship between the human’s external (e.g., facial expression, physiological signals) and internal states (e.g., affect, workload). The difficulty lies in overreliance on training data collected in controlled experimental conditions, as designing experiments that precisely manipulate independent variables while capturing realistic human-robot teaming dynamics is challenging. This manuscript proposes a set of principles for designing human subject evaluations that emphasize the practical and ecological validity considerations necessary for constructing sufficiently diverse datasets. An example evaluation is also presented, demonstrating how these principles can be applied in practice.