Trust in Algorithmic Advice Increases with Task Complexity

The use of algorithms in decision-making has increased in various fields, such as medicine, government, and business. Despite their proven accuracy, people often disregard algorithmic advice. When it comes to complex tasks, however, there is some evidence that people are more inclined to follow the advice of algorithms. This evidence is largely based on decision-making in rather artificial contexts, however, and studies in this field tend to rely on rather crude measures of complexity. We therefore investigate the effect of task complexity on trust in model-based advice in a realistic setting, measuring complexity in several standardized ways. We conducted an experiment with 151 participants, each assessing 20 real-life court cases of crimes (in the Dutch legal system). Participants were first asked to estimate the jailtime for each crime. They then received algorithmic advice, and were allowed to adjust their initial estimate. We measured task complexity in several ways. First, by simply counting the number of violations per case. Second, we focused on all the mitigating and aggravating circumstances associated with a specific case. Finally, we narrowed our focus to only the circumstances mentioned in the case text and concentrated on the violated sections of the law. We then used multi-level regression analysis (assessments within participants) on the target variable Weight on Advice (WOA) to assess the impact of complexity on trust in algorithmic advice. Our findings indicate that participants were more inclined to trust algorithmic advice as the complexity of tasks increased, for two of the three operationalizations of task complexity.