[Context and motivation] Empirical research on software explainability is challenging, as users’ needs for explanations are inherently subjective. In requirements engineering, these needs are often studied through hypothetical scenarios that assume users require certain explanations. [Question/problem] However, such scenarios rely on tacit knowledge and risk introducing hypothetical bias. This reduces the methodological robustness of the research and threatens the validity of its findings. [Principal ideas/results] To address this issue, we designed and conducted an experiment that induces genuine explanation needs by deliberately creating flawed mental models through erroneous tutorial material. Participant behavior during the experiment indicates that this approach successfully triggered authentic needs for explanations. [Contribution] Our methodology contributes to requirements engineering for explainable systems, by providing a way to systematically observe and examine users’ explanation needs under realistic conditions.