<p>Digital survey tools have all but replaced paper and pen in the psychological sciences, and consequently new forms of potentially useful research paradata are now routinely gathered. A particularly common byproduct of research is questionnaire timestamps, which some have suggested can be used as a measure of cognitive function. Here, we conducted a comprehensive validation of this measure, which we call the “digital questionnaire response time,” or “DQRT.” Using data from <i>N</i> = 2,977 users of a smartphone app, we first ran a data-driven bootstrapping approach to examine how best to quantify DQRT. DQRT was slower in older adults (<i>r</i> = 0.26) and in those with lower educational attainment and socioeconomic status. Testing the association between DQRT and working memory (range <i>r</i> = 0.11–0.14), model-based planning (range <i>r</i> = 0.03–0.06), and processing speed (range <i>r</i> = 0.29–0.39) across cross-sectional and longitudinal subsamples, we found support for a cognitive characterization of DQRT as a measure of cognitive processing speed. DQRT was more strongly correlated with nine out of 13 lifestyle and health factors, and four out of nine mental health factors than a task-based measure of processing speed. DQRT showed good test–retest reliability, and associations between DQRT and task-based processing speed were higher within individuals (<i>r</i> = 0.35) than between individuals (<i>r</i> = 0.25). Finally, we highlight substantial, but addressable, potential confounds inherent in the measure. We conclude that DQRT has important limitations, but overall can serve as a valid and reliable index of cognitive processing speed that can be gathered at unprecedented scale, unobtrusively, and repeatedly, during a variety of real-world digital behaviors.</p>