Scholarlink

<p>Careless responses (CR) in surveys undermine data quality, and their detection is essential for quality research. Existing CR detection methods include post-hoc and a priori approaches; however, the latter rely on assumptions and are content-irrelevant. To address these limitations, this study introduces time-management items, which capture participants’ typical daily time allocations within ecologically plausible routines. To provide evidence for these items in detecting CR, the study examines their construct, convergent, and concurrent validity by utilizing established post-hoc indicators, including long string, randomness, outliers, and person-total correlation. Additionally, the study discusses their limitations, applications, and variations to inform future research. To achieve these objectives, data were collected from 812 Chinese undergraduates. A method is proposed to flag CR for these items when participants select extreme sleep options or when their reported daily time deviates substantially from 24&#xa0;h. Principal component analysis revealed a single component for CR indicators in time-management items, reflecting multiple CR behaviors and supporting construct validity. The proposed method converged significantly with the observed and carelessness component. Regression analysis revealed that the method predicted log-transformed response time at a level comparable to established post-hoc methods, supporting concurrent validity. Findings support the validity of this content-integrated approach. Although cut-off values for the proposed method may vary by context, the approach has broader applications across disciplines beyond Chinese culture. Potential variations include time-distracting items and subject-specific learning time items. Future directions should compare the method with other a priori and post hoc methods and integrate this approach with other survey design features to improve the current detection and prevention practices.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Validating common-sense survey items as a priori indicators of careless responses: Evidence from construct, convergent, and concurrent validity

  • 期刊论文

  • 发表日期：

    2026年7月11日

查看全文
  • Yingchen Wang,
  • Yiyuan Gao,
  • Ziyu Fan

摘要

Careless responses (CR) in surveys undermine data quality, and their detection is essential for quality research. Existing CR detection methods include post-hoc and a priori approaches; however, the latter rely on assumptions and are content-irrelevant. To address these limitations, this study introduces time-management items, which capture participants’ typical daily time allocations within ecologically plausible routines. To provide evidence for these items in detecting CR, the study examines their construct, convergent, and concurrent validity by utilizing established post-hoc indicators, including long string, randomness, outliers, and person-total correlation. Additionally, the study discusses their limitations, applications, and variations to inform future research. To achieve these objectives, data were collected from 812 Chinese undergraduates. A method is proposed to flag CR for these items when participants select extreme sleep options or when their reported daily time deviates substantially from 24 h. Principal component analysis revealed a single component for CR indicators in time-management items, reflecting multiple CR behaviors and supporting construct validity. The proposed method converged significantly with the observed and carelessness component. Regression analysis revealed that the method predicted log-transformed response time at a level comparable to established post-hoc methods, supporting concurrent validity. Findings support the validity of this content-integrated approach. Although cut-off values for the proposed method may vary by context, the approach has broader applications across disciplines beyond Chinese culture. Potential variations include time-distracting items and subject-specific learning time items. Future directions should compare the method with other a priori and post hoc methods and integrate this approach with other survey design features to improve the current detection and prevention practices.