<p>Detecting careless respondents in self-report data is crucial for ensuring survey validity, yet current methods often fall short in identifying complex careless response patterns. This study evaluates an approach that uses simulated data with deep neural networks (DNNs) and support vector machines (SVMs) to detect careless response patterns at the respondent level. Five careless response pattern types were analyzed: straight-lining, diagonal bouncing, midpoint responding, extreme alternating, and random responding. Simulation results indicate that both DNN and SVM models achieved high accuracy. Extreme alternating was the easiest to detect, whereas random responding was the most difficult. Overall, SVM models with radial basis function kernels performed best. The proposed approach was applied to empirical data collected under attentive and experimentally induced careless responding conditions. Results indicate significantly higher occurrences of diagonal bouncing, midpoint responding, and random responding among inattentive respondents, while differences for straight-lining and extreme alternating were not statistically significant.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detecting careless response patterns in Likert scales using supervised machine learning and Monte Carlo simulations

  • Artur Pokropek

摘要

Detecting careless respondents in self-report data is crucial for ensuring survey validity, yet current methods often fall short in identifying complex careless response patterns. This study evaluates an approach that uses simulated data with deep neural networks (DNNs) and support vector machines (SVMs) to detect careless response patterns at the respondent level. Five careless response pattern types were analyzed: straight-lining, diagonal bouncing, midpoint responding, extreme alternating, and random responding. Simulation results indicate that both DNN and SVM models achieved high accuracy. Extreme alternating was the easiest to detect, whereas random responding was the most difficult. Overall, SVM models with radial basis function kernels performed best. The proposed approach was applied to empirical data collected under attentive and experimentally induced careless responding conditions. Results indicate significantly higher occurrences of diagonal bouncing, midpoint responding, and random responding among inattentive respondents, while differences for straight-lining and extreme alternating were not statistically significant.