Scholarlink

<p>This study investigates the performance and fairness of predictive models designed to predict data science programming performance using fine-grained log data from a sample of students solving Data Science tasks in <i>DaTu</i>. Our analysis reveals that Logistic Regression outperformed other models in terms of accuracy. Through Variable Importance analysis, we found that the 'paste answer' variable emerged as a critical predictor for Logistic Regression and KNeighbors classifier models, whereas multiple variables contributed equally to the GaussianNB model's predictive power. We evaluated fairness, focusing on the variable 'chatbot access' (with or without access&#xa0;to chatbot), finding evidence of bias. The models demonstrated lower score distributions for the unprivileged group (no-AI access).&#xa0;To mitigate this bias, we employed various techniques:&#xa0;Resampling, Reweighting, and ROC pivot. Of these, only Reweighting significantly improved fairness metrics. As an alternative approach, we considered the ceteris paribus cutoff method to minimize parity loss. In conclusion, this study emphasizes the significance of evaluating fairness in&#xa0;educational predictive modeling. This work offers a nuanced and data-driven approach to assessing students' data science programming skills.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Predicting Data Science Programming Performance: Using Fair ML to detect Bias

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月20日

查看全文
  • Tenzin Doleck,
  • Pedram Agand,
  • Dylan Pirrotta

摘要

This study investigates the performance and fairness of predictive models designed to predict data science programming performance using fine-grained log data from a sample of students solving Data Science tasks in DaTu. Our analysis reveals that Logistic Regression outperformed other models in terms of accuracy. Through Variable Importance analysis, we found that the 'paste answer' variable emerged as a critical predictor for Logistic Regression and KNeighbors classifier models, whereas multiple variables contributed equally to the GaussianNB model's predictive power. We evaluated fairness, focusing on the variable 'chatbot access' (with or without access to chatbot), finding evidence of bias. The models demonstrated lower score distributions for the unprivileged group (no-AI access). To mitigate this bias, we employed various techniques: Resampling, Reweighting, and ROC pivot. Of these, only Reweighting significantly improved fairness metrics. As an alternative approach, we considered the ceteris paribus cutoff method to minimize parity loss. In conclusion, this study emphasizes the significance of evaluating fairness in educational predictive modeling. This work offers a nuanced and data-driven approach to assessing students' data science programming skills.