Tracking ability bias in generative AI: a comparative analysis of ChatGPT 3.5, ChatGPT 4.0, Gemini 2.0, and Grok 3
摘要
To evaluate ability bias in AI-generated descriptions of individuals with and without disabilities across ChatGPT 3.5 (May 2023), ChatGPT 4.0 (February 2025), Gemini 2.0 (February 2025), and Grok 3 (February 2025), focusing on unprompted disability representation and sentiment-based linguistic patterns. Observational study using a mixed-methods framework to assess sentiment and representation in AI-generated text. Controlled environment with outputs from ChatGPT 4.0, Gemini 2.0, and Grok 3 generated in isolated browser sessions; ChatGPT 3.5 data were sourced from prior publications. A total of 450 AI-generated descriptions (150 per model) were analyzed across four prompt categories: a person without disability (baseline), a person with a disability, a patient with a disability, and an athlete with a disability. ChatGPT 3.5 data were drawn from previously published results. Each model was prompted to generate five-sentence descriptions across all four prompt types. Responses were analyzed for spontaneous mention of disability and the prevalence of favorable or limiting sentiment-based terms. (1) Rate of spontaneous disability representation in baseline prompts, and (2) proportion of favorable vs. limiting language using the Linguistic Sentiment Dictionary 2015 (LSD2015). Spontaneous mention of disability was 0% for ChatGPT 4.0, Gemini 2.0, and Grok 3—lower than the 5% and 11.7% previously reported for ChatGPT 3.5 and Gemini/Bard. All models showed increased limiting language when disability was mentioned. ChatGPT 4.0 and Grok 3 also showed decreased favorable language, while Gemini 2.0 improved modestly for athletes with disabilities. Although ChatGPT 4.0 and Gemini 2.0 demonstrated improvements in describing patients with disabilities, spontaneous disability representation has declined. Limiting language increased across all models, underscoring persistent ability bias. Ongoing monitoring and inclusive training data are essential to ensure equitable representation.