Large Language Models (LLMs) have been integrated into various domains, including inclusive education and healthcare, demonstrating the ability to generate human-like text with advanced learning capabilities. This paper evaluates four open-source LLMs specifically tuned for the sports domain, assessing their effectiveness and potential application in supporting athletes in the areas of nutrition, training, injury recovery and mental health. The evaluation of the models includes 16 benchmarks created using trusted sports-related sources for the four domains, comparing the models’ response with real ones. The models were evaluated both with and without fine-tuning, and both by human evaluation and automated scoring. The results show that models with small parameters tend to hallucinate and struggle to answer health related questions. The models with larger parameters show some challenges, such as occasional hallucinations and difficulty with complex queries, but their performance was satisfactory as they provided relevant and coherent responses in most scenarios. Gemma2 and Phi4 improve their performance when subjected to fine-tuning, in contrast to Llama3.2 and Deepseek_r1. The open source LLMs show promise as a support guide in sports domain. However, the effectiveness of tuning varies between models, suggesting that tuning needs to be tailored to each architecture.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fine-Tuned Large Language Models for Inclusive Education Athlete Support: Performance Evaluation and Benchmarking

  • Ana Costa,
  • Renata Magalhães,
  • Paulo Novais,
  • Dalila Durães

摘要

Large Language Models (LLMs) have been integrated into various domains, including inclusive education and healthcare, demonstrating the ability to generate human-like text with advanced learning capabilities. This paper evaluates four open-source LLMs specifically tuned for the sports domain, assessing their effectiveness and potential application in supporting athletes in the areas of nutrition, training, injury recovery and mental health. The evaluation of the models includes 16 benchmarks created using trusted sports-related sources for the four domains, comparing the models’ response with real ones. The models were evaluated both with and without fine-tuning, and both by human evaluation and automated scoring. The results show that models with small parameters tend to hallucinate and struggle to answer health related questions. The models with larger parameters show some challenges, such as occasional hallucinations and difficulty with complex queries, but their performance was satisfactory as they provided relevant and coherent responses in most scenarios. Gemma2 and Phi4 improve their performance when subjected to fine-tuning, in contrast to Llama3.2 and Deepseek_r1. The open source LLMs show promise as a support guide in sports domain. However, the effectiveness of tuning varies between models, suggesting that tuning needs to be tailored to each architecture.