This study explores the integration of Large Language Models (LLMs) in an experimental real-world scenario laboratory setting for third-semester undergraduate students in process engineering. Unlike conventional laboratory instruction, the students were required to act autonomously without cookbook-style instructions. This research investigates five key questions about: (1) the students’ engagement with LLMs in an unfamiliar environment, (2) the issues they used the LLM for, (3) the domains of AI literacy they applied LLMs to, (4) their perceived competence gains regarding LLMs, and (5) how they regarded the experiment compared to usual experiments. The results indicate that approximately 40% of the students utilised an LLM, while 36% did not, with some perceiving its use as a “joke.” Most students interacted only somewhat with the LLM, while only a few used it extensively or effectively. Many recognised its potential yet struggled with appropriate use due to a lack of guidance and cognitive overload. The students tended to apply LLMs to issues where the model demonstrated only surface-level understanding according to the SOLO taxonomy. They seldom used it for tasks within the more metacognitive domains of AI literacy and prompting strategies were consistently suboptimal. Finally, the students reported only moderate perceived competence gains regarding LLM interaction in the open problem. While those who actively engaged perceived increased understanding of appropriate applications and limitations, all of the participants benefited from the discussions. Using LLMs also increased the perceived competence gains of this experiment compared to former runs. This study highlights the necessity of providing clear use cases and instructional guidance to enhance self-efficacy in initial AI interactions and indicates that the SOLO taxonomy is suitable for rating the quality of the answers given by the LLMs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Autonomous Usage of LLMs in Scenario-Based Laboratory Learning in Engineering Education

  • Konrad E. R. Boettcher

摘要

This study explores the integration of Large Language Models (LLMs) in an experimental real-world scenario laboratory setting for third-semester undergraduate students in process engineering. Unlike conventional laboratory instruction, the students were required to act autonomously without cookbook-style instructions. This research investigates five key questions about: (1) the students’ engagement with LLMs in an unfamiliar environment, (2) the issues they used the LLM for, (3) the domains of AI literacy they applied LLMs to, (4) their perceived competence gains regarding LLMs, and (5) how they regarded the experiment compared to usual experiments. The results indicate that approximately 40% of the students utilised an LLM, while 36% did not, with some perceiving its use as a “joke.” Most students interacted only somewhat with the LLM, while only a few used it extensively or effectively. Many recognised its potential yet struggled with appropriate use due to a lack of guidance and cognitive overload. The students tended to apply LLMs to issues where the model demonstrated only surface-level understanding according to the SOLO taxonomy. They seldom used it for tasks within the more metacognitive domains of AI literacy and prompting strategies were consistently suboptimal. Finally, the students reported only moderate perceived competence gains regarding LLM interaction in the open problem. While those who actively engaged perceived increased understanding of appropriate applications and limitations, all of the participants benefited from the discussions. Using LLMs also increased the perceived competence gains of this experiment compared to former runs. This study highlights the necessity of providing clear use cases and instructional guidance to enhance self-efficacy in initial AI interactions and indicates that the SOLO taxonomy is suitable for rating the quality of the answers given by the LLMs.