This paper presents a novel approach to Artificial Intelligence (AI) education for non- Science Technology Engineering and Mathematics (STEM) students using an adapted Turing test as a pedagogical tool. As AI increasingly permeates disciplines beyond computer science, students in humanities and social sciences face barriers to effective AI literacy due to the predominantly technical focus of existing AI education. While many non-STEM students use AI tools such as ChatGPT, they often treat these systems as black boxes, limiting their ability to engage critically with AI technology. Drawing on AI literacy frameworks that emphasize critical assessment capabilities, we have developed a web-based platform specifically designed for non-STEM contexts that addresses the risks of uncritical AI dependence. Our platform adapts the traditional Turing test concept by challenging students to guide an AI system to produce content indistinguishable from human writing. Students are presented with human-authored reference texts and must use prompt engineering to help an AI generate comparable content. The AI is deliberately programmed to introduce factual errors and unnatural phrasing, requiring students to identify these issues through critical evaluation and develop prompts that lead to accurate, natural-sounding outputs. We implemented this approach in two educational settings: a graded assignment in a first-year undergraduate AI literacy course with 40 non-STEM students, and as an ungraded activity in a Digital Literacy orientation lecture. Our experience suggests that, while students generally detected obvious factual errors, many struggled with identifying technical inaccuracies. Unfamiliarity with the platform emerged as a significant barrier to engagement in the ungraded setting. In summary, we implement a pedagogical tool that centers AI critical evaluation and refinement in engagement with AI, in an effort to nudge students towards maintaining intellectual autonomy when engaging with increasingly sophisticated AI systems.