The proliferation of online scams has become a pressing concern in the digital age, exacerbated by the rise of Artificial Intelligence. Malicious actors now employ sophisticated techniques to create convincing fraudulent schemes, targeting vulnerable individuals through personalized approaches on social media. This paper addresses the challenges of scam website detection by leveraging the capabilities of Large Language Models (LLMs). While other papers have focused on fine-tuning LLMs, our research investigates if readily available LLMs can be directly applied to scam website detection. This paper explores text-based and screenshot-based methods, utilizing five prominent LLMs to analyze website content. The findings indicate that existing LLMs are effective in identifying scam websites and providing rapid, expert responses for assessing website legitimacy. A novel categorization of criteria is proposed based on the LLMs’ decision-making processes. By comparing these models’ performances, this paper aims to develop a more efficient and accessible solution for identifying fraudulent websites. This work contributes to enhancing cybersecurity measures, potentially reducing online scams and increasing user trust in digital interactions.