This paper evaluates the performance of phishing attacks generated manually versus those created using ChatGPT, Copilot, and Gemini. The study examines how non-experts can exploit chatbots to launch email-based phishing attacks, even without prior knowledge of cybersecurity or coding. By applying prompt engineering techniques, participants were able to bypass chatbot safeguards and generate malicious content. The resulting phishing emails were then evaluated and compared to determine which performed better. The research methodology consisted of two parts: (1) creating phishing emails with the assistance of chatbots or manually, and (2) conducting a survey to rate the effectiveness of each email. Findings show that ChatGPT, Copilot, and Gemini can be easily manipulated to generate phishing emails that incorporate convincing characteristics, often outperforming those written manually. These results highlight the urgent need to raise awareness among developers and researchers so that stronger security filters and safeguards are implemented to prevent such exploitations by cybercriminals.