There is a consensus that Artificial Intelligence (AI) has transformed lives across various industries, elevating them to new heights. One notable application is the integration of AI to combat social engineering attacks, such as phishing attempts. This paper focuses on three key areas: the role of AI in mitigating phishing attacks, the limitations and threats associated with AI in this context, and recommendations for staying updated on the latest advancements in AI. We conducted a systematic review using the PRISMA framework and analysed 33 studies to provide valuable insights. The top three common keywords from those 33 studies are phishing (35 occurrences), learning (27 occurrences), and machine (19 occurrences). If learning and machine are combined, machine learning would be the top keyword. Our findings indicate that AI excels in detecting and preventing a range of phishing techniques (16 papers out of 33 papers), with some solutions operating in real-time, thus representing groundbreaking innovations in cybersecurity defence. We discovered that cybercriminals are leveraging AI, and without keeping pace with evolving phishing tactics and lacking expertise, access to high-quality data, resources, and effective optimisations poses significant challenges. This paper recommends establishing a secure cloud infrastructure, leveraging AI-powered security tools, adopting a zero-trust security model, improving security awareness, and staying updated on advancements in Generative AI and Large Language Models, such as Microsoft Security Copilot and Google Cloud Security. Its goal is to help individuals with a security mindset understand the intersection of AI and cybersecurity, especially in preventing phishing attacks.