Phishing attacks are one of the most challenging threats in the cyberspace. Recently, rapid advancements of (generative) AI and its wide applicability has been exploited by attackers to perform phishing attacks. However, limited studies exist regarding AI-based phishing and apt defence strategies. In this work, we explore a wide variety of AI-leveraging techniques, adopted by attackers to conduct successful phishing campaigns. Additionally, we highlight the negative impact of AI-driven phishing in real-world and the attendant challenges that it has on cybersecurity. We also examine various factors and features of AI-powered phishing which makes these difficult to identify and complicated to defend. Consequently, we evaluate the crucial aspects of phishing attacks and discuss its defence strategies, human-centred preventive measures, and ethical considerations for enhancing security against AI-based phishing threats. Our findings provide valuable insights to the evolving cybersecurity threats and effective approaches to defend against these sophisticated AI-driven attacks.