This comprehensive review delves into the extensive application of multi-agent systems (MAS) in power systems. It provides an in-depth exploration of the fundamental concepts of MAS and their relevance in addressing the dynamic challenges within the power system landscape, including decentralization, integration of renewable energy sources, and demand response. The analysis encompasses an examination of various MAS platforms and tools tailored for power system applications, offering valuable insights into their capabilities and potential impact. Furthermore, the review presents a detailed survey of successful MAS implementations in power systems, spanning decentralized control, self-healing techniques, demand response, and market-based systems. Additionally, it discusses the future research directions and challenges in the field of MAS for power systems, serving as a valuable resource for researchers, practitioners, and students seeking to understand and leverage the potential of MAS in the power system domain.