Large Language Models (LLMs) have propelled the emergence of sophisticated Multi-Agent Systems (MAS) that leverage language-driven reasoning, collaboration, and autonomous decision-making. This paper presents a comprehensive review of state-of-the-art LLM-based frameworks for building MAS - including AutoGen, CrewAI, CAMEL, ChatDev, LangGraph, and Google DeepMind’s Agent Development Kit (ADK)- highlighting their architectural designs, agent coordination mechanisms, and operational strengths. We critically analyze fundamental challenges such as dynamic task decomposition, persistent multi-agent memory, communication efficiency, emergent collective behaviours, explainability, trust, and the lack of standardized evaluation benchmarks. Drawing from recent advances, we identify promising research frontiers encompassing meta-learning for adaptive task assignment, multimodal grounding to bridge language and perception, swarm-inspired emergent phenomena, and robust memory architectures supporting long-term agent continuity. By synthesizing these insights, this work offers a diagnostic and prescriptive roadmap to enhance the scalability, interpretability, and resilience of LLM-powered multi-agent systems, thereby accelerating the development of robust, collaborative AI agents for real-world deployment.