The rapid advancement of artificial intelligence (AI) has led to an increasing application of Large Language Models (LLMs) in psychological counseling. This study focuses on a comprehensive evaluation of LLMs in this domain, moving beyond traditional case-based reasoning. We introduce a novel multi-agent LLM framework that enhances the analysis of psychological case interactions. Our approach involves expanding the Emotional First Aid dataset with diverse client backgrounds, enhancing its applicability and generalizability. A sophisticated user profile model, incorporating eight critical dimensions, is developed and applied within a multi-agent system to examine counseling scenarios. The system’s performance is extensively evaluated based on accuracy, robustness, consistency, and fairness. The findings reveal significant differences among LLMs in these areas, highlighting their strengths and limitations in psychological interventions. This research underscores the need for ongoing refinement in LLM applications to ensure equitable and reliable support in psychological counseling. The detailed results and methodologies are available on the GitHub platform for further academic scrutiny and development.