Bias in Artificial Intelligence (AI)-driven mental health analysis refers to systematic errors that lead to unfair or inaccurate predictions due to imbalanced training data, model design, or inherent societal inequalities. Bias can arise from demographic under-representation, cultural nuances, or linguistic variations, leading to skewed diagnoses across different populations. Detecting bias is crucial to ensure equitable mental health assessments; undetected bias reinforces disparities, misdiagnoses individuals, or limits access to appropriate care. In this paper, we analyze the state-of-the-art models for detecting bias in mental health analysis solutions. We also design a taxonomy for the same to provide insight into the bias classifications. Ultimately, we provide a list of open research problems that enhance ongoing research in the direction of AI-based mental health analysis. The information and analysis provided in the paper are useful for clinical trials, patients and relatives, and psychiatric counsellors. It will further help the tool developer to mitigate the bias as necessary before analyzing the mental health parameters.