Researchers increasingly believe that artificial intelligence (AI) can discover the underlying pathological processes of psychiatric disorders. They hope this will bolster a move toward precision psychiatry by providing personalized diagnoses, treatments, and predictions. However, AI research is predominantly committed to a one-sided, biologically oriented medical conception of disorders and neglects the non-epistemic values researchers necessarily rely on in their classification. I argue, instead, that psychiatric disorders are complex, socioculturally shaped, interactive, and that their classifications are value-laden. Unless these factors are addressed, AI applications in psychiatry may bring about unintended ethical and social implications due to the inherent nature of AI systems. Due to lack of agency, AI raises concerns over the therapeutic efficiency of the human-AI relationship, epistemic injustices, and responsibility gaps, which relate to the difficulty of allocating responsibility. Because of the AI systems’ opacity, they can contribute to looping effects, so that the automated diagnoses interact with the conditions they aim to identify. Automated looping effects can amplify errors, biases, or non-pathological patterns found in the training data, raising their own responsibility questions. Through datafication, they can also lead to the neglect and alteration of uncommon ways of experiencing and expressing disorders, thereby resulting in ontological thinning of mental disorders. To address these challenges, I argue that AI-driven precision psychiatry should shift its focus from simplified precision to socially just AI by examining the domain relativity of precision and adopting a value-sensitive design approach in clinical applications. This approach should involve all the affected stakeholders in investigating and evaluating the social consequences of AI-driven psychiatry.