Background <p>The “up-or-out” system is designed to determine faculty retention based on research performance; however, its potential impact on the mental health of early-career faculty remains unclear.</p> Objective <p>To evaluate the impact of the “up-or-out” system on the mental health of early-career faculty and to explore the underlying mechanisms.</p> Methods <p>A three-year longitudinal study was conducted among faculty members from 39 universities who had received job offers but had not yet signed employment contracts. Generalized estimating equation models, mediation analysis, subgroup analysis, sensitivity analysis, and attrition analysis were used to examine the effects of research performance indicators (frequency of project applications, funding level, and number of published papers) on the primary outcome (PHQ-9) and secondary outcomes (GAD-7 and PSS-10).</p> Results <p>A total of 2,632 early-career faculty were included in the analysis. The study found significant interaction effects of research performance on both primary and secondary outcomes (<i>P</i> &lt; 0.05). For the primary outcome, faculty who published an average of ≥ 3 papers per year had a significantly higher depression score (MD [95% CI] = 5.67 [4.14 to 5.98]) than those who published an average of ≤ 2 papers per year (MD [95% CI] = 4.41 [3.06 to 4.86]). Similarly, faculty with ≥ 3 grant applications showed a significantly higher level of depression (MD [95% CI] = 6.06 [5.46, 6.69]) than those with &lt; 3 applications (MD [95% CI] = 3.32 [2.95 to 4.02]). Compared with faculty who received projects lower level, those who obtained national-level funding exhibited a significant reduction in depression scores (MD [95% CI] = − 2.31 [–3.18 to − 1.44]), whereas provincial-level funding was associated with a non-significant change (<i>P</i> &gt; 0.05). Regarding secondary outcomes, stress and anxiety scores increased as the periodic evaluation approached (<i>P</i> &lt; 0.05). Obtaining project funding and publishing more papers exerted a buffering effect on stress (<i>P</i> &lt; 0.05), whereas a higher frequency of grant applications increased the risk of anxiety (<i>P</i> &lt; 0.05). Mediation analyses showed that the frequency of grant applications had a significant positive indirect effect on depression and anxiety (<i>P</i> &lt; 0.05), while project funding level (for stress and anxiety) and number of published papers (for stress) exhibited significant negative indirect effects (<i>P</i> &lt; 0.05). Gender differences showed a convergent trend over time (<i>P</i> &gt; 0.05). Sensitivity analyses supported the robustness of the main findings, and attrition analysis indicated no selective bias, and Harman’s single-factor test did not reveal serious common method bias.</p> Conclusion <p>The up-or-out system is associated with the high-output–high-depression paradox among early-career faculty. Therefore, optimizing the evaluation system could focus on balancing research performance and mental health, for example, by extending the evaluation cycle, increasing funding coverage, and establishing psychological support mechanisms.</p>