<p>Self-excited vibrations in turbomachinery may lead to high-cycle fatigue and structural failure, highlighting the need for effective vibration mitigation. Friction dampers play a critical role to reduce vibration through passive energy dissipation. Among the various friction dampers, wedge dampers are widely used in turbomachinery.Wedge dampers may give rise to non-unique structural responses due to the nonlinear nature of frictional contacts. They cause multiple possible amplitudes for self-excited vibration, including limit cycle oscillations (LCOs) and stability limits (SLs). This uncertainty makes it crucial to determine the amplitude boundaries of LCOs and SLs during the design process. This study presents a novel method for determining the amplitude boundaries of LCO and SL in turbine blades with wedge dampers. The proposed method solves a constrained optimization problem, using the dissipative energy of the system as target function and system balance equations in the frequency domain as well as system’s energy balance as constraints. In this way, the admissible envelope, which defines the amplitude region associated with stable self-excited behavior, is determined. Results obtained from a lumped-mass model of two blades with a wedge damper show how vibration amplitudes in LCO and SL and system stability are affected by friction-induced non-uniqueness. Additionally, the effect of the wedge damper geometry is investigated to determine the effect of the wedge angle on the system’s amplitude. In particular, results show that neglecting the existence of multiple solutions can lead to overestimate the range of stability of the system in the design process.</p>