In this chapter, based on the basic formulation of conjugate FORM-based FR, several improving first order reliability methods formulated by conjugate search direction vectors are established including (i) limited FR, (ii) Enhanced FR-based variant type in gradient vector, (iii) Chaotic conjugate chaos control method, (iv) chaotic conjugate stability transformation method. The step size is adaptively adjusted based on the sufficient decent condition and the Armijo-type condition in these formulations. These formulations are compared with step size and without step size for several engineering problem.