In this paper, based on the H2N2 method (a method for approximating the Caputo fractional derivative of order \(\alpha \in (1\text {, } 2)\) derived by the quadratic Hermite and Newton interpolation polynomials), a direct finite difference scheme with second-order accuracy in space and \((3-\alpha )\) th order accuracy in time is constructed for the fractional sine-Gordon equation. The stability and convergence of the difference scheme are analyzed theoretically. Further, to improve the computational efficiency, a fast algorithm based on the sum-of-exponentials method is adopted to construct a fast difference scheme. In addition, the initial singularity of the solution is also discussed. Numerical outcomes verify the validity of the direct and fast difference schemes and the correctness of theoretical results.