In this article, we study the Yang–Mills connection A on a principal G-bundle P over a compact Kähler surface (X, g) with a \(\textit{generic}\) Kähler metric g. We first construct a gluing theorem of the approximate Hermitian–Yang–Mills connections. As an application, we prove that if the \(L^{2}\) -norm of the self-dual part of the curvature of a Yang–Mills connection is small enough, then the curvature is harmonic and trace free. For \(G=SU(2)\) or SO(3), we have a key observation that is the self-dual part of a harmonic and trace free curvature on a Kähler surface with \(H^{1}(X,\mathbb {Z}_{2})=0\) vanishes. Therefore, we obtain an \(L^{2}\) -energy gap result for Yang–Mills connection on principal SU(2) or SO(3)-bundle over a compact Kähler surface with \(H^{1}(X,\mathbb {Z}_{2})=0\) which admits a \(\textit{generic}\) Kähler metric.