Let G be an arbitrary (not necessarily isomorphic to a closed subgroup of \(\textrm{GL}(r,\mathbb {C})\) ) complex Lie group, U a complex manifold and \(p:P\rightarrow U\) a \(\mathcal {C}^\infty \) principal G-bundle on U. We introduce and study the space \(\mathcal {J}^\kappa _P\) of bundle almost complex structures of Hölder class \(\mathcal {C}^\kappa \) on P. To any \(J\in \mathcal {J}^\kappa _P\) we associate an \(\textrm{Ad}(P)\) -valued form \(\mathfrak {f}_J\) of type (0,2) on U which should be interpreted as the obstruction to the integrability of J. For \(\kappa \ge 1\) we have \(\mathfrak {f}_J\in \mathcal {C}^{\kappa -1}(U,{\mathchoice{{\textstyle \bigwedge }}{{\bigwedge }}{{\textstyle \wedge }}{{\scriptstyle \wedge }}}\hspace{-2pt}^{0,2}_{\,\,U}\otimes \text {Ad}(P))\) whereas, for \(\kappa \in [0,1)\) , \(\mathfrak {f}_J\) is a form with distribution coefficients. Let \(J\in \mathcal {J}^\kappa _P\) with \(\kappa \in (0,+\infty ]{\setminus }\mathbb {N}\) . We prove that J admits locally J-pseudo-holomorphic sections of class \(\mathcal {C}^{\kappa +1}\) if and only if \(\mathfrak {f}_J=0\) . If this is the case, J defines a holomorphic reduction of the underlying \(\mathcal {C}^{\kappa +1}\) -bundle of P in the sense of the theory of principal bundles on complex manifolds. The proof is based on classical regularity results for the \(\bar{\partial }\) -Neumann operator on compact, strictly pseudo-convex complex manifolds with boundary. The result will be used in forthcoming articles dedicated to moduli spaces of holomorphic bundles (on a compact complex manifold X) framed along a real hypersurface \(S\subset X\) .