Let Ω be a bounded smooth domain in ℝN (N ≥ 3). Assuming that 0 < s < 1, \(1 < p,q \le {{N + 2s} \over {N - 2s}}\) with \((p,q) \ne ({{N + 2s} \over {N - 2s}},{{N + 2s} \over {N - 2s}})\) , and a, b > 0 are constants, we consider the existence results for positive solutions of a class of fractional elliptic system below,
\(\left\{{\matrix{{(a + b[u]_s^2){{(- \Delta)}^s}u = {v^p} + {h_1}(x,u,v,\nabla u,\nabla v),} \hfill & {x \in \Omega,} \hfill \cr {{{(- \Delta)}^s}v = {u^q} + {h_2}(x,u,v,\nabla u,\nabla v),} \hfill & {x \in \Omega,} \hfill \cr {u,v > 0,} \hfill & {x \in \Omega,} \hfill \cr {u = v = 0,} \hfill & {x \in {\mathbb{R}^N}\backslash \Omega.} \hfill \cr}}\right.\)
Under some assumptions of hi(x, u, v, ∇u, ∇v)(i = 1, 2), we get a priori bounds of the positive solutions to the problem (1.1) by the blow-up methods and rescaling argument. Based on these estimates and degree theory, we establish the existence of positive solutions to problem (1.1).