Let \(\mathbb {F}_q[t]\) be the polynomial ring over the finite field \(\mathbb {F}_q\) of q elements. For a natural number \(N\ge 1,\) let \(\mathbb {G}_N\) be the subset of \(\mathbb {F}_q[t]\) containing all polynomials of degree less than N. Let \(h\in \mathbb {F}_q[t][x]\) be a polynomial of degree \(2\le k<p,\) the characteristic of \(\mathbb {F}_q.\) Suppose that for every \(d\in \mathbb {F}_q[t]\setminus \{0\},\) there exists \(m\in \mathbb {F}_q[t]\) such that \(d\mid h(m)\) and \((d,m)=1.\) Let \(A\subseteq \mathbb {G}_N\) with \(|A|=\delta q^N.\) Suppose further that \((A-A)\cap \left( h(\Omega )\setminus \{0\}\right) =\emptyset ,\) where \(A-A\) is the difference set of A and \(\Omega \) denotes the set of all monic irreducible polynomials in \(\mathbb {F}_q[t]\) . It is proved that \(\delta \ll N^{-\mu }\) for any \(0<\mu <1/(2k-2),\) where the implied constant depends only on \(q,\ h\) and \(\mu .\)