The higher-order nonlinearity of Boolean functions is an important parameter in designing robust stream cipher and block cipher based cryptosystems. In terms of coding theory the maximum mth-order ( \(m \ge 1\) is a positive integer) nonlinearity of a Boolean function equals the covering radius of the mth-order Reed-Muller code. It is a challenging task to compute the mth-order nonlinearity of a given Boolean function, particularly when \(m>1\) . This paper is concerned with the computation of lower bounds of the third-order nonlinearity of biquadratic monomial Boolean functions of the form \(g_{\mu }(x) = Tr_1^n( \mu x^{2^{p}+2^{q}+2^{r}+1}) \) , where \(n>p>q>r \ge 1\) with n, p, q, r being positive integers. A general lower bound on the third-order nonlinearity for the functions of this form have earlier been obtained by Singh (Int J Math:1–7, [25]) for \(n > 2p\) . In this paper, we have obtained bounds for the cases when \(4 \le n \le 2p\) and also in certain instances improved the known bounds for \(n>2p\) . We determine some values of p, q and r, for which \(g_{\mu }\) has better lower bound on the third-order nonlinearity. Further we tighten lower bounds of the third-order nonlinearity of some subclasses of \(g_{\mu }\) . Our results are helpful in selecting the values of p, q and r for which \(g_{\mu }\) has high third-order nonlinearity. We determine that \(Tr_1^8(\mu x^{85})\) has the highest known value of third-order nonlinearity among all 8 variable Boolean functions.
