Linear and nonlinear analysis of the viscous Rayleigh–Taylor system with Navier-slip boundary conditions

In this paper, we are interested in the linear and the nonlinear Rayleigh–Taylor instability for the gravity-driven incompressible Navier–Stokes equations with Navier-slip boundary conditions around a smooth increasing density profile \(\rho _0(x_2)\) ${\rho }_0(x_2)$ in a slab domain \(2\pi L\mathbb T\times (-1,1)\) $2\pi L\mathbb{T}\times (-1,1)$ ( \(L>0\) $L>0$ , \(\mathbb T\) $\mathbb{T}$ is the usual 1D torus). The linear instability study of the viscous Rayleigh–Taylor model amounts to the study of the following ordinary differential equation on the finite interval \((-1,1)\) $(-1,1)$ , 0.1 \(\begin{aligned} -\lambda ^2 [ \rho _0 k^2 \phi - (\rho _0 \phi ')'] = \lambda \mu (\phi ^{(4)} - 2k^2 \phi '' + k^4 \phi ) - gk^2 \rho _0'\phi , \end{aligned}\) $$\begin{array}{c}\hfill -{\lambda }^2[{\rho }_0{k}^2\varphi -{({\rho }_0{\varphi }^{\prime })}^{\prime }]=\lambda \mu ({\varphi }^{(4)}-2k^2{\varphi }^{\prime \prime }+k^4\varphi )-g{k}^2{\rho }_0^{\prime }\varphi ,\end{array}$$ with the boundary conditions 0.2 \(\begin{aligned} {\left\{ \begin{array}{ll} \phi (-1)=\phi (1)=0,\\ \mu \phi ''(1) = \xi _+ \phi '(1), \\ \mu \phi ''(-1) =- \xi _- \phi '(-1), \end{array}\right. } \end{aligned}\) $$\begin{array}{c}\hfill \left\{\begin{array}{c}\varphi (-1)=\varphi (1)=0,\hfill \\ \mu {\varphi }^{\prime \prime }(1)={\xi }_{+}{\varphi }^{\prime }(1),\hfill \\ \mu {\varphi }^{\prime \prime }(-1)=-{\xi }_{-}{\varphi }^{\prime }(-1),\hfill \end{array}\right)\end{array}$$ where \(\lambda >0\) $\lambda >0$ is the growth rate in time, \(g>0\) $g>0$ is the gravity constant, k is the wave number and two Navier-slip coefficients \(\xi _{\pm }\) ${\xi }_{\pm }$ are nonnegative constants. For each \(k\in L^{-1}\mathbb Z{\setminus }\{0\}\) $k\in L^{-1}\mathbb{Z}\backslash \{0\}$ , we define a k-supercritical regime of viscosity coefficient, i.e. \(\mu >\mu _c(k,\Xi )\) $\mu >{\mu }_c(k,\mathrm{\Xi })$ with \(\Xi =(\xi _+,\xi _-)\) $\mathrm{\Xi }=({\xi }_{+},{\xi }_{-})$ to describe a spectral analysis by adapting an operator method of Lafitte and Nguyễn (Water Waves 4:259–305, 2022) and to prove that there are infinite nontrivial solutions \((\lambda _n, \phi _n)_{n\ge 1} \) ${({\lambda }_n,{\varphi }_n)}_{n\ge 1}$ of (0.1)–(0.2) with \(\lambda _n \rightarrow 0\) ${\lambda }_n\to 0$ as \(n\rightarrow \infty \) $n\to \infty$ and \(\phi _n\in H^4((-1,1))\) ${\varphi }_n\in H^4((-1,1))$ . Hence, we prove the linear Rayleigh–Taylor instability for any viscosity coefficient \(\mu >0\) $\mu >0$ . As a by-product, based on the existence of infinitely many normal modes of the linearized problem, we construct a wide class of initial data to the nonlinear equations, being inspired by the previous framework of Guo–Strauss (Commun Pure Appl Math 48:861–894, 1995) and of Grenier (Commun Pure Appl Math 53:1067–1091, 2000) with a refinement, to prove the nonlinear Rayleigh–Taylor instability in a high regime of viscosity coefficient, namely \(\mu >3\sup _{k\in L^{-1}\mathbb Z{\setminus }\{0\}}\mu _c(k,\Xi )\) $\mu >3{sup}_{k\in L^{-1}\mathbb{Z}\backslash \{0\}}{\mu }_c(k,\mathrm{\Xi })$ .