Analysis of Optimal Control of Fractional-Order Predator-Prey System Based on Invasion of Alien Species

This paper examines a fractional-order prey–predator system in the context of alien species invasion, where the food chain consists of prey, an intermediate predator, and an invasive top predator. The model incorporates fear effects, prey refuge, and the Sokol–Howell functional response to capture ecological realism. We first prove the existence, uniqueness, and boundedness of solutions. Then, we conduct a local and global stability analysis of the system’s equilibrium points, deriving sufficient conditions for stability. We then apply Pontryagin’s maximum principle to derive the optimal control of the system and discuss the existence of the corresponding optimal solution. Finally, we conduct numerical simulations under different fractional orders to analyze the dynamic responses of all species, comparing cases with and without optimal control. The outcomes show that optimal control can efficiently stabilize population dynamics, and that the fractional order considerably affects the convergence rate to reach the equilibrium. These findings provide theoretical insights and methodological tools for maintaining ecosystem stability, protecting biodiversity, and mitigating the ecological risks posed by invasive species.