In this paper, a nonlinear generalized system of variable order (VO) of fractional differential equations (FDEs) based on the \( ^R\mathbb {D}^{\beta _i(x)}\) Riemann–Liouville’s operators such that \(0<\beta _1<\beta _2<\ldots <\beta _n=1\) is constructed. The system is considered within the two phases, the numerical and qualitative analysis. In qualitative analysis, we study the solution existence, stability works and uniqueness of solution. Once it is guaranteed that the solution possesses these properties then a computational scheme for the novel system of FDEs with boundary conditions will be developed. In the computational analysis, we will approximate the nonlinear terms of the model by using Lagrange’s interpolation techniques. Different variable orders will be tested on a dynamical model to examine the behavior of the model’s dynamics. We also design a control scheme using adaptive finite-time fractional-order sliding mode control (AFtFSMC) for the fractional-order dynamical leukemia model. The objective is to effectively eliminate leukemic cells (L-cells) while ensuring the preservation of a sufficient population of normal cells. This is achieved through the use of a chemotherapeutic substance that has been found to be safe. Moreover, an adaptive scheme is used to compensate for the unknown disturbance. The analysis of the Lyapunov theorem has been utilized for the overall system. The comparative simulations are included in order to provide a more comprehensive depiction of the study and to demonstrate the improved tracking control and convergence capabilities of the suggested approach.