<p>In this paper, a critic-only self learning optimal control structure combined with the linear extended state observer is presented for trajectory tracking problem of the continuum robots (CRs) with unknown disturbances. The CRs demonstrate exceptional potential in space-constrained applications due to their inherent flexibility and compliant bending capabilities. The CRs operate with a desired trajectory that needs to be tracked, interacting with the environment and applying effects through multidimensional information such as forces and images. However, designing trajectory tracking controllers becomes notably challenging when factoring in the unknown lumped disturbances associated with both the environment and the robots themselves, including unknown external disturbances and unmodeled internal dynamics. To this end, a novel self learning method is developed based on disturbance rejection approach and optimal regulation law with adaptive dynamic programming. A tailored extended state observer is designed to estimate the implicit state space parameters and the unknown lumped disturbances of the CRs. Then, a conducted performance index function is proposed to integrate the tracking errors and inputs of the robots. A critic-only neural network (NN) is trained to approximate the solution of the derived Hamilton-Jacobi-Bellman (HJB) equation with simple structure and less computational cost. Particularly, an improved updating algorithm is inserted for ensuring the finite time convergence of the critic-only NN weights. The effectiveness and feasibility of proposed approach are demonstrated through stability analysis and numerical simulations, illustrating its potential in practical applications.</p>