<p>This paper introduces an efficient first-order optimization framework designed to address the optimal control of nonlinear dynamical systems. These systems pose nontrivial computational and theoretical challenges due to their inherently nonlinear dynamics and the nonconvex constraints often imposed on both state and control variables. Although sequential convex programming (SCP) methods are widely applied to problems of this nature, their heavy computational demands and pronounced sensitivity to initial conditions can limit their practical utility. To circumvent these challenges, we propose a novel reformulation that recasts a nonlinear optimal control problem as a linear control problem with nonconvex constraints. Specifically, our method integrates a first-order optimization strategy that relies on projection steps onto nonconvex sets, thereby reducing the computational effort to that of solving a single convex optimization problem. Furthermore, the approach leverages GPU-based parallelization to enhance computational efficiency and scalability, enabling real-time or near-real-time performance in demanding applications. Numerical experiments on a spacecraft orbit transfer problem with complex constraints underscore the effectiveness and robustness of the proposed algorithm in resolving large-scale nonlinear optimal control problems under nonconvex constraints.</p>