This paper proposes a novel adaptive parameter estimation (APE) with directional forgetting (DF) based on the parameter estimation error for nonlinear systems. Most of APE methods are designed to forget the information of the regressor matrix in all direction in order to update the system information. However, some directions of the regressor matrix for systems are insufficient such that forgetting in all directions may trigger windup phenomenon. To prevent this phenomenon, the proposed method in this paper is to divide the regressor matrix into a forgotten part and a retained part via oblique projection. Then, the constant forgetting factor is applied to the forgotten part such that forgetting the regressor matrix is along the direction that the new information comes. Furthermore, the adaptive law is designed based on the parameter estimation error and the exponential convergence of the proposed method can be guaranteed under the persistent excitation (PE) condition. Finally, comparative simulation related to the proposed method and the classic APE method driven by the parameter estimation error is provided. Simulation results show that the proposed APE with DF algorithm has a better estimation performance than the classic APE method.