In recent years, Graph Neural Networks (GNNs) have emerged as powerful tools for analyzing graph-structured data, demonstrating remarkable success in various applications. However, most existing GNN models inherently assume that the input graph data is clean and accurate, which often does not hold in real-world scenarios due to the presence of noise and adversarial attack. These imperfections can significantly degrade the quality of the graph, leading to erroneous predictions and reduced classification accuracy. To address these challenges, it is crucial to develop robust algorithms capable of mitigating adversarial perturbations over graph structure. In this work, we propose a novel robust graph structure learning framework called Robust Adaptive Graph Neural Network (RA-GNN), which is designed to enhance the robustness of GNNs against structural perturbations. Specifically, we introduce a novel loss function considering the dynamic structural perturbations, and a learnable combined model consisting of a multi-layer perceptron and a GNN. RA-GNN adaptively recovers the unperturbed graph structure from noisy graphs with various perturbation rate during training. Extensive experiments demonstrate that our method achieves superior robustness across varying levels of noise intensity, effectively eliminating perturbations, and significantly improving node classification performance.