In the burgeoning field of fractional calculus, Neural Networks stand out as a particularly promising area. Fractional-based optimization algorithms have demonstrated remarkable effectiveness in the realm of Neural Networks. Nonetheless, the intricacies introduced by fractional derivatives are significant and pose challenges when applied to deep neural networks, additionally failing to guarantee convergence. This study introduces a simplified version of fractional gradient descent that ensures convergence. The mathematical convergence for the proposed method has been verified. Additionally, the effectiveness of the modified fractional gradient descent approach was assessed on a number of datasets and contrasted with other well-known methods. Comparing the suggested method to the other methods, the analysis showed that it obtained a faster rate of convergence and provide better results.