The issue of financial fraud detection is an acutely burning problem of the modern digital banking system as the complexity of fraudulent activities and financial flows of greater magnitude increases. Traditional machine learning and rule-based systems tend to not be effective when dealing with the high-dimensional data, redundant features, and changing patterns of fraud, leading to the decline in the detection rates and false alarm rates. To address these limitations, this work presents AI-driven hybrid optimal deep reinforcement learning for effective workday financial fraud detection to minimizing computational cost and false positives in large-scale financial datasets. The methodology uses pre-trained Residual Network (ResNet) and Dense Convolutional Network (DenseNet) for deep feature extraction, followed by the Enhanced Red Piranha Optimization (ERPO) algorithm to select the most relevant features. These refined features are processed by a Deep Reinforcement Learning (DRL) model, which adaptively learns fraud patterns and optimizes detection strategies. Experiments are conducted on the UK financial fraud dataset, and the results demonstrate that the proposed model outperforms conventional fraud detection models. Experiments conducted on the UK financial fraud dataset demonstrate that the proposed DenseNet + ERPO + DRL model achieves an accuracy of 97.602%, precision of 87.125%, recall of 85.234%, F1-score of 86.172% and an AUC of 0.984, outperforming conventional models including Naïve Bayes, KNN, Random Forest, and CNN-based approaches. Comparative analysis shows improvements of 6.218% in accuracy, 14.668% in precision, and 19.352% in recall over the best-performing benchmark VP model, while maintaining a significant reduction in false positive rate.