The meteoric growth of voice cloning technology has led to increased availability of high realistic audio deepfakes posing severe trust, privacy, and security issues. State-of-the-art detection models perform well on benchmark datasets yet in cross-dataset, adversarial, and in noisy settings are found to fail often. To overcome these shortcomings, we introduce the Ensemble Audio Deepfake Network (EADNet), which is a resilient framework, that takes advantage of Convolutional Neural Networks (CNNs) to extract spectrogram representations, Recurrent Neural Networks (RNNs) to model temporal features, and Generative Adversarial Network (GAN)-based discriminator to learn to identify synthetic generation artifacts. All of these components work in parallel, and their predictors are combined in an equal-weighting 33:33:33 decision strategy, which results in well-balanced contributions by spatial, temporal, and adversarial feature streams. This architecture is also backed by studies conducted through ablation in weights different weighting schemes. EADNet has been tested on datasets ASVspoof 2019-LA, FakeAVCeleb, SONAR. The model had better accuracy and generalization than baseline CNNs, RNNs, and GAN-based-only models, and also was robust to adversarial perturbations (FGSM, PGD) and noisy views. We also include SHapley Additive exPlanations (SHAP) as an interpretability measure following the ranking of the most important spectral-temporal regions that affect the classification. Simply, EADNet provides a better-performing, explainable, and scalable approach to detecting audio deepfakes, which further contributes to the practical robustness and forensic interpretability of the state-of-the-art ensemble methods.