Rapid outburst of Novel Corona Virus Disease 2019 (COVID–19) epidemic turned out to demand additional screening methodologies to serve the rising need. For diagnosing the COVID-19 infections from the imaging modalities, deep neural nets take a leading role. The current deep learning models, in spite of the exceptional performance in terms of accuracy, it lacks explainability on the decision made and any misdiagnosis or false positive cannot be tolerated. A trustworthy deep learning system for COVID-19 diagnosis is desired during this current pandemic. The work implemented in this chapter aims at providing trustworthy diagnostic model for COVID-19 by providing explanations behind the network’s decision. This is implemented in two modules: classification and network visualization. This chapter explores the functionality of AlexNet, SqueezeNet, and ShuffleNet in classifying chest X-ray COVID-19 dataset. Experimental results demonstrate the proficiency of ShuffleNet in the categorization of this infectious virus dataset with an enhanced accuracy of 95.42%. Following the classification, network visualization is realized using three approaches: gradient-weighted class activation mapping (Grad-CAM), occlusion sensitivity mapping and local interpretable model-agnostic explanations (LIME). The deep learning framework suggested in this chapter for COVID-19 diagnosis, in addition to offering reasoning behind the decision also brings in the clinicians into the loop in subjective assessment of the network’s prediction thereby improving the trustworthiness of the diagnostic system.