Predictions from deep learning models need to be perfect in radiology, where incorrect predictions challenge their trustworthiness. As models of different architectures enter the field of image analysis, it is important to uncover the source and quantify the extent of the uncertainty in the models’ predictions. In this study, uncertainty methods are applied to a downstream task in classifying maxillary sinus images using three trained models: convolutional neural network (CNN), vision transformer (ViT), and gated multilayer perceptron (gMLP). The uncertainty is explored through probability distribution, expected calibration error, calibration curves, predictive entropy, and second-order representation. Of the three models, the ViT was the most uncertain in its predictions. Images that affected model confidence were identified. Regardless of the model architecture, most ‘clear’ sinus images were identified with certainty, while considerable uncertainty in predicting ‘opaque’ and ‘thick’ radiographic appearances was noted. These findings represent a novel direction towards the explainability of deep learning models as uncertainty estimation is a viable method for scrutinizing their inner workings.