Sorting bananas based on their visual features helps meet consumer requirements in various markets and food processing industries. Many banana varieties look similar in terms of their size and shape, making differentiation challenging and leading to misclassification without detailed visual data. Our pro-posed system focusses on distinguishing these visually similar varieties, especially the closely sized Elakki-bale and Rasbale. We employ transfer learning techniques for banana variety classification and grading to achieve high accuracy. Our bunch-level banana image dataset addresses the gap in existing dataset that are smaller segments mainly hand-level or finger-level, serving the large-scale food processing industry needs. This study also comprises a system for grading bananas into low, medium, and high quality. In our proposed work, during classification, deep features were extracted and various classifiers were tested. The study revealed that the Linear SVM classifier works better for Integrated Model. For Convolutional Model MobileNet features lead in accuracy and most time-efficient in terms of training. EfficientNetB7 gives high accuracy but requiring the longest training duration. In this study, a hierarchical approach was employed for variety classification and grading. The study revealed that the accuracy of our proposed hierarchical classification approach using Convolutional Models is 96.85% which is higher than the conventional single shot multi-class classification approach (94.87%). Our proposed hierarchical approach, focused on bunch-level classification, is compared against state-of-the-art single shot multi-class classification, which primarily targets hand or finger-level classification.