Automatic food product grading based on deep learning (DL) is widely adopted in a variety of food and packing industries. Vegetable grading is one of the common practices considered to separate the vegetables based on its quality. This research aims to develop a DL-approach to grade the tomato into accept/reject class based on its condition for automatic packing process. The stages of the proposed DL-approach consist the following; (i) collect the tomato images and resize it based on the need, (ii) deep features extraction using ResNet model, (iii) feature reduction with 50% dropout and features fusion to get a new feature vector, and (iv) binary classification with threefold cross validation. The merit of the proposed tool is tested using the ResNet-variants and during this task, the SoftMax classifier is considered to achieve the classification task. The performance of the proposed tomato grading system is verified using conventional and fused-features and the proposed tool provided a detection accuracy of > 97% when the fused-features based classification is executed. This confirms the merit of the developed DL-approach.