Various types of rocks are distributed all over the Earth. Within the last few decades, geologists and civil and mining engineers have tried classifying rocks into three groups: igneous, metamorphic, and sedimentary. They based their classification on particular characteristics exhibited by the rocks. However, classification processes done by humans are tedious and prone to errors because of endless factors. Thus, the deployment of deep learning methods aids in the optimization of the classification of rocks, which also cuts down the time spent on such tasks and improves accuracy. This study employed two deep learning models, CNN and MobileNetV2, to classify images of rock samples into the three types. Each of the three classes of rocks, namely igneous (Andesite and Rhyolite), metamorphic (Gneiss, Marble, Quartzite, and Schist), and sedimentary (Coal, Limestone, and Sandstone), included many rock images, totaling 30,000 in total. These models performed the classification based on features such as color, edges, texture, and contrast. The conventional CNN architecture consisted of three convolutional layers and a dense layer for classification at the end. The model was performant, achieving a satisfying accuracy of 92.98% and a loss of 0.3129. Conversely, the transfer learning model used, MobilNetV2, and produced similar results with an accuracy of 91.94% and a loss of 0.1934. Both models were effective with the dataset, and one possible reason for the transfer learning model’s lower accuracy compared to the CNN model is the transfer learning model’s lower complexity. It is reasonable to suggest that transfer learning models of greater complexity, for instance, ResNet50 and VGG16, can significantly reduce loss and improve accuracy, thereby enhancing the classification process through the application of technology.