This study uses machine learning techniques to classify the surface roughness using the laser speckle images of the machined samples, an intriguing yet relatively less explored field of research in the realm of speckle metrology. The laser speckle imaging technique is sensitive to surface roughness paving the way for the classification of the machined specimen based on surface roughness using the distinct speckle pattern. The paper presents the analysis of the performance of the state-of-the-art machine learning techniques on the preliminary dataset of the speckle pattern of the machined sample. The gray level co-occurrence matrix is used for feature extraction. The model performance with various combinations of features is studied to distinguish the most descriptive feature for generalization. The assessment of the classifiers’ performance aids in the generalization of the classification and prediction of the roughness classes in the range \(R_a =0.1\ \mu m-1.6\ \mu m\) using the speckle images.