Machine learning techniques have become ubiquitous and find application in various domains, including natural language processing and handwriting recognition. This paper centers around a fundamental aspect of digit handwriting analysis: feature extraction and clustering. The objective of this work is to explore diverse combinations of feature extraction and clustering methods to identify the optimal approach for addressing handwritten digit recognition using the MNIST dataset. Through rigorous analysis, mean shift clustering with the utilization of Histogram of Oriented Gradients (HOG) emerges as the most promising approach. This outcome not only contributes to the advancement of digit recognition techniques but also underscores the importance of appropriate method selection in enhancing accuracy and efficiency in various machine learning applications.