Intra-Gap Method: An Enhanced Elbow Method for Determining the Initial Cluster Numbers in Clustering Analysis

Unsupervised clustering is crucial in data analysis. However, determining an appropriate initial number of clusters (k) poses challenges for methods like k-means, which typically require this initial number to be predefined. The Elbow method has been widely used to identify the ideal number of clusters (k). Still, its effectiveness diminishes when the resulting plot lacks a clear elbow or knee point, thus complicating the selection of k. To overcome the limitation, this study introduces the Intra-Gap method to enhance the traditional Elbow approach. It is done by refining the gap analysis used in the Elbow method for application. Based on the analysis conducted on the three benchmark datasets (Iris, Breast Cancer, and Vehicles) for the data clustering analysis using the k-means algorithm, the study concludes that the Intra-Gap method offers a promising solution. The study found that the accuracy of the output matched the benchmark number of clusters, k. This method has proven effective and has made a significant contribution to the field of data clustering techniques.