Epistemic stability and nonlinear dynamics in selection of suboptimal cluster counts in medical images validation dataset as a cluster homogeneity measure

This paper presents a compact method for selecting the suboptimal number of clusters in unsupervised clustering, which is comparable in complexity to the well-known elbow method, silhouette score and gap statistic, but has been upgraded for clustering with the dominant overlaps. It is primarily based on heuristics and combines human reasoning with the deterministic qualitative analysis of clustering as a mapping from Euclidean space to the unstructured space of cognitive decision making and labeling. Epistemic stability as a notation involves a methodological approach that takes into account the available knowledge in the observed data set and avoids biases and effects on the objectivity of clustering. The stability discussion is also integrated into the assigned and naturally quantitative analysis of the dataset, not to regulate the clustering, but to learn from the pattern of instability. The method is applied to the specific task of classifying medical images from pre-selected image sets based on the associated metadata to determine the homogeneity of the analyzed clusters. However, it is also applicable to any other example of datasets that exhibit statistically multivariate problems with dominant overlaps. In the mathematical discussion of the method presented, the constrained conditions for datasets are established by definitions and assumptions, followed by the central mathematical propositions of the method of epistemic stability.