Algorithms for biodistance analysis based on various squared Euclidean and generalized Mahalanobis distances combined with probabilistic hierarchical cluster analysis and multidimensional scaling

Biodistance analysis identifies groups that exhibit biological affinity based on phenotypic data. This study proposes and evaluates the performance of algorithms for biodistance analysis based on various squared Euclidean and generalized Mahalanobis distances by combining them with probabilistic hierarchical cluster analysis (HCA) and multidimensional scaling (MDS). Four archaeological datasets of human dental metrics and/or non-metric traits were used. To analyze the data, we integrated our previous work on biodistances and developed algorithms that calculate various types of squared Euclidean and generalized Mahalanobis distances, estimate various parameters, apply modified MDS and HCA methods to compute all possible cluster probabilities, and provide MDS confidence ellipses and dendrograms with cluster probabilities. All algorithms are implemented in R. From the data analysis, we found that all distances studied are simulated very satisfactorily by the Monte-Carlo method, resulting in the estimation of accurate cluster probabilities. Examining the probabilities of expected cluster formation, we found that these probabilities are higher when calculated using generalized Mahalanobis distances than the corresponding Euclidean distances. Therefore, the cluster probabilities supported that the generalized Mahalanobis distances are better than the corresponding Euclidean distances in cluster analysis. From a methodological point of view, clustering information concerning population affinities should not be based on a single dendrogram but instead be extracted from the list of the most frequent clusters obtained from all simulated dendrograms.