X-Vector-Based Speaker Diarization Using Bi-LSTM and Interim Voting-Driven Post-processing

In this work, we propose a voting-driven post-processing strategy for enhancing the efficacy of supervised speaker diarization models. Speaker embeddings, x-vectors, are used to train deep learning architectures such as convolutional neural networks (CNN), Bi-directional gated recurrent units (Bi-GRU) and Bi-directional long short-term memory (Bi-LSTM). The state-of-the-art unsupervised diarization is implemented using agglomerative hierarchical clustering (AHC) with cosine affinity measure and obtained a DER of 26.07%. Among the supervised frameworks, Bi-LSTM achieves the lowest diarization error rate (DER) of 18.42% on the CallHome dataset. To further enhance the performance of the supervised diarization models, we introduce an interim voting-driven post-processing strategy using dynamic time warping (DTW) and euclidean distance (ED) on the predicted speaker labels. This interim voting and centroid-distance metric framework assigns mispredicted speakers to the most probable speakers’ feature space leading to a notable reduction in DER. The experiments demonstrate that integrating the proposed approach with Bi-LSTM significantly reduces the DER to 10.26%, marking a relative improvement of 8.16% over the non-voting Bi-LSTM framework.