Given the inherent rarity observed in imbalanced datasets, adopting one-class classification (OCC) stands out as a pragmatic approach to counteract bias toward the predominant class. This research endeavors to thoroughly assess ten state-of-the-art OCC methodologies across a spectrum of five diverse challenges within the Banking, Insurance, and Cybersecurity sectors. Moreover, we introduce an innovative unsupervised learning technique wherein (i) K-Means clustering is utilized during the training phase on negative sample data. To determine the optimal number of clusters, we utilized the Silhouette index and employed the maximum intra-class centroid distance as a cluster-specific threshold. These thresholds play a pivotal role in distinguishing between positive and negative samples. (ii) In the testing phase, a majority voting mechanism is employed to evaluate the discriminative capability of these thresholds, facilitating precise classification of test data. Empirical findings unequivocally demonstrate that our proposed approach outperforms several state-of-the-art techniques, achieving superior classification accuracy across four out of the five datasets. This underscores the effectiveness and potential applicability of our novel methodology in tackling the intricate challenges prevalent in the Banking, Insurance, and Cybersecurity sectors, particularly in the domains of fraud detection and related areas.