Machine learning models and new techniques have been widely researched in credit scoring. For most credit-scoring datasets, data is unbalanced since the “bad” class is usually lower in proportion than the “good” class. Also, the rejection rate is high in some fields, leading to sample bias when training the scores. This paper presents a literature review to address these concerns, bringing the most known techniques to solve them. 490 articles were initially screened in Scopus and Web of Science, of which 88 were subject to content analysis. The results show that a significant number of algorithms have been tested in different datasets. For the class imbalance problem, SMOTE (synthetic minority oversampling technique) is the most used technique, but robust machine learning techniques have also been introduced. Finally, it was noticed that there is a noticeable opportunity for combining different techniques for imbalanced data that can be explored in future research works as a research gap.