Machine Learning (ML) models are used in a wide range of applications, which affects societies either directly or indirectly in daily life. Ensuring fairness in the decisions of these applications is a challenging task that has attracted the attention of researchers from different fields. However, most of the approaches consider a single sensitive attribute when measuring the fairness of a dataset or the outcomes of an ML model, which could be misleading when there are multiple sensitive attributes. In this paper, we study the problem of unfair decisions of the ML models for data with multiple sensitive attributes that are used to identify different demographic subgroups. Our study shows that the under-representation of certain demographic subgroups in the population is one of the most important reasons behind the biased predictions of the ML models. As a result, we propose a framework called ‘SynthFair’ to ensure fairness among the subgroups without changing the original class labels or removing the sensitive attributes. SynthFair uses synthetic data generation to train the classifiers on balanced datasets with similar number of records per subgroup. Experimental results over widely used benchmarks show that our framework yields consistent improvements compared to a set of bias mitigation methods.