In recent years, large-scale synthetic image datasets have proven to be a boon for training deep semantic segmentation models due to their easy scalability and cost-effective annotation processes. However, models trained on synthetic images often fail to generalize well when deployed in the real world. To solve this problem, various domain randomization (DR) techniques have been introduced to help generalize the models in real settings. One common aspect of such DR techniques is their usage of low-order statistics, particularly the mean and standard deviation for generating new styles during training. However, real images have more complex distributions than Gaussian and thus, high-order statistics also need to be considered for generating new styles. Towards this goal, this paper proposes Low order High order Style Consistency (LoHoSc), a new Domain Randomization framework consisting of two modules, LoSC and HoSC. During training, LoSC and HoSC generate random styles using low-order statistics (e.g., mean and standard deviation) and high-order statistics (e.g., empirical Cumulative Distribution Functions), respectively, in the feature space. The predictions corresponding to the two styles are then constrained in the loss space to learn content-relevant information while discarding any style variant information. Evaluation of LoHoSC on various benchmark datasets shows that it achieves state-of-the-art Domain Generalization capabilities, both quantitatively and qualitatively.