Under complex operating conditions, rolling bearings have few fault samples, inconspicuous fault characteristics and are severely disturbed by noise, leading to difficulties in effective diagnosis and prediction of rolling bearings. Based on the above problems, a digital twin-based fault diagnosis model for rolling bearings with few samples across working conditions is proposed in this paper. First, the virtual vibration signal is obtained by constructing a virtual twin of the rolling bearing to provide a data source for fault diagnosis of rolling bearings. Second, the time-frequency map can be obtained by feature extraction utilizing continuous wavelet transform on the virtual vibration signal, and it can characterize the fault characteristics of rolling bearings more clearly. Third, the high-fidelity fitting of the time-frequency map of the virtual data using the cyclic consistent adversarial network (CycleGAN) improves the high fidelity of the virtual data of the digital twin. Finally, a DJP-DANN is proposed for enhancing inter-domain transferability and discriminability. It utilizes the discriminative joint probability maximum mean difference (DJP-MMD) as the loss function of the domain adversarial neural network (DANN) and uses the fault time-frequency map as the input for migration learning of rolling bearing less sample faults. In the end, six migration learning experiments were conducted in each of the two testbed datasets, and the experimental results show that the proposed fault diagnosis model has good high fidelity as well as effectiveness.