The complex latent features in bearing fault signals can be automatically extracted and recognized through deep learning. However, numerous interference factors in real-world operating environments, such as different types and strengths of noise, variations in operating conditions, and instability in the quality of training data, can significantly affect fault diagnosis accuracy. To solve this problem, this study proposes a bearing fault diagnosis method that offers enhanced robustness under low SNR in varying working conditions: Firstly, neural networks are used to adaptively embed signals under different working conditions. Secondly, a novel fault diagnosis model, SFFormer, is proposed based on the Transformer architecture, incorporating LSTM units and residual structures to improve the model's ability to capture the temporal relationships of features in sequential data. Finally, the rationality of the model design is verified through ablation experiments, which also enhance the interpretability of the model. This research, based on various operational data from the CWRU public dataset, designs 14 tasks with varying conditions and adds Gaussian white noise, pink noise, and Laplace noise with fluctuations ranging from 10 dB to −5 dB. Experimental results show that the proposed model achieves an accuracy of 98.36% in variable operating condition tasks. The model exhibits strong robustness in all experiments, providing a new solution for the fault diagnosis field.