The spreading of fake news in online social networks has profound consequences for both society and individuals, highlighting the crucial need for fake news detection. Fake news often exploits emotional triggers to spread misinformation. However, existing methods primarily concentrate on the textual semantics of news content, with minimal consideration for subtle emotional nuances. Furthermore, aligning with the spreading character of news on social media, capturing propagation structures is also instrumental. Therefore, considering the semantics, emotions, and propagation patterns of news, we propose a method named Ega-DeFake that leverages the propagation of semantic graphs and emotion graphs for enhanced fake news detection. We utilize the pre-trained language model and four kinds of emotion signals to extract semantic features and emotion features for Ega-DeFake, respectively. Moreover, an adversarial perturbation is employed to endow short-text news (e.g., tweets) with more generalized features, which enhances the robustness of our model in real-world scenarios. We formulate the fake news detection task as a graph classification problem and compare our approach with eleven baseline algorithms. Our method Ega-DeFake maintains its superiority on all datasets.