To enhance the effectiveness of fake audio detection techniques, researchers have developed multiple datasets such as those for the ASVspoof and ADD challenges. These datasets typically focus on capturing non-emotional characteristics in speech, such as the identity of the speaker and the authenticity of the content. However, they often overlook changes in the emotional state of the audio, which is another crucial dimension affecting the authenticity of speech. Therefore, this study reports our progress in developing such an emotion fake audio detection dataset involving changing emotion state of the origin audio named EmoFake. The audio samples in EmoFake are generated using open-source emotional voice conversion models, intended to simulate potential emotional tampering scenarios in real-world settings. We conducted a series of benchmark experiments on this dataset, and the results show that even advanced fake audio detection models trained on the ASVspoof 2019 LA dataset and the ADD 2022 track 3.2 dataset face challenges with EmoFake. The EmoFake is publicly available ( https://zenodo.org/records/10443769 ) now.