Dynamic Graph Representation Learning methods have achieved significant success. However, real-world data often do not strictly follow the smoothness assumption in time-dependent relationships between dynamic graph snapshots. Existing methods sometimes fail to capture these relationships effectively. To address this issue, we propose the Community Enhanced Transformer for Dynamic Network Embedding (CeDFormer), which leverages transformers to capture temporal structures in dynamic graphs. CeDFormer is designed to adapt to highly variable graph structures and complex temporal dependencies. To mitigate the high computational cost and limited scalability of transformers on large-scale graph data, we introduce an optimization strategy involving parameter sharing within stable communities from a global perspective. This strategy enhances training speed by \(30\% \sim 35\%\) without compromising model performance. Extensive experiments on real-world datasets demonstrate that CeDFormer outperforms most other methods on the majority of datasets. Code is available at https://github.com/gjqwanttogjq/CeDFormer .