Multi-hop reasoning enables models to answer complex queries by inferring answers through multiple reasoning steps across entities and relationships in a knowledge graph. Early methods based on reinforcement learning are favored for their ability to obtain complete reasoning paths. However, these methods face several challenges, including their inability to handle queries with missing edges, slow convergence, and low efficiency in model training. To address the aforementioned issues, we propose Alternating Entity and Relation Generation (AESG), a novel multi-hop reasoning framework. This framework combines a sequence generation module and an entity prediction module to iteratively generate reasoning paths. In the sequence generation module, we decompose complex relations into a combination of simpler, consecutive relations, thus reducing reasoning complexity and improving training efficiency. The entity prediction module leverages knowledge graph embedding techniques to facilitate the prediction of intermediate and tail entities, effectively addressing the issue of missing edges. Experimental results demonstrate that AESG outperforms existing methods across all four benchmark datasets, showing robust performance even in sparse graphs.