Traditional reinforcement learning algorithms often face problems such as difficult convergence in task scenarios with large decision spaces and sparse rewards. The hierarchical reinforcement learning mechanism based on task decomposition, which achieves spatial dimensionality reduction by constructing a hierarchical structure, effectively alleviates the problem of reward sparsity. In some complex game confrontation fields such as Go and electronic games, a lot of expert experience data has been accumulated. How to effectively integrate expert experience and reinforcement learning mechanisms to further improve the level of intelligent decision-making models still needs to be studied and promoted. Therefore, this article mainly proposes a hybrid learning architecture that introduces an imitation mechanism, which decomposes complex tasks into multiple subtask modules through a hierarchical structure, and based on domain expert data-driven approaches, reconstructs and optimizes subtask modules through empirical imitation, providing new ideas for improving the efficiency of decision model generation and task completion level in related research.