In recent years, the energy-efficient hybrid flow shop scheduling problem (EHFSP) has gained significant attention in both academia and industry. However, most existing studies focus solely on either energy supply or demand strategies, which leads to suboptimal overall system efficiency and energy waste. To address this challenge, this paper proposes a novel multi-objective mixed-integer nonlinear model that integrates time-of-use pricing with machine power-down strategy, achieving a coordinated optimization of production efficiency and energy costs. To solve the proposed model, we develop a Q-learning-driven multi-crossover operator non-dominated sorting genetic algorithm II (QMCO-NSGA-II). By embedding Q-learning into the multi-crossover operator framework, the algorithm adaptively selects crossover operators, effectively balancing solution diversity and convergence. Additionally, an improved Nawaz-Enscore-Ham heuristic is introduced to generate high-quality initial solutions, enhancing both the convergence speed and solution quality. Extensive experiments on standard EHFSP datasets demonstrate that QMCO-NSGA-II significantly outperforms four state-of-the-art multi-objective algorithms in terms of convergence and solution diversity, showcasing its practical applicability to complex industrial scenarios.