In multi-modal multi-objective optimization (MMO), multiple Pareto optimal solutions with distinct decision variables can be projected onto an identical objective vector on the Pareto front. Numerous optimization algorithms develop sophisticated diversity preserving mechanisms to extensively explore the Pareto set (PS). However, existing work has neglected explicit learning of the Pareto set, while the intersection of machine learning and MMO remains largely unexplored. To advance the field, a multi-modal multi-objective particle swarm optimization that incorporates a growing neural gas network is proposed, termed MMPSO-GNG. The algorithm incrementally learns the topological structure of the PS to construct the network, in which a network-based solution generator and a selector are developed to facilitate exploration and maintain diversity. The generator leverages the network nodes to identify the neighborhood of each particle and guide the update of its position. The selector combines crowding distance and node-associated particle count to maintain a diverse set of particles. Performance evaluation on the CEC 2020 benchmark suite reveals that MMPSO-GNG surpasses five competing algorithms, therefore validating the effectiveness of integrating machine learning into particle swarm optimization to address complex multi-modal multi-objective problems.