In response to the problem of unreasonable spatial layout caused by low space utilization in current environmental design, genetic algorithm is introduced to optimize spatial configuration, improving the effectiveness and practicality of design from the perspectives of space utilization, functional matching, aesthetic index, and environmental sustainability. Firstly, a genetic algorithm based model is constructed by encoding spatial layout features, designing fitness functions, and setting selection, crossover, and mutation operations, evaluation indicators and constraints are defined to reflect the rationality of spatial layout. Then, the selection, crossover, and mutation mechanisms of genetic algorithms are utilized to evaluate fitness and apply genetic operations to generate new spatial layout schemes, continuously approaching the optimal design to optimize the design scheme and explore more innovative spatial configurations. Finally, the effectiveness of the optimization results is verified through experiments, and the effectiveness of traditional methods is compared to analyze their potential application in practical environmental design. By applying genetic algorithm, the rationality index of spatial layout has been significantly improved. Compared with traditional evaluation methods, the optimized design shows significant advantages in both functionality and aesthetics. The spatial utilization rate of green space under genetic algorithm is 67.5%, which is significantly higher than the traditional method’s 60.1%. This study indicates that genetic algorithms provide a new and effective tool for environmental design, which helps to achieve more rational and efficient spatial layout.