A large number of manual interventions make the modeling efficiency low, ignore the impact of environmental factors, the design result is not ideal, cannot make full use of the advantages of the algorithm, and the optimization effect is not good. This paper aims to study a 3D environmental design modeling method based on algorithm optimization, which can automatically optimize the design results and fully consider the influence of environmental factors to improve the modeling efficiency and design quality. Three-dimensional environmental design modeling refers to the simulation and optimization of environmental elements through certain algorithms in computer-aided design software to generate three-dimensional models that meet the design requirements. The process mainly includes data acquisition, model construction, parametric modeling, visual analysis and so on. The main ideas of the modeling method proposed in this paper are as follows: obtaining environmental data, such as terrain, soil and vegetation, through remote sensing technology, field survey and other means, preprocess these data, such as denoising, interpolation, resampling, etc., and divide complex environmental problems into multiple sub-problems by adopting dive-and-conquer strategy. For each subproblem, genetic algorithm, particle swarm optimization algorithm and neural network were selected for modeling, and model parameters meeting the design requirements were automatically searched to reduce the degree of manual intervention. The optimized model parameters were applied to the parametric modeling method to generate a three-dimensional model, which was then displayed visually. Finally, the model was iteratively optimized according to feedback. In this paper, the function of algorithm optimization is confirmed by means of experimental simulation and data comparison. The experimental results showed that the accuracy of the experimental group was 0.8, 0.85, 0.9, 0.95, 0.98, and the accuracy of the control group was 0.7, 0.78, 0.85, 0.92, 0.96.