Cutter posture determination is the key task to make good use of 5-axis milling. In this paper, an experiment-based method is firstly proposed to optimize cutter posture for minimum cutting force, surface residual stress, and roughness, which represent the surface quality. The effects of cutter posture on cutting force, surface residual stress, and roughness are experimentally investigated with the Pareto analysis of variance. Subsequently, mathematical models are established using response surface methodology. The models are validated by various experiments and the average error-rate is only 9.1%. Finally, a multi-objective optimization method of cutter posture is proposed based on D-optimization method and gray wolf optimizer algorithm. The surface quality is greatly improved after cutter posture optimization. The proposed study presents a new perspective for improving surface quality through cutter posture optimization and greatly benefit 5-axis machining. It will be used for tool path optimization in the future work.