In plastic surgery, an objective technique for evaluating the esthetic outcome of reconstructed breasts from breast images is required, because current evaluation relies on visual assessment by plastic surgeons, which is subject to inter-rater bias and variability. Conventional two-dimensional (2D) breast images are not always suitable for examining the esthetic contribution of intrinsic breast features, as they only provide partial appearances of the breast. While, specially-acquired three-dimensional (3D) images reflect those intrinsic features more comprehensively, its application has been hindered by the complexity of 3D geometry and the extensive implementation efforts required for a large number of cases. In this study, we proposed a novel technique to examine the contribution of intrinsic breast features to specific esthetic viewpoint, even with a limited number of cases, by pre-extracting relevant 3D geometric and color features. This approach enabled the analysis of the correlation between all combinations of 3D features and esthetic scores rated by plastic surgeons. Results from 174 clinical cases revealed that 3D geometric and color features exhibited distinct correlation patterns across different viewpoints, suggesting viewpoint-specific esthetic contributions. For each viewpoint, the top 10 correlated features were identified and used as predictors for multiple linear regression (MLR) modeling. The total esthetic score, aggregated from all the viewpoint-specific MLR models trained on high inter-rater agreement cases, demonstrated a strong correlation \(r\)  = 0.89 with the rater average score. These findings support the effectiveness of the proposed technique and its potential for appearance-based esthetic grading using conventional 2D breast images.