This study proposes a new framework to improve significant wave height (Hs) estimation from X-band radar imagery. Conventional Hs estimation relies on the linear relationship between wave height and signal-to-noise ratio (SNR), but its accuracy is often degraded by disturbances like rain. Although Artificial Neural Networks (ANNs) have been used for correction, applying a single model to all data, regardless of quality, can paradoxically reduce accuracy for high-quality images. To overcome this, this paper introduces a selective estimation system that first classifies radar images using physical metrics to detect precipitation and assess wave pattern clarity. An ANN model is then selectively applied only to the images identified as low-quality or difficult to predict, while a traditional linear regression model is used for high-quality ones. This selective approach maximizes overall accuracy by applying complex ANN correction only when necessary, preserving the integrity of reliable data. Data for this study was collected from an X-band radar in Gangneung, South Korea, from 2020 to 2023.