Millimeter-wave (mm-Wave) communication plays an important crucial role in radar and 5G/6G wireless systems because of its capacity to provide high bandwidth and low latency. Over the frequency range (30–300 GHz), minimum attenuation region is called “window frequency.” Conventional window frequencies are 30, 94, 140, and 220 GHz which is correct for temperate region. However, in the tropical location, it was found 37.17 and 72 GHz instead of the above-mentioned conventional values. This paper investigates machine learning techniques for the identification of “window frequency” in the mm-Wave spectrum, using supervised, unsupervised, and reinforcement learning methods. Here, in this paper, a hybrid learning framework that combines signal processing techniques with machine learning algorithms to enhance estimation accuracy. Experimental results show that this strategy outperforms conventional techniques in terms of precision, recall, and mean absolute error (MAE).