<p>A millimeter-wave communication system uses multiple inputs and multiple outputs, which has high gains and spectral efficiency. To overcome empty path costs and create interactions using a suitable signal-to-noise ratio, large antenna arrays are used to perform precoding. To solve the complex problem without incurring significant performance losses, a novel deep learning-based method rather than methods with high delay, such as greedy search and saber selection, is proposed in this paper. For antenna selection, an optimized convolutional neural network (CNNs) is presented. In order to select antennas, the neural network takes the signal matrices as entries and returns the subset with the highest spectrum efficiency. An adaptive coati optimization technique is proposed for optimizing the weighting and bias of all of the layers in the CNN. As a consequence, a successive interference cancelation algorithm is used for prior coding with choice detectors to mitigate the route loss caused by high-frequency transmission. Simulation results show that the proposed model improves the throughput of the network. Besides, bit error rate and mean square error are reduced significantly by 0.44% and 1.54% than the existing antenna selection models.</p>