Gender recognition of insects plays a crucial role in various ecological, agricultural, biological studies, farming, and biodiversity research. However, gender identification by conventional methods is frequently a time-consuming and labor-intensive process. In this study, we leverage a hybrid Convolutional Neural Network and Vision Transformer model to construct a novel approach for automatic gender recognition of cocoons using X-ray images. The suggested approach starts by preprocessing the X-ray images to improve attributes associated with gender traits. Then, a hybrid architecture that combines the self-attention mechanism of Vision Transformers for capturing global dependencies inside the image and the strengths of CNN in feature extraction is proposed for gender classification. In addition, a dataset of X-ray images of silkworms with gender labels is used in the study, guaranteeing reliable model training and assessment. We use pre-trained CNN and Vision Transformer architectures on large-scale image datasets to initialize the model weights. We conducted experiments on the dataset using different assessment metrics, including accuracy, precision, recall, and F1-score, to assess the performance of the suggested approach. To illustrate the efficacy of the hybrid architecture, comparisons with the basic CNN and Vision Transformer models are carried out. The experimental findings show that the proposed strategy delivers 95% classification accuracy.