The process of automating handwritten manuscripts into text that can be read by machines is advancing significantly. Convolutional neural networks (CNNs) are used to classify images of handwritten characters to achieve human-like accuracy. Extensive parameter training is a drawback of the state-of-the-art CNN models and is a problem of storage of a lot of parameters and increases computational cost. The objective of the study is to design an efficient model to address the challenges posed by CNNs. The CNN model utilizes the global pooling layers, specifically global average pooling and global max pooling, which are alternatives to flattening layers. Layers of global pooling lower the dimensions of input feature maps before passing them to a fully connected layer. Our study utilizes the global pooling layers to create feature descriptors of machine learning models. The features obtained from global average and max pooling (GAP and GMP) layers are combined to form feature descriptors of classification models. Three Odia datasets of handwritten images in the Odia language are used to evaluate the efficacy of the GAP_GMP model. Comparing our suggested model to other cutting-edge techniques, it outperforms them in classification accuracy. Finally, a comparative analysis is carried out with other contemporary methods to support the effectiveness of the proposed strategy. By incorporating GMP and GAP layers instead of traditional flattening techniques after convolution operations and combining features from these global pooling layers, the study created robust feature descriptors for machine learning models.