Handwritten Bangla-digit recognition holds significant relevance across various domains. While deep convolutional neural network (DCNN) models have demonstrated promising accuracy in recognizing these digits, achieving enhanced accuracy for real-life applications remains a priority. This paper addresses this challenge by investigating a transfer learning approach utilizing four pre-trained DCNN models: GoogLeNet (Inception V3), AlexNet, CapsuleNet, and MobileNet. Transfer learning involves reusing the initial layers and training only the final layers of a DCNN architecture on a source task to classify a new task. The unified approach combines the outputs of these models using a normalized probability score (voting technique) through an inference engine to determine the ultimate recognition decision. Extensive experimentation conducted on the benchmark NumtaDB dataset yields an optimal accuracy of 99.58%, surpassing the results achieved by other recent methods.