This paper gives a systematic evaluation of character categorization for the Tal-Patra script, which is one of the most difficult task in contemporary research. Pre-processed and segmented characters from the Tal-Patra manuscript, particularly related to healthcare science, were used for classification. Additionally, datasets such as the Tamil vowels character dataset, the University of Jaffna Tamil character dataset, and the Arabic handwritten character dataset were employed to assess the performance of the proposed models. Two models were proposed: an optimized LeNet-based Convolutional Neural Network (CNN) and a Convolutional Bidirectional Long Short-Term Memory (BLSTM) model. He initialization was applied for weight optimization. The classification of Tamil Tal-Patra manuscript characters achieved 97.9% accuracy with the optimized LeNet-based CNN and 85.2% with the convolutional BLSTM. The optimized LeNet-based CNN outperformed state-of-the-art models, establishing new benchmarks in the analysis of the Tamil Tal-Patra manuscript.