This paper presents a transformer-driven approach for non-standard word (NSW) normalization in Bengali text-to-speech synthesis (TTS) systems. Our text normalization (TN) approach is realized over three modules: pre-processing, NSW classification, and token-to-word expansion. The pre-processing module tokenizes the input sentences into words. The non-standard words are classified based on their format and other discriminative features in the next module. Apart from our proposed transformer-based approach, the present study explored four baseline TN approaches: rule-based, Conditional Random Field (CRF)-based, Long Short-Term Memory (LSTM)-based, and biLSTM-based. Based on the predicted types of the NSWs, standard words (SWs) are generated through token-to-word expansion using hand-crafted expansion rules. The performance of these five TN approaches is assessed on a corpus of around 6,000 NSWs, consisting of 6 initial and 14 final NSW types. The performances of these approaches are assessed using measures representing (i) direct accuracy and (ii) enhancements in text-to-speech synthesis systems. Based on direct accuracy, our proposed approach predicted correct NSW types for about 92% of cases. This performance is around 2% better than the performance of a rule-based approach. On the other hand, application-based measures guarantee that the generated utterances (i) sound natural and (ii) improve the quality of speech synthesis systems.