A Romanization Method for the Bengali Language with Efficient Encoding Scheme

Transliteration is crucial in natural language processing as it enables the conversion between two languages while retaining their phonetic representation. The ability to transliterate the Bengali language is essential for cross-lingual communication. Most Bengali machine transliteration techniques rely on one-hot coding to numerically represent features, which is computationally intensive, requires significant memory, and results in lower accuracy in some cases. Addressing these limitations, we present an efficient feature representation method utilizing binary coding and compare it against the one-hot coding approach using two machine learning models: SVM and random forest. The proposed method is evaluated on the Dakshina dataset and the NEWS 2018 dataset for Bengali to English transliteration. The experimental results indicate that our approach outperforms the traditional methods while requiring significantly less memory and training time. Overall, the proposed method enables a better romanization of frequently used words.