In this research, we introduce a bidirectional Neural Machine Translation (NMT) model that employs the Transformer architecture to facilitate translations between Konkani and English languages. Inspired by “Attention is All You Need”, we delve into variations in hyperparameters, including dropout rates and vocabulary size, to assess their impact on translation performance. The Transformer model, renowned for its capacity to handle long-range dependencies efficiently and in parallel, proves highly suitable for sequence-to-sequence tasks such as machine translation. We meticulously conduct a series of experiments, training the model with diverse hyperparameters configuration, and subsequently compare the outcomes to discern the optimal setup. Our findings underscore the superiority of the dash model, as it attains the highest BLEU score on the 3k test dataset, demonstrating its enhanced translation accuracy for both Konkani to English and English to Konkani translations. These results shed light on the paramount importance of hyperparameters selection in bidirectional NMT systems, underscoring the Transformer architecture’s potential in multilingual translation tasks. The study’s outcomes represent a noteworthy contribution to the advancement of machine translation techniques and set the stage for more effective communication tools in multilingual contexts.