Machine Translation (MT) witnessed drastic change in its design and approach in the last few years. These days we exploit neural based translation systems known as Neural Machine Translation Systems which has almost replaced the Statistical Machine Translation Systems. In this paper we attempted to evaluate the performance of NMT while translating English to Bangla. One of the important requirements of NMT systems is availability of sufficient corpora during training. However, in English to Bangla, scenario we don’t have sufficient corpora. Hence, we used a data augmentation technique to enhance our data size. We trained our attentional LSTM and attentional Transformer models with original corpus as well as augmented corpus separately. We achieved a better BLEU score when our models are trained with augmented data set. Apart from that we also attempted to investigate which augmented technique generates better BLEU score in our chosen English➔Bangla language pair. Finally, we draw some useful conclusions.