This paper presents AttentionEnsemble, a model based on a combination of multi-head self-attention for long-term context retention and local attention for short-term context, to significantly enhance contextual embeddings for Named Entity Recognition (NER) task. NER, an integral natural language processing (NLP) task, involves determining entities in text, such as names of persons, locations, and organizations. We have used tokenizer and the contextual embeddings obtained from the pre-trained DistilBERT model as a foundation for capturing linguistic structures and context. Our Attention Ensemble model refines these embeddings with the fusion approach for improved NER performance when fine-tuned over CoNLL-2003 and HiNER datasets. The experiments conducted on these datasets demonstrated substantial improvements in comparison to five baseline models taken into this study. Notably, our approach achieved an F1 score of 0.9757 on CoNLL-2003 and 0.9495 on HiNER datasets surpassing the scores achieved by the baseline models ACE + document-context (0.946 on CoNLL 2003) and XLM-Roberta (Large) (0.8878 on HiNER) datasets respectively. The substantial improvement exhibited by our model underscores its distinct contribution towards balancing long-term and short-term contexts in English as well as non-English languages, thus establishing its effectiveness over the reputed models available in the literature.