Named Entity Recognition is an important task in the biomedical domain. State-of-the-art models for this task use neural network encoders for capturing contextual information of words in the input sentence and linear conditional random fields decoders to predict the most probable label sequence. However, the linear conditional random fields can only model dependencies among adjacent output labels. In this paper, we propose a bidirectional decoder model that uses two recurrent decoders to handle all the dependencies between labels. We introduced attention mechanisms in each decoder layer to help them to focus on the important words in the input sequence. We experimented with three types of attention mechanism. We observed that these attention mechanisms improve the performance of our model in the MedMentions and i2b2 2010 datasets.