PWAM-LSTM: prime word focused attention mechanism in long short term memory

Attention helps by limiting the data presented to the deep neural network and hence is computationally more efficient. In addition, it can be used for variable size input problems. This paper designs an attention mechanism in Recurrent Neural Network (RNN). Depression Analysis is one of the most fundamental applications in human health systems. The proposed attention mechanism is implemented in depression analysis using Bidirectional Long Short Term Memory (BiLSTM). Experiments are conducted on the most popular Dreaddit dataset and Sentiment_tweet3 dataset. It is then validated on Stress Annotated Dataset (SAD). Existing BiLSTM network is compared with the proposed attention based BiLSTM network. Various other recent attention mechanism models are also compared with the proposed one. The results proved there is a drastic improvement of 4% in accuracy in Dreaddit dataset and 10% increase of accuracy in Sentiment_tweet3 dataset when compared to BiLSTM without attention mechanism. The proposed attention mechanism can be implemented in suicidal detection, mental illness, stress analysis and so on.