Social media is a platform where users give expression to their feelings and opinions on different topics. The content of the social media posts may indicate symptoms of depression or poorer mental health. In this paper, we present an automated procedure for depression analysis of social media posts on the Reddit social media platform which allows users to write longer posts expressing their emotions and state of mind. To achieve this aim, we train a transformer model that employs multi-head attention, which is known to have achieved remarkable success in sequence modeling. Specifically, we employ the pre-trained transformer-based language model BERT to classify Reddit posts into depression and non-depression categories. A comprehensive comparison with nine traditional machine learning algorithms is provided. The transformer model outperformed all other models with a high classification accuracy of 97%. The results establish the utility of multi-head attention transformers for modeling long sequences and understanding the emotive state of the user from social media posts.