<p>Yoga is a predominant form of exercise that involves a predefined order of specific postures. Regular practice of yoga can enhance physical and mental wellbeing for a performer. Yoga actions have to be practice under the supervision of a yoga instructor, as inappropriate practicing of yoga poses may lead to a health problem. Due to busy routine, nowadays people are unable to go yoga classes for practice. Though this is necessary for having a smart automated model for the self- learning process of yoga poses. This paper is proposing a DDQN–LSTM based yoga activity recognition system for self-learning yoga poses with automated recognition of postures by utilizing deep learning classifier. In this research, to classify the different yoga postures, we used the deep learning based long short term memory (LSTM) classifier. A double deep Q-network (DDQN) approaches has been used as an auto-labelling technique for unlabeled data. We constructed skeleton based dataset for respective yoga poses using a non-wearable Kinect sensor. The experimental outcomes determine that the proposed system efficiently enhances the yoga recognition accuracy. Our proposed system has achieved a remarkable accuracy of 96.70% for yoga pose recognition.</p>