<p>In the modern era, sensor based Human Activity Recognition (HAR) has been of great help in case of health monitoring and rehabilitation. Deep learning models are increasingly used in this emerging field to capture the temporal patterns. However, most of the deep learning based HAR models have used devices of similar hardware configurations. Moreover, the usage behavior, that is, how the user is holding the device while collecting the data also plays an important role. Device configurational or positional dependency may cause incorrect recognition of activities. The publicly available benchmark datasets on HAR suitable for deep learning based analysis does not reflect such information either. In this paper, we have proposed a device and position independent smartphone-based HAR framework through designing an ensemble of conditional classifiers. LSTM has been utilized to learn long-term dependency from the time series data. Detailed activities are predicted using the probability based majority voting ensemble model. The proposed framework is implemented for a dataset collected from 15 users using 8 different smartphone configurations. They kept the device in right pant pocket, shirt pocket, or at hand. If the training and testing are performed on the same dataset, then the system performs with more than 90% accuracy. This is pulled down to 60% approximately when training and test datasets are formed using different devices. When the proposed ensemble model is used, the system is found to be capable enough to identify both static and dynamic activities even from our moderately sized dataset, with an accuracy of around 75% even when the training devices and usage pattern differ from the test device and usage pattern combination.</p>