In the aftermath of the COVID-19 pandemic, it became crucial to understand the public’s sentiments towards different vaccination perspectives in real time. For the last few decades, OSM has become the most important repository to capture public opinion in real-time. To this end, some prior studies utilized deep learning based methods to categorize individuals based on their stances (Pro-Vax, Anti-Vax, or Neutral) regarding vaccination by analyzing COVID-19-related tweets. However, going beyond a simple sentiment analysis, the analysis reveals a range of concerns that are responsible for people’s hesitancy towards vaccines. These concerns encompass a broad spectrum, including conspiracy theories, political suspicions, and various uncertainties. Moreover, multiple concerns are expressed in a single tweet. Thus, we conceived of this as a multi-label classification problem and adopted various deep learning based approaches to address it. We observed that for the unseen test dataset, among all the models, fine-tuned Covid-Twitter-BERT (CT-BERT) with Distribution Balanced Loss (DBLoss) outperformed all other models with a weighted-F1 score of 0.74.