Social media plays a crucial role in providing timely information during disasters. On social media platforms, people share information about victims conditions, impacted infrastructure, and the severity of damage. Given the vast amount of information available on social media, machine learning techniques can accelerate the analysis process. Despite the usefulness of social media data, deploying it in operational settings remains challenging. One of the key challenges is the availability of labeled data for analysing the sentiment. This paper compares human-labeled and machine-labeled data. The data are evaluated using various machine learning algorithms, including Support Vector Machine, Logistic Regression, and Naive Bayes. The experimental results demonstrate that human understanding captures nuances more effectively when assigning labels, leading to better classifier performance. This research is expected to assist policymakers in considering the proposed approach for disaster response.