Social media is a popular source of information for the public and therefore it is critical that health officials are able to engage with the public in an effective way on these platforms. We were interested in the sentiment reflected in tweets from the public responding to messaging from these agencies. This paper reports on a comparison of different machine learning models for use in the multi-classification of the sentiment of such tweets. Tweets were first collected and manually labeled into seven different sentiment classes. The labelled tweets were then processed to form a core dataset. Several machine learning models were compared using this dataset and augmentation of the dataset, including using up-scaling and the use of “artificial” tweets constructed from the core dataset. The paper reports on the techniques used during preprocessing, augmentation of the dataset, the machine learning models, and the results obtained.