SMS Spam is a big security issue at the global scale. Static spam prevention techniques such as blocking known spam SMS senders do not work very well in practice because it is pretty easy for spam senders to change their mobile numbers. In the last decade, several machine learning techniques have been studied to deal with SMS spam issues. These techniques achieved a lot of success but still lack the capability to deal with very complicated SMS spam content as the content is usually short. In this study, we use a different approach. We utilize the recent advances in large language models (LLMs) and fine-tune a specific model to deal with SMS spam. The evaluation using a real-world dataset showed that a fine-tuned LLM can improve the performance of the classification task.