Sleep is an essential biological function that significantly contributes to maintaining optimal health and overall well-being in humans. Accurate classification of sleep stages is of paramount importance for diagnosing sleep disorders and comprehending the underlying sleep mechanisms. Recent sleep staging methods have yielded promising results. However, the application of Large Language Models (LLMs) enhanced with prompt-aware learning for sleep stage classification remains largely unexplored. To address this gap, we propose a novel sleep stage classification framework, Sleep-LLM, which integrates LLMs with prompt-aware learning techniques. By adopting the Low-Rank Adaptation (LoRA) strategy, we fine-tune the LLMs to better adapt to the sleep stage classification task. Additionally, we design a detailed-cueing prompt which provides rich contextual information and classification guidance, significantly enhancing the performance of sleep stages classification. Extensive experiments on ISRUC-S3 dataset demonstrate that our proposed Sleep-LLM achieves competitive results. We hope this study paves the way for further exploration of integrating LLMs with prompt-aware learning in sleep stages classification.