Text classification is a pivotal task in natural language processing (NLP), underpinning a wide range of applications such as sentiment analysis, spam detection, and topic categorization. While deep learning-based models have significantly advanced the state of the art in this domain, their complex decision-making processes raise concerns regarding explainability and trust, particularly in high-stakes applications. This study investigates the use of large language models (LLMs) as a post hoc explainer for text classification tasks. We propose a novel, prompt-based framework that harnesses the semantic reasoning and generative capabilities of pretrained LLMs to identify influential input features and produce coherent, human-interpretable explanations. Furthermore, we conduct a comprehensive faithfulness evaluation, comparing our method to traditional explanation techniques. The results highlight the potential of LLMs not only as effective classification tools but also as explainable agents, offering a promising direction for explainability in deep learning-based NLP systems.