Despite the impressive capabilities of current Large Language Models (LLMs), their interactions remain limited, particularly in high-stakes applications such as autonomous vehicles (AVs) where trust is paramount. We argue that this limitation arises from the LLMs’ insufficient understanding and expression, which are critical components of human communication. To address this, we propose adapting J.L. Austin's Speech Act Theory, as refined by Searle, to model LLMs in a way that enhances their ability to engage in trust-building conversations with users. By considering the locutionary, illocutionary, and perlocutionary acts within the framework of Speech Act Theory, we explore how LLMs can be designed to better recognize and respond to user intentions. We discuss the application of these intention-focused enhancements to the specific context of driver-passenger interactions in AVs. By demonstrating how an LLM service provider can engage in more nuanced, intention-aware communication, we illustrate the potential for increased user trust and confidence in the system, paving the way for the future of human-computer interaction. This research contributes to the ongoing efforts to develop trustworthy AI systems for high-stakes applications.