The integration of Large Language Models (LLMs) with Internet of Things (IoT) devices presents significant opportunities for enhancing edge computing capabilities, but it also poses challenges due to the computational demands of LLMs and the resource constraints of IoT devices. This paper introduces LLMIoT, a novel framework that enables efficient interaction between IoT devices and a localized LLM. The framework utilizes a Raspberry Pi 4B as a central hub, hosting a quantized version of the qwen2:0.5b model, and employs FastAPI with a Uvicorn ASGI server to manage communications. The study implements the framework using commercially available ESP8266, ESP32, and Arduino MKR1000 modules as IoT clients, which communicate with the hub over a local Wi-Fi network via representational state transfer—RESTful application programming interface (API) calls. To evaluate the system’s performance, we conduct experiments using 18 carefully selected test prompts across various categories. The study includes a comprehensive analysis of send and receive durations for each IoT module, utilizing analysis of variance (ANOVA) and Tukey’s honest significant difference (HSD) test to identify performance variations. Results indicate that while send durations were similar across all modules, significant differences were observed in receive durations, particularly for the MKR1000 module. The LLMIoT framework demonstrates the feasibility of integrating LLMs with resource-constrained IoT devices, offering insights into performance characteristics and addressing key concerns such as latency, privacy, and offline capability in resource-constrained edge computing.