Large Language Models (LLMs) such as LLaMA are computationally intensive and raise concerns about energy consumption. This study investigates LLaMA’s execution on IMAX3, a novel coarse-grained reconfigurable array (CGRA)-based in-memory accelerator architecture designed for both high-performance computing and low-power edge devices. We ported llama.cpp, a GPU-compatible LLM execution environment, to IMAX3, enabling direct performance and energy comparisons with CPUs and GPUs. Results show that while IMAX3 has longer execution times compared to CPUs and GPUs for both Q8_0 and Q3_K quantization, it achieves competitive PDP and EDP in certain thread configurations. Limitations such as the lack of ARM cores for control and large, power-hungry local memory present opportunities for optimization. Future work includes investigating alternative memory technologies, reducing memory capacity, and scaling with additional ARM/Intel cores to further improve energy efficiency and reduce power consumption of the IMAX3 prototype.