Large Language Models (LLMs) have shown remarkable capability in answering complex questions, but their reasoning processes remain largely ungrounded. This opacity is problematic in domains like medicine, where ungrounded answers can erode user trust. We address the lack of grounded reasoning in LLMs by introducing an approach that integrates LLMs with a reasoning framework based on an external ontology. In our method, we employ an agentic reasoning strategy: the LLM traverses a hypergraph to iteratively gather grounded evidence and infer answers in a transparent manner. The reasoning capabilities are evaluated on the MedQA medical question answering benchmark. Experimental results show that our agent, equipped with hypergraph search, achieves competitive accuracy with state-of-the-art baselines while providing ontology-grounded reasoning traces.