This study examines the challenges of identifying a universally effective prompt for medical question answering with large language models (LLMs). The complexity of medical terminology and the need for multi-step reasoning make this task particularly difficult, as even minor prompt variations can lead to significant differences in model responses. The research compares various prompting strategies, such as in-context learning, chain-of-thought reasoning, and self-consistency, demonstrating that the effectiveness of a prompt is influenced by the interaction between the prompt itself, the model’s architecture, and the specific characteristics of the medical query. Remarkably, one of the smallest LLMs showed a 25-point accuracy improvement in one case and almost obtained medical license in another, using the same inference technique, underscoring the importance of adaptive prompt strategies tailored to the input and model.