Sepsis exhibits high mortality rates, and efficient data exploration is vital for advancing clinical research. However, the complexity of sepsis research poses significant challenges for healthcare professionals, as navigating large and intricate medical databases demands specialized skills and substantial effort. Natural Language to SQL (NL2SQL) frameworks, leveraging few-shot learning, offer a promising solution by allowing clinicians to query databases like MIMIC-IV using natural language, thereby bypassing the need for deep SQL expertise. Yet, current NL2SQL systems, built on language models pretrained on general-domain data, encounter critical issues in the healthcare domain due to data scarcity and insufficient domain-specific training. To address these challenges, we present a customized NL2SQL framework for medical data exploration that integrates data augmentation techniques to mitigate data scarcity and a query-retrieval mechanism to select relevant SQL examples. This approach significantly enhances few-shot learning capabilities, boosting accuracy from 0.76 to 0.94 on unseen queries, and thereby strengthens the utility of NL2SQL in clinical research.