Prudent Promotion, Steady Development: Capability and Safety Considerations for Applying Large Language Models in Medicine

The powerful capabilities of large language models (LLMs) in the medical field have been affirmed by various benchmark tests. However, safety assessments are indispensable for high-risk medical applications. We systematically analyzed LLMs, including their technical principles, applicability in healthcare, medical capability assessment, potential security risks, and countermeasures. The study found that LLMs demonstrate strong capabilities in medical text processing, decision support, and text generation, but also have risks like “hallucination”. To ensure safe and effective applications of LLMs, we analyzed the causes of hallucination and proposed using output detection and prompting techniques to mitigate hallucinations generated by LLMs. The results affirm that LLMs can advance AI innovation in healthcare, but need to be introduced prudently without comprehensive safety assessments.