Overview of CHIP 2023 Shared Task 5: Medical Literature PICOS Identification

Evidence-based medicine has become widely adopted among medical professionals, emphasizing the need for efficient information retrieval and evidence-based clinical practice. The PICOS framework, which stands for Population, Intervention, Comparison, Outcome, and Study design, has gained popularity as a structured approach for formulating research questions and retrieving relevant literature. In this paper, we present an overview of the “Medical Literature PICOS Identification” task organized at the CHIP 2023. The task aims to develop automated systems that accurately extract key information related to the PICOS components from Chinese medical research papers. The dataset consists of 4500 academic papers, divided into training, development, and test sets. Macro-F1 score are used as evaluation metric. Thirteen participating teams submitted their solutions, with the top-ranked team achieving a score of 0.81. This paper provides valuable insights into the challenges, methodologies, and performance of the top-ranked teams, serving as a resource for future research and system development in this domain.