CBR Evaluation Pyramid: A Pragmatic Process for Evaluating Case-Based Reasoning Systems

Case-based reasoning (CBR) systems can mimic human reasoning by referencing precedent cases. This paper proposes an integrated evaluation strategy for such systems by addressing the interdependence between the CBR process phases and the CBR knowledge containers that support them. The proposed evaluation process is termed as CBR evaluation pyramid. In real-world settings, this evaluation methodology has been instrumental in the development of a CBR system for a medical study, improving the CBR systems used by physiotherapists to treat musculoskeletal disorders. The proposed evaluation pyramid not only provides researchers with a comprehensive guide to develop CBR-specific evaluation strategies, but also offers practitioners a pragmatic tool for the ongoing evaluation and maintenance of CBR systems.