Background <p>Cardiovascular magnetic resonance (CMR) offers state-of-the-art volume, function, fibrosis and oedema imaging. Quality assurance (QA) tasks, such as quantitative parameter reproducibility assessments, the evaluation of AI methods, and the assessment of trainees have become essential to CMR. However, the explainability of how qualitative differences impact quantitative differences remains underexplored. Our aim is to demonstrate a semi-automated QA tool, Lazy Luna’s (LL) applicability to typical CMR QA application cases.</p> Methods <p>A software feature error-tracing is designed that allows for quickly pinpointing qualitative reasons for quantitative differences and outliers. Three QA application cases were designed. First, LL was applied to perform <b>outlier detection</b> for inter- and intraobserver analyses to detect failure cases and provide qualitative explanations. Outlier detection was performed on several typical images types. Second, LL supported an <b>Artificial intelligence (AI) evaluation</b>, in which an AI method was compared to a CMR-expert of 144 patients. LL assessed the acceptability of AI biases for left and right ventricular (LV, RV) end-systolic, –diastolic, and stroke volumes (ESV, EDV, SV), ejection fractions (EF) and the myocardial mass (LVM). Annotations were examined to explain the qualitative differences that resulted in good and poor parameters. The AI investigation was recorded as a video. Third, LL was used to provide a <b>Trainee Feedback</b> to a CMR beginner. The trainee was compared to an expert on several imaging techniques to investigate outliers.</p> Results <p>For the outlier detection, LL detected segmentation differences that caused parameter differences on multiple sequences. For the AI evaluation calculated clinical parameter biases to be: LVESV:-3.1 ml, LVEDV:2.1 ml, LVSV:6.5 ml, LVEF:3.0 ml, RVESV:0.3 ml, RVEDV:-3.8 ml, RVSV:-4.2 ml, RVEF:-1.4 ml, LVM:-2 g. Inspecting the causes for outlier differences revealed that juxtaposed basal slice failures caused unacceptable LVSV deviations between AI and expert. For the trainee assessment, LL showed that trainee parameters exceeded tolerance ranges. The segmentations could be improved to better mirror expert segmentations and close the parameter gaps.</p> Conclusion <p>Lazy Luna, as a semi-automated quality assurance tool, is applicable to several quality assurance application cases in CMR.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A semi-automated quality assurance tool for cardiovascular magnetic resonance imaging: application to outlier detection, artificial intelligence evaluation and trainee feedback

  • Thomas Hadler,
  • Leonhard Grassow,
  • Johanna Kuhnt,
  • Richard Hickstein,
  • Hadil Saad,
  • Maximilian Fenski,
  • Jan Gröschel,
  • Ralf-Felix Trauzeddel,
  • Edyta Blaszczyk,
  • Clemens Ammann,
  • Darian Viezzer,
  • Anja Hennemuth,
  • Steffen Lange,
  • Jeanette Schulz-Menger

摘要

Background

Cardiovascular magnetic resonance (CMR) offers state-of-the-art volume, function, fibrosis and oedema imaging. Quality assurance (QA) tasks, such as quantitative parameter reproducibility assessments, the evaluation of AI methods, and the assessment of trainees have become essential to CMR. However, the explainability of how qualitative differences impact quantitative differences remains underexplored. Our aim is to demonstrate a semi-automated QA tool, Lazy Luna’s (LL) applicability to typical CMR QA application cases.

Methods

A software feature error-tracing is designed that allows for quickly pinpointing qualitative reasons for quantitative differences and outliers. Three QA application cases were designed. First, LL was applied to perform outlier detection for inter- and intraobserver analyses to detect failure cases and provide qualitative explanations. Outlier detection was performed on several typical images types. Second, LL supported an Artificial intelligence (AI) evaluation, in which an AI method was compared to a CMR-expert of 144 patients. LL assessed the acceptability of AI biases for left and right ventricular (LV, RV) end-systolic, –diastolic, and stroke volumes (ESV, EDV, SV), ejection fractions (EF) and the myocardial mass (LVM). Annotations were examined to explain the qualitative differences that resulted in good and poor parameters. The AI investigation was recorded as a video. Third, LL was used to provide a Trainee Feedback to a CMR beginner. The trainee was compared to an expert on several imaging techniques to investigate outliers.

Results

For the outlier detection, LL detected segmentation differences that caused parameter differences on multiple sequences. For the AI evaluation calculated clinical parameter biases to be: LVESV:-3.1 ml, LVEDV:2.1 ml, LVSV:6.5 ml, LVEF:3.0 ml, RVESV:0.3 ml, RVEDV:-3.8 ml, RVSV:-4.2 ml, RVEF:-1.4 ml, LVM:-2 g. Inspecting the causes for outlier differences revealed that juxtaposed basal slice failures caused unacceptable LVSV deviations between AI and expert. For the trainee assessment, LL showed that trainee parameters exceeded tolerance ranges. The segmentations could be improved to better mirror expert segmentations and close the parameter gaps.

Conclusion

Lazy Luna, as a semi-automated quality assurance tool, is applicable to several quality assurance application cases in CMR.