Scholarlink

<p>The extensive application of deep learning (DL) makes ensuring its reliability crucial, especially in the safety-critical domain. Mutation testing has been used to assess the test data quality for DL systems, but generating substantial mutants makes it expensive. The cost could be reduced by excluding mutation operators that are not beneficial. However, what DL mutation operators contribute to test effectiveness is still unknown. Therefore, determining which mutation operators are helpful is challenging. In this paper, we provide a fine-grained evaluation of DL mutation operators usefulness by introducing two measures that incorporate the classification results: Redundancy Score (RS), which qualities the redundancy of mutation operators, and Quality Score (QS), which qualities the ability of mutation operators in guiding the generation of high-quality test cases. Our empirical evaluation suggests that RS and QS could evaluate DL mutation operators from a dual perspective. In the context of selective mutation, utilizing RS and QS to prioritize DL mutation operators provides the benefit of reducing a significant number of mutants while maintaining high test effectiveness, thus helping optimize DL mutation testing. Additionally, for a more comprehensive analysis of selective mutation, we introduce the definition of FaultType for DL mutation testing and further evaluate the performance of the measures in aiding the detection of diverse faults. Experimental results show that RS and QS could also expedite the detection of various fault types in DL models, which contributes to a more thorough evaluation of test data.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A fine-grained evaluation of mutation operators to boost mutation testing for deep learning systems

  • 期刊论文

  • 发表日期：

    2025年2月11日

查看全文
  • Zhiyi Zhang,
  • Yichun Wang,
  • Yongming Yao,
  • Ziyuan Wang,
  • Zhiqiu Huang

摘要

The extensive application of deep learning (DL) makes ensuring its reliability crucial, especially in the safety-critical domain. Mutation testing has been used to assess the test data quality for DL systems, but generating substantial mutants makes it expensive. The cost could be reduced by excluding mutation operators that are not beneficial. However, what DL mutation operators contribute to test effectiveness is still unknown. Therefore, determining which mutation operators are helpful is challenging. In this paper, we provide a fine-grained evaluation of DL mutation operators usefulness by introducing two measures that incorporate the classification results: Redundancy Score (RS), which qualities the redundancy of mutation operators, and Quality Score (QS), which qualities the ability of mutation operators in guiding the generation of high-quality test cases. Our empirical evaluation suggests that RS and QS could evaluate DL mutation operators from a dual perspective. In the context of selective mutation, utilizing RS and QS to prioritize DL mutation operators provides the benefit of reducing a significant number of mutants while maintaining high test effectiveness, thus helping optimize DL mutation testing. Additionally, for a more comprehensive analysis of selective mutation, we introduce the definition of FaultType for DL mutation testing and further evaluate the performance of the measures in aiding the detection of diverse faults. Experimental results show that RS and QS could also expedite the detection of various fault types in DL models, which contributes to a more thorough evaluation of test data.