<p>Testing autonomous vehicles is a challenging task, usually carried out using test scenarios, which are often derived manually from accident statistics and real traffic datasets, relying on expert knowledge and intuition to select the most relevant ones, a cumbersome task given the large size of these datasets. In this article, we suggest a model-based methodology to compare scenarios using quantitative measures computed from a formal model of an autonomous vehicle in its environment and a set of user-defined interesting event sequences representing evaluation criteria. Quantitative measures are computed using two different approaches: probabilistic model checking of temporal logic properties and conformance testing guided by test purposes—both temporal properties and test purposes being derived from the considered event sequences. This methodology facilitates the selection of the scenarios having the best tradeoff between coverage and overall testing cost (both for simulation and field testing). We illustrate the methodology by comparing variations of several scenarios, derived from frequent situations in accident statistics, using evaluation criteria such as occurrence of collision, successful arrival, and duration.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Assessing Test Scenarios for Autonomous Driving

  • Jean-Baptiste Horel,
  • Philippe Ledent,
  • Radu Mateescu,
  • Wendelin Serwe,
  • Aline Uwimbabazi

摘要

Testing autonomous vehicles is a challenging task, usually carried out using test scenarios, which are often derived manually from accident statistics and real traffic datasets, relying on expert knowledge and intuition to select the most relevant ones, a cumbersome task given the large size of these datasets. In this article, we suggest a model-based methodology to compare scenarios using quantitative measures computed from a formal model of an autonomous vehicle in its environment and a set of user-defined interesting event sequences representing evaluation criteria. Quantitative measures are computed using two different approaches: probabilistic model checking of temporal logic properties and conformance testing guided by test purposes—both temporal properties and test purposes being derived from the considered event sequences. This methodology facilitates the selection of the scenarios having the best tradeoff between coverage and overall testing cost (both for simulation and field testing). We illustrate the methodology by comparing variations of several scenarios, derived from frequent situations in accident statistics, using evaluation criteria such as occurrence of collision, successful arrival, and duration.