Towards a Competence Profile for Civil Protection Exercise Evaluators: Findings from the INEGMA-E² Delphi Survey

Scientific-based evaluation has become a widely accepted instrument of quality assurance for civil protection (CP) exercises. To enhance reliability and richness of evaluation data and thus quality of evaluation standards, researchers have developed a variety of various tools and methods. However, the mere accessibility of tools and methods does not necessarily improve evaluation quality. Among others, evaluation quality also hinges on competent evaluators, who are able to handle demands and challenges associated with exercise evaluation. However, research has barely focused on competence requirements for evaluators within the context of CP exercises, even though professionalisation and professionalism of evaluators have been discussed in general evaluation research. There is a need to determine whether the requirements for evaluators identified within this broader field of study can be applied to the specific realm of CP exercise evaluation. This paper aims to bridge this gap by examining the competence requirements for evaluators within the context of European CP. The study conducts a two-wave delphi survey. It solicited experts to assess the priorities of 106 competencies for exercise evaluation. The findings suggest that competence models developed in general evaluation research are proper to describe the competence requirements for exercise evaluators within the context of CP. Considering average rating scores and indicators for consent and dissent, this study concludes on the categorisation of standard requirements, context-specific requirements, and ambiguous attributes.