NERWEB: User-Friendly Software for Caption Quality Assessment Using the NER Model

Canadian regulations require broadcasters to report the quality of Closed Captioning (CC) by using the Number-Edition-Recognition (NER) model, which requires certified human evaluators to assess CC in English programming. However, current methods to perform these evaluations are inefficient and lack a dedicated tool. This paper explores the development of NERWEB, a dedicated software tool designed to improve the efficiency and user experience of CC quality evaluation under the Canadian NER model. Employing a User-Centered Design approach, we conducted interviews and user studies with five certified evaluators. The resulting prototype offers a more integrated, efficient, and user-friendly interface compared to traditional methods. We present a set of design recommendations for developing a software interface dedicated to performing the NER quality evaluation.