Towards Developing an Open-Infrastructure for Assessing the Progress, Success, and Impacts of CyberInfrastructure Projects

Metrics for objectively assessing the CyberInfrastructure (CI) projects are important for not only obtaining the required resources for the long-term sustainability of the projects and community engagement but also for identifying issues and prioritizing the areas of improvement. This paper presents an overview of a model named MICI for assessing the progress, success, and impacts of a diverse range of CI projects. As an initial step, we developed a taxonomy of the CI projects and defined a set of standard metrics for the assessment of the projects belonging to the different categories in the taxonomy. If all the projects in the same category adopt a standard set of metrics for their assessment in addition to their project-specific and general metrics for assessment, it can help in focusing on gathering the assessment related data uniformly and understanding the short-term and long-term impacts created by the investments and the projects. We also introduce the MICI model for measuring the impacts of CI projects. Models like MICI along with the periodic peer-review can be useful in standardizing the process of assessing the impacts of the CI projects while ensuring accountability and transparency. With community engagement, the taxonomy of the CI projects, the associated metrics, and the impacts model should be refined further for ensuring their continued relevance and usefulness. Additionally, quantitative metrics should be used to supplement human judgment in the evaluation process and not to replace it.