This summary paper presents the outcomes of the RSEHPC workshop at ISC25, which convened experts from the Research Software Engineering and High-Performance Computing communities to examine the integration of Continuous Integration and Continuous Benchmarking (CI/CB) throughout the HPC system life cycle. Through keynote and lightning talks as well as through lively discussions, participants explored the use of CI/CB from system design and procurement to deployment and production use. The paper highlights three case studies: KIT’s CI/CB services in production for flexible HPC benchmarking; JSC’s deployment of exaCB for system-wide benchmarking during the JUPITER exascale system preparations; and RIKEN R-CCS’s integration of CI/CB in the early design phase of the FugakuNEXT project. User perspectives from software developers of projects like OpenFOAM, PIConGPU, and libNEGF underscore both the growing importance and the practical challenges of implementing CI/CB in diverse and complex HPC environments. A recurring theme is the tension between the need for customized solutions and the drawbacks of fragmentation, which hinder reproducibility, portability, and collaboration. The paper concludes with a call to action for the HPC community to establish common standards for CI/CB workflows and data formats. By making benchmarking practices more FAIR (Findable, Accessible, Interoperable, Reproducible), the community can enhance software sustainability and operational excellence in HPC ecosystems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

RSEHPC@ISC25: Tools and Techniques for Continuous Integration and Benchmarking

  • René Caspart,
  • Robert Speck,
  • Michele Mesiti

摘要

This summary paper presents the outcomes of the RSEHPC workshop at ISC25, which convened experts from the Research Software Engineering and High-Performance Computing communities to examine the integration of Continuous Integration and Continuous Benchmarking (CI/CB) throughout the HPC system life cycle. Through keynote and lightning talks as well as through lively discussions, participants explored the use of CI/CB from system design and procurement to deployment and production use. The paper highlights three case studies: KIT’s CI/CB services in production for flexible HPC benchmarking; JSC’s deployment of exaCB for system-wide benchmarking during the JUPITER exascale system preparations; and RIKEN R-CCS’s integration of CI/CB in the early design phase of the FugakuNEXT project. User perspectives from software developers of projects like OpenFOAM, PIConGPU, and libNEGF underscore both the growing importance and the practical challenges of implementing CI/CB in diverse and complex HPC environments. A recurring theme is the tension between the need for customized solutions and the drawbacks of fragmentation, which hinder reproducibility, portability, and collaboration. The paper concludes with a call to action for the HPC community to establish common standards for CI/CB workflows and data formats. By making benchmarking practices more FAIR (Findable, Accessible, Interoperable, Reproducible), the community can enhance software sustainability and operational excellence in HPC ecosystems.