Scholarlink

<p>The rapid growth of data-driven biomedical research has increased pressure on shared compute infrastructure, where a persistent challenge is the widespread over-allocation of resources. This practice increases queue wait times, costs, and carbon emissions. This study investigates whether job resource requests can be automatically right-sized and efficiently scheduled at scale within a science gateway. We developed Total Perspective Vortex (TPV), a software library integrated into the Galaxy platform. TPV introduces (i) dynamic adjustment of CPU, memory, and GPU requirements for individual jobs, (ii) meta-scheduling across heterogeneous resources using configurable rules, and (iii) a community-curated database of default resource requirements for nearly 1,000 bioinformatics tools. Deployments across Galaxy Australia, Europe, and the United States provided quantitative and qualitative evidence of performance and usability. TPV has scheduled tens of millions of jobs in production, with scheduling decisions executed in milliseconds. At Galaxy Australia, TPV reduced average queue wait times by 15% despite a 27% increase in workload. Administrators reported major time savings due to the shared database and simplified configuration, while retaining flexibility for local customization. TPV improves efficiency and sustainability of scientific gateways by reducing waste, shortening job turnaround times, and lowering administrative burden. While developed for Galaxy, the approach generalizes to other workflow systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Right-Sizing Compute Resource Allocations for Bioinformatics Tools with Total Perspective Vortex

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月13日

查看全文
  • Nuwan Goonasekera,
  • Catherine Bromhead,
  • Simon Gladman,
  • Keith Suderman,
  • Nathan Coraor,
  • Björn Grüning,
  • Enis Afgan

摘要

The rapid growth of data-driven biomedical research has increased pressure on shared compute infrastructure, where a persistent challenge is the widespread over-allocation of resources. This practice increases queue wait times, costs, and carbon emissions. This study investigates whether job resource requests can be automatically right-sized and efficiently scheduled at scale within a science gateway. We developed Total Perspective Vortex (TPV), a software library integrated into the Galaxy platform. TPV introduces (i) dynamic adjustment of CPU, memory, and GPU requirements for individual jobs, (ii) meta-scheduling across heterogeneous resources using configurable rules, and (iii) a community-curated database of default resource requirements for nearly 1,000 bioinformatics tools. Deployments across Galaxy Australia, Europe, and the United States provided quantitative and qualitative evidence of performance and usability. TPV has scheduled tens of millions of jobs in production, with scheduling decisions executed in milliseconds. At Galaxy Australia, TPV reduced average queue wait times by 15% despite a 27% increase in workload. Administrators reported major time savings due to the shared database and simplified configuration, while retaining flexibility for local customization. TPV improves efficiency and sustainability of scientific gateways by reducing waste, shortening job turnaround times, and lowering administrative burden. While developed for Galaxy, the approach generalizes to other workflow systems.