Workshops Archive

How far can network congestion awareness improve scientific workflow scheduling?


Workshop: 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26)

Authors: Luis Antonio Momm Duarte and Guilherme Piêgas Koslovski (Santa Catarina State University)

Abstract: Scientific workflows execute on shared HPC infrastructures, where network congestion remains a persistent phenomenon. Congestion arises from communication contention among tasks within the same workflow or across concurrent workflows, often increasing makespan. We show that the impact of congestion can be mitigated by delaying task execution; however, only when two conditions are satisfied: (i) a shared communication bottleneck exists, and (ii) there is available slack outside the workflow critical path. We propose the Valley-Filling Admission (VFA) mechanism, which holds off-critical-path transfers out of the congestion crest and releases them into the idle-bandwidth valley. Using realistic network telemetry, VFA reduces workflow makespan by up to 9.2% for SoyKB and 11.2% for the 771 GB Chip-Seq workflows. We evaluated VFA with simulations across three network topologies, driving eight workflows and a real execution trace against a congestion-free baseline, and systematically sweeping the slack margin, contention factor, workflow concurrency, and telemetry staleness.


Back to 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26) Archive Listing

Back to Full Workshop Archive Listing