Workshops Archive

Dynamic and Static Workflow Orchestration for Large-Scale Uncertainty Quantification Campaigns


Workshop: 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26)

Authors: Karol Zajac (Sano Centre for Computational Medicine), Levente Sandor (Budapest University of Technology and Economics), Piotr Nowakowski (Sano Centre for Computational Medicine), Benjamin Csippa (Budapest University of Technology and Economics), Gabor Zavodszky (University of Amsterdam), and Maciej Malawski (Sano Centre for Computational Medicine)

Abstract: Uncertainty Quantification (UQ) is an essential component of computational science workflows. As the size of UQ campaigns continues to grow, efficiently orchestrating large ensembles of parallel simulations on High Performance Computing (HPC) systems becomes increasingly challenging. While workflow management frameworks simplify campaign execution, the choice of an orchestration strategy can significantly influence resource utilization, execution efficiency, and overall time to solution.

This paper evaluates two paradigms for large-scale campaigns on a parallel hemodynamics application use-case. The first approach employs dynamic task scheduling using EasyVVUQ and Dask Distributed, where simulations are executed through a pool of dynamically managed workers. The second approach relies on scheduler-native job arrays, where simulation instances are submitted as independent batch jobs.

The methodology combines strong scaling experiments with UQ campaigns executed under both orchestration strategies. The results provide practical guidance for selecting workflow execution strategies and highlight the trade-offs between dynamic and static workflow orchestration models.


Back to 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26) Archive Listing

Back to Full Workshop Archive Listing