Workshops Archive

nFlows: Enabling Efficient Data Movement in Data-Intensive Scientific Workflows on Non-Uniform Memory Access Architectures


Workshop: 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26)

Authors: Aurelio Vivas and Thomas Applencourt (Argonne National Laboratory) and Harold Castro (Universidad de los Andes)

Abstract: Data movement across the memory subsystems of modern supercomputers is a major performance bottleneck for data-intensive scientific workflows. Preserving locality and minimizing data movement are key strategies to address this, yet support in workflow scheduling remains limited. nFlows is a runtime system for studying data movement in scientific workflows and prototyping memory-topology-aware scheduling algorithms. This version targets the main memory subsystem, focusing on Non-Uniform Memory Access (NUMA) architectures, which dominate modern compute-node design. We use nFlows to execute workflows under a traditional FIFO scheduler and a NUMA-aware FIFO scheduler from prior work, collecting traces for both. Our analysis shows some workflows are negatively affected when execution spreads across NUMA nodes, and NUMA-aware scheduling mitigates this: executions reading at least 60% of their data locally rise from 27.64% to 68.33%, executions with a makespan improvement rise from 3.07% to 11.71%, and best-case makespan reduction increases from 10.22% to 44.28%.


Back to 21st Workshop on Workflows in Support of Large-Scale Science (WORKS26) Archive Listing

Back to Full Workshop Archive Listing