Workshops Archive

Automated Task Placement for Hybrid HPC-Serverless Workflows


Workshop: The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO)

Authors: Vijay Thurimella (HPE Labs), Anderson Andrei Da Silva (Inria France), Gourav Rattihalli (HPE Labs), Tainã Coleman (San Diego Supercomputing Center), Rafael Ferreira Da Silva (Oak Ridge National Laboratory), and Dejan Milojicic (HPE Labs)

Abstract: Serverless computing offers HPC workflows elastic capacity, but only some tasks fit its runtime and memory limits, and finding them has required manual placement. We separate two questions that manual placement conflates: deployment-independent candidacy, which combines platform limits with task-level heuristics and is settled before execution, and deployment-dependent placement, which turns on how much workflow parallelism the local allocation absorbs. Our framework combines static DAG analysis with one bounded profiling run per task type to assign each a candidacy class, then splits candidates across local and serverless backends without per-task input. In a single-run campaign on Epigenomics, Seismology, and Cycles, hybrid execution reaches the lowest observed makespan of three strategies, but the margin tracks how capacity-constrained each workflow is: 65.0% on Seismology against 4.9% and 9.0%. Withholding the task types the analysis rejects lowers observed makespan a further 9.8–39.9%, and dispatch costs a median of 221 µs.


Back to The 1st International Workshop on Edge-Cloud-HPC Operational Continuum (ECHO) Archive Listing

Back to Full Workshop Archive Listing