Workshops Archive

Stress-Testing Exascale Scientific Data Output - ADIOS2 Configuration Strategies for WarpX under Extreme Load Imbalance


Workshop: PDSW’26: The 11th International Parallel Data Systems Workshop

Authors: Greg Eisenhauer (Georgia Institute of Technology); Junmin Gu and Axel Huebl (Lawrence Berkeley National Laboratory); Scott Klasky and Norbert Podhorszki (Oak Ridge National Laboratory); Alexander Sim and Jean-Luv Vay (Lawrence Berkeley National Laboratory); and Kesheng (John) Wu (Lawrence Berkeley National Laboratory (LBNL), ESnet; Lawrence Berkeley National Laboratory (LBNL))

Abstract: Spatially imbalanced, temporally irregular simulation output challenges I/O configurations tuned for balanced workloads. This work evaluates ADIOS2 BP5 configurations through openPMD-api using WarpX on Frontier at ORNL and Perlmutter at NERSC, contrasting balanced 3D Cartesian output with boosted-frame back-transformed diagnostics (BTD). BTD combines rank-level output skew with snapshot contributions spanning multiple simulation steps. Balanced output reaches 3.64~TB/s without compression on 2,048 Frontier nodes and over 1~TB/s on 512 Perlmutter nodes; lossless Blosc2 compression achieves approximately 7~TB/s application-level effective throughput on Frontier. In the tested BTD configurations, SPAN combined with asynchronous writing provides the highest throughput on Frontier, FlattenSteps supports assembly of complete lab-frame snapshots for checkpointing without reader modification and with negligible write-time overhead and no penalty on read performance, and DataSizeBased aggregation improves throughput when subfile counts are constrained. These results identify complementary mechanisms for handling irregular output and provide configuration guidance for these workloads and systems.


Back to PDSW’26: The 11th International Parallel Data Systems Workshop Archive Listing

Back to Full Workshop Archive Listing