Workshop: PDSW’26: The 11th International Parallel Data Systems Workshop
Authors: Lena Oden (FernUniversität in Hagen)
Abstract: GPU-accelerated scientific computing in Python needs to move data from on-disk HDF5 files into GPU memory. We present a pure-Python extension to h5py adding GPU-aware reading, writing, and reduction without modifying its C extension or the HDF5 library. It carries h5py's index-based, NumPy-style slicing unchanged from disk through host memory to GPU-resident CuPy or PyTorch arrays, addressing a selection identically regardless of the result's location. Internally, every read, write, and reduction path pins host memory once, reuses it across calls, and overlaps host-to-device transfer with the next piece's storage read, optionally fusing a caller-supplied transform or reduction into the same stream at no extra cost until the computation outweighs the transfer. Measurements on real hardware show index-addressed reads costing only for what is selected, host memory staying flat as dataset size grows over an order of magnitude, and fused compute staying hidden behind I/O until per-piece compute cost overtakes it.
Back to PDSW’26: The 11th International Parallel Data Systems Workshop Archive Listing
Back to Full Workshop Archive Listing