Workshops Archive

Measurement-Driven Contention-Aware, Herd-Safe Stripe Placement for Lustre


Workshop: PDSW’26: The 11th International Parallel Data Systems Workshop

Authors: Zhaobin Zhu and Sarah Neuwirth (Johannes Gutenberg University Mainz)

Abstract: Parallel file systems such as Lustre are shared by many concurrent jobs, so the write bandwidth observed by a single job fluctuates over time as other jobs place load on the same object storage targets (OSTs). Users can change how files are placed across OSTs, yet it remains unclear when doing so is beneficial. Based on measurements from three production Lustre systems, this paper finds that the contrast between busy and idle OSTs is a property of the moment rather than of the system, so the decision to tune has to be measured per job: always tuning costs 13% in the worst case, never tuning 43%.

Building on this observation, the paper presents CASTOR, a client-side tuner that operates without special privileges. It measures per-OST latency using a small write-probe, determines whether tuning is worthwhile, selects a layout with a simple model, and randomly distributes jobs across cool OSTs to prevent them from converging on the same targets. Across three file systems and six workloads, CASTOR demonstrates that an unprivileged write probe can detect contention and that randomly selecting among cool OSTs prevents concurrent tuners from colliding. Where a contrast exists, placement raises bandwidth by up to 1.85x and cuts its variance by up to 4.7x. The central lesson is that whether to tune is itself a measurement problem, and one that has to be solved before any layout is chosen.


Back to PDSW’26: The 11th International Parallel Data Systems Workshop Archive Listing

Back to Full Workshop Archive Listing