Optimizing HDF5

NERSC staff partnered with the Hierarchical Data Format (HDF) Group to optimize HDF5 performance.

One of the most commonly used I/O packages, HDF5 plays a critical role in many simulations, including climate model runs for the Intergovernmental Panel on Climate Change. NERSC staff profiled & optimized HDF5 for Lustre and helped Cray optimize their MPI-IO, achieving a 10X improvement. This will be of immediate benefit to many NERSC applications and part of HDF 1.8.6 release.