Parallel I/O Benchmarks

Since I am often asked if I know of any parallel I/O benchmarks that are available, I have put together a list of the ones I am aware of (in no particular order). If you know of any benchmarks/applications that can be added to this list, let me (Rajeev Thakur) know.

Updated: July 18, 2014

Benchmarks

  1. PnetCDF Benchmarks
    http://cucis.ece.northwestern.edu/projects/PnetCDF/#Benchmarks

  2. HACC I/O
    https://asc.llnl.gov/CORAL-benchmarks/#hacc

  3. FLASH I/O
    http://www.ucolick.org/~zingale/flash_benchmark_io/
    An updated version for newer versions of parallel NetCDF and HDF-5 is available
    here.

  4. IOR
    https://github.com/chaos/ior

  5. Effective I/O Bandwidth Benchmark (b_eff_io)
    http://www.hlrs.de/mpi/b_eff_io/

  6. NAS BTIO Benchmark
    http://www.nas.nasa.gov/Resources/Software/npb.html (see BT in NPB3.3-MPI)

  7. MPI Tile I/O
    http://www.mcs.anl.gov/pio-benchmark/

  8. FS_TEST
    https://github.com/fs-test/fs_test (see this as a README)

  9. Trace-Driven Scientific I/O Benchmarks
    http://www.cs.rochester.edu/research/os/iobench/

  10. HPIO and S3asim
    http://www.eecs.northwestern.edu/~aching/research_webpage/io.html

  11. Noncontig
    http://www.mcs.anl.gov/pio-benchmark/

  12. ioperf test in Silo
    https://wci.llnl.gov/codes/silo/

  13. Pattern I/O Benchmark (John Shalf, LBNL)
    pattern_io.tar.gz

  14. MADbench2
    http://crd.lbl.gov/~borrill/MADbench2/

  15. QIO
    http://usqcd.jlab.org/usqcd-docs/qio/

  16. PIO-Bench
    ftp://ftp.parl.clemson.edu/pub/pio-bench
    An article describing it is at
    http://www.clustermonkey.net/content/view/87/32/.

  17. Parkbench I/O Benchmarks
    http://www.performance.ecs.soton.ac.uk/projects.html

Applications

  1. Pagoda : Parallel Analysis of Geoscience Data
    https://svn.pnl.gov/gcrm/wiki/Pagoda

  2. Earth System Modeling Framework (ESMF)
    http://www.earthsystemmodeling.org/

  3. Nek5000
    http://nek5000.mcs.anl.gov/

  4. NekCEM
    http://wiki.mcs.anl.gov/nekcem/index.php/Main_Page

  5. PISM: Parallel Ice Sheet Model
    http://www.pism-docs.org

  6. MOAB
    http://trac.mcs.anl.gov/projects/ITAPS/wiki/MOAB

  7. PFLOTRAN
    http://www.secure-water.org/wiki/index.php?title=PFLOTRAN_Instructions_2011

  8. Cactus
    http://www.cactuscode.org/

  9. mpiBLAST
    http://www.mpiblast.org/

  10. FLASH
    http://www.flash.uchicago.edu/
    Also see FLASH I/O Benchmark above.

  11. Community Earth System Model (CESM)
    http://www.cesm.ucar.edu/

  12. Regional Ocean Modeling System (ROMS)
    https://www.myroms.org/

  13. Global Cloud Resolving Model (GCRM)
    https://svn.pnl.gov/gcrm

  14. Weather Research & Forecasting Model (WRF)
    http://www.mmm.ucar.edu/wrf/users/downloads.html

  15. CM1
    http://www.mmm.ucar.edu/people/bryan/cm1/

  16. Qbox
    http://eslab.ucdavis.edu/software/qbox/

  17. GADGET
    http://www.mpa-garching.mpg.de/gadget/

  18. MAESTRO
    NERSC web site

  19. MILC
    http://www.physics.utah.edu/~detar/milc/

  20. MPQC
    http://www.mpqc.org/

  21. NaSt3DGP
    http://wissrech.ins.uni-bonn.de/research/projects/NaSt3DGP/index.htm

  22. SIESTA
    http://www.icmab.es/siesta/

  23. CP2K
    http://cp2k.berlios.de/

  24. LAMMPS
    http://lammps.sandia.gov/

Traces

  1. Darshan
    http://www.mcs.anl.gov/research/projects/darshan/data/

  2. PLFS
    http://institutes.lanl.gov/plfs/maps/

  3. LANL
    http://institute.lanl.gov/data/tdata/

  4. Sandia
    http://www.cs.sandia.gov/Scalable_IO/SNL_Trace_Data/index.html

  5. MAMBO
    http://www.cs.umd.edu/projects/hpsl/mambo/index.html