C-MPI: A DHT Implementation for Grid and HPC Environments

TitleC-MPI: A DHT Implementation for Grid and HPC Environments
Publication TypeConference Paper
Year of Publication2010
AuthorsWozniak, JM, Jacobs, B, Latham, R, Lang, S, Son, SW, Ross, RB
Date Published04/2010
Other NumbersANL/MCS-P1746-0410

We describe a new implementation of a distributed hash table for use as a distributed data service for grid and high performance computing. The distributed data structure can offer an alternative to existing checkpointing, caching, and communication strategies due to its inherent survivability and scalability. The effective use of such an implementation in a high performance setting faces many challenges, including maintaining good performance, offering wide compatibility with diverse architectures, and handling multiple fault modes. The implementation described here, called Content-MPI (C-MPI), employs a layered software design built on MPI functionality, and offers a scalable data store that is fault tolerant to the extent of the capability of the MPI implementation.