Mercury: Enabling Remote Procedure Call for High-Performance Computing

TitleMercury: Enabling Remote Procedure Call for High-Performance Computing
Publication TypeConference Paper
Year of Publication2013
AuthorsSoumagne, J, Kimpe, D, Zounmevo, JA, Chaarawi, M
Conference NameCluster 2013
Conference LocationIndianapolis, IN
Other NumbersANL/MCS-P4082-0613_1
AbstractRemote Procedure Call (RPC) is a technique that has been largely used by distributed services. This technique, now more and more used in the context of High-Performance Computing (HPC), allows the execution of routines to be delegated to remote nodes, which can be set aside and dedicated to specific tasks. However, existing RPC frameworks assume a sockets based network interface (usually on top of TCP/IP) which is not appropriate for HPC systems, as this API does not typically map well to the native network transport used on those systems, resulting in lower network performance. In addition, existing RPC frameworks often do not support handling large data arguments, such as those found in read or write calls. We present in this paper an asynchronous RPC interface specifically designed for use in HPC systems that allows asynchronous transfer of parameters and execution requests and direct support of large data arguments. The interface is generic to allow any function call to be shipped. Additionally, the network implementation is abstracted, allowing easy porting to future systems and efficient use of existing native transport mechanisms.