J. R. Hammond, J. Dinan, P. Balaji, I. Kabadshow, S. Potluri, and V. Tipparaju, "OSPRI: An Optimized One-Sided Communication Runtime for Leadership-Class Machines," The 6th Conference on Partitioned Global Address Space Programming Models, Santa Barbara, CA, 2011, . Also Preprint ANL/MCS-P1880-0411, April 2011. [pdf]
Partitioned Global Address Space (PGAS) programming models provide a convenient approach to implementing complex scientific applications by providing access to a large, globally accessible address space. Global Arrays (GA) is a popular PGAS model that is focused on providing an efficient, productive interface to distributed shared global arrays and is used by several important scientific computing applications including the NWChem computational chemistry suite. While the communication runtime of GA (named ARMCI) has been optimized for several platforms, its architecture was fundamentally designed for general purpose cluster computing systems with full-fledged Operating Systems. In the recent past the largest systems in the world have been increasingly moving towards custom lightweight Operating Systems that are more tightly coupled with the hardware architecture and its usage environment. For such platforms, however, communication runtime architectures such as ARMCI might not map optimally. In this work, we describe a new communication runtime for PGAS models such as GA, termed OSPRI (One-Sided PRImitives). OSPRI presents several changes in architecture from conventional one-sided communication systems that make it better suited for emerging leadership class machines. We describe the implementation of the the IBM Blue Gene/P target for OSPRI and demonstrate significant improvements in latency, bandwidth, and scalability over well tuned ARMCI and GA implementations on this system.