Argonne National Laboratory

Memory Compression Techniques for Network Address Management in MPI

TitleMemory Compression Techniques for Network Address Management in MPI
Publication TypeReport
Year of Publication2016
AuthorsGuo, Y, Archer, CJ, Blocksome, M, Parker, S, Bland, W, Raffenetti, K, Balaji, P

MPI allows applications to treat processes as a logical collection of integer ranks for each MPI communicator, while internally translating these logical ranks into actual network addresses. In current MPI implementations the management and lookup of such network addresses uses memory sizes that are proportional to the number of processes in each communicator. In this paper, we propose a new mechanism, called AV-Rankmap, for managing such logical addressing. AV-Rankmap takes advantage of logical patterns in rank-address mapping that most applications naturally tend to have, and it exploits the fact that some parts of network address structures are naturally more performance critical than others. It uses this information to compress the memory used for network address management. We demonstrate that AV-Rankmap can achieve similar or better performance compared with that of other MPI implementations while using significantly less memory.