K. Yoshii, K. Iskra, H. Naik, P. Beckman, and P. C. Broekema, "Performance and Scalability Evaluation of Big Memory," Preprint ANL/MCS-P1690-1109, November 2009. [pdf]
We address memory performance issues observed in Blue Gene Linux, and discuss the design and implementation of "Big Memory”—an alternative, transparent memory space introduced to eliminate the memory performance issues. We evaluate the performance of Big Memory using custom memory benchmarks, NAS Parallel Benchmarks, and the Parallel Ocean Program, at a scale of up to 4096 nodes. We find that Big Memory is successful in resolving performance issues we had encountered. For the ocean simulation program, we even find that Linux provides better scalability than does the lightweight compute node kernel designed solely for high-performance applications. Originally intended exclusively for compute node tasks, our new memory subsystem turns out to dramatically improve the performance of certain I/O node applications as well. We demonstrate this performance using the central processor of the LOw Frequency ARray radio telescope as an example.