Darius Buntinas, Guillaume Mercier, and William Gropp, "Data Transfers Between Processes in an SMP System: Performance Study and Application to MPI," Preprint ANL/MCS-P1306-1105, November 2005. [pdf]
This paper focuses on the transfer of large data in SMP systems. Achieving good performance for intranode communication is critical for developing an efficient communication system, especially in the context of SMP clusters. We evaluate the performance of five transfer mechanisms: shared-memory buffers, message queues, the Ptrace system call, kernel module-based copy, and a high-speed network. We evaluate
each mechanism based on latency, bandwidth, its impact on application cache usage, and its suitability to support MPI two-sided and one-sided messages.