Argonne National Laboratory

An Analysis of 10-Gigabit Ethernet Protocol Stacks in Multicore Environments

TitleAn Analysis of 10-Gigabit Ethernet Protocol Stacks in Multicore Environments
Publication TypeConference Paper
Year of Publication2007
AuthorsNarayanaswamy, G, Balaji, P, Feng, W
Conference Name15th Annual IEEE Symposium on High-Performance Interconnects
Conference LocationPalo Alto, CA
Other NumbersANL/MCS-P1432-0607

This paper analyzes the interactions between the protocol stack (TCP/IP or iWARP over 10-Gigabit Ethernet) and its multicore environment. Specifically, for host-based protocols such as TCP/IP, we notice that a significant amount of processing is statically assigned to a single core, resulting in an imbalance of load on the different cores of the system and adversely impacting the performance of many applications. For host-offloaded protocols such as iWARP, on the other hand, the portions of the communication stack that are performed on the host, such as buffering of messages and memory copies, are closely tied with the associated process and hence do not create such load imbalances. Thus, in this paper, we demonstrate that by intelligently mapping different processes of an application to specific cores, the imbalance created by the TCP/IP protocol stack can be largely countered and application performance significantly improved. At the same time, since the load is a better balanced in host-offloaded protocols such as iWARP, such mapping does not adversely affect performance, thus keeping the mapping generic enough to be used with multiple protocol stacks.