Tuning Performance of 40Gbps NIC cards in Linux and VMWare

Abstract

Even if the network cards are capable of supporting 40Gbps, processing capabilities of CPU can be a bottleneck. CPU runs the software component which processes the TCP/IP over Ethernet. At this speed, millions of packets are transmitted/received per second, CPU will always be loaded because of various reasons, for instance: (1) Huge amount of TCP/IP processing (2) Adding/removing various protocol headers (3) Retransmissions (4) Handling out-of sequence order packets, interrupts and memory copies; and will degrade the performance. There are conventional solutions to this problem; System administrator can upgrade the Servers with more CPU cores, increase the existing RAM size or admin can systems with bigger L2, L3 caches to enhance the performance. However these solutions are not proper because of various reasons: They are not cost effective; no management will allow a higher budget for new servers with richer hardware resources like CPU, RAM, caches etc. Moreover existing infrastructure will be discarded and will create more e-waste hence load on the environment. Tuning for these speeds of NICs is required in OS to mitigate this problem and efficiently use the bandwidth of a NIC. Moreover, existing networking software stack should not be broken during migration from 1 to 40 Gbps. This thesis is focused on techniques for tuning high speed network cards in Linux and VMWare because Linux is a robust open source server platform which can support wide range of systems. VMWare on the other side is a futuristic commercial hypervisor a basis for all cloud computing infrastructure in coming years. Their default design is not tuned for 40 Gbps NICs. Various combinations of NIC HW accelerations are used in tandem to achieve the right performance for a given use case. This doesn’t require any additional hardware so no extra capital budget is required. Various empirical studies have been done to find out impact of different NIC HW features on performance. Based on the insight into overall networking system, combinations of various parameters which can be tuned and implementations of various features in L2 have been shown.