Deep Pipelining: Efficient Pipelining of Network Function Chains with Coroutines

Network Functions Virtualization emphasizes the importance of performance in software-based network functions and in-network computing on general-purpose computers. Efficient resource utilization is challenging to run multiple network functions at high performance. This paper proposes a generalized performance optimization technique for chains of network functions on commodity CPUs, named deep pipelining. The key idea of deep pipelining is the combination of software pipelining and coroutines to reduce the overhead of high latency instructions such as memory access of network algorithms. Coroutines are used to schedule the execution of multiple pipelines of chained network functions. Microbenchmark and synthetic benchmark results shed light on both the effectiveness and the small overhead of deep pipelining. By taking the advantage of deep pipelining, we prove the effectiveness in chaining network functions with software-based layer-3 switch implementation consisting of Ethernet switching and IP routing on our operating system. The evaluation demonstrates that deep pipelining improves packet switching and routing performance by 0.11-78% compared to the existing batching-with-prefetch approach.