An investigation of stage-based parallel programming model for stateful network services on COTS platforms

Stateful network services are a growing population of applications in the Internet, whose architecture relies more and more on COTS components. A good parallel programming model is the key to the success of network services. We argue that stage-based parallel programming model is the most suitable candidate, which takes the whole service logic as a set of stages interconnected by event queues, and in each stage, one or multiple threads reside to service messages. And we also reveal several key design principles for this model. 1) The number of stages should not be over 10 for a single CPU system. 2) For stages with blocking calls, a good estimation of the number of threads for that stage can be gotten by (call arrival rate blocking time) and moreover excessive threads give minor performance degradation. 3) To deliver message quickly, it is much better to locate this part of service in a separate nonblocking stage.