Selecting the Best Simulated System: Thinking Differently About an Old Problem

The methods known collectively as “ranking & selection” have been a theoretical and practical success story for the optimization of simulated stochastic systems: they are widely used in practice, have been implemented in commercial simulation software, and research has made them more and more statistically efficient. However, “statistically efficient” has meant minimizing the number of simulation-generated observations required to make a selection, or maximizing the strength of the inference given a budget of observations. Exploiting high-performance computing, and specifically the capability to simulate many feasible solutions in parallel, has challenged the ranking & selection paradigm. In this paper we review the challenge and suggest an entirely different approach.