Second-order Møller-Plesset theory (MP2) notoriously breaks down for π-driven dispersion interactions and dative bonds in transition metal complexes. Herein, we investigate three physically justified forms of single-parameter, energy-gap dependent regularization which can yield high and transferable accuracy for a variety of noncovalent interactions (including S22, S66, and L7 test sets) and (mostly closed shell) transition metal thermochemistry. Regularization serves to damp overestimated pairwise additive contributions, renormalizing first-order amplitudes such that the effects of higher-order correlations are incorporated. The optimal parameter values for the noncovalent and transition metal sets are 1.1, 0.7, and 0.4 for κ, σ, and σ2 regularizers, respectively. However, such regularization slightly degrades the accuracy of conventional MP2 for some small-molecule test sets, most of which have relatively large average frontier energy gaps. Our results suggest that appropriately regularized MP2 models may improve double hybrid density functionals, at no additional cost over conventional MP2.