Search

Models are often formulated to aid in decision-making. However, the details included or excluded are often determined with minimal examination of the effects. There is a tendency to make models more complex than merited. Yet, decision makers’ risk preferences are often ignored without considering the effect on recommendations. Additionally, modelers do not always understand the difference between preferences toward deterministic outcomes and risk preferences or the impact of modeling conflicting risk and deterministic preferences together in a single component. In this paper, I investigate the relationship between complexity and accuracy, using COVID-19 forecasting as a case study. I find our simple model is comparable in accuracy to highly publicized models, generating among the best-calibrated forecasts. This may be surprising, given the complexity of many high-profile models supported by large teams. However, it is consistent with research suggesting simple models perform very well in a variety of settings. Although utility functions are a fundamental component of decision analysis, they can assume many forms. For small decisions, the choice might not change the decision. But it can greatly affect recommendations for large decisions. There are qualitative recommendations on which functional form to use. But there is no quantitative recommendation relating size of uncertainties to choice of utility function. By maximizing error in certain equivalents when using different utility functions, this paper provides guidance into when to use different utility functions. Although decision makers should be approximately risk neutral for small problems, they are often “risk averse.” Rabin and Thaler showed utility functions modeling small-scale risk aversion result in absurd risk aversion for large uncertainties. They explain small-scale risk aversion is due to loss aversion, where pain from losses exceeds benefit from gains. But preferences for deterministic losses and gains is a deterministic preference and is not equivalent to risk preference. They argue loss aversion caused the observed behavior, yet modelled deterministic and risk preferences in a single factor. In this paper, I show modeling risk and deterministic preference separately can resolve Rabin’s Paradox, underscoring the need to explicitly model both when deterministic preferences can influence decision making or conflict with risk preferences.

The Department of Defense (DoD) and Engineered Resilient Systems (ERS) community seek to improve decision making in the Analysis of Alternatives (AoA) process by incorporating resilience and leveraging the capabilities of model-based engineering (MBE) early in the design process. Traditional tradespace exploration utilizing Point-Based Design (PBD) often converges quickly on a solution with subsequent engineering changes to modify the design. However, this process can lead to a suboptimal solution if an incorrect initial solution is chosen. Enabled by MBE, Set-Based Design (SBD) considers sets of all possible solutions and enables down-selecting possibilities to converge on a final solution. Using a US Army Armament Research, Development, and Engineering Center case study and an open source Excel® add-in called SIPmath, this research develops an integrated MBE case study demonstration that simultaneously generates numerous designs using physics models into the value and cost tradespace allowing for tradespace exploration and SBD. In addition, this research explores incorporating resilience quantification and uncertainty into SBD.