Search

We present a novel framework for assessing the fit of cogni-tive models. Using this framework, we highlight limitationsof existing methods of model evaluation, and derive new ap-proaches to validating cognitive models. Tests of topographi-cal consistency emphasize how a model’s structure constrainsbehavior on pairs of coupled stimuli, even when point predic-tions on individual stimuli depend on estimates of the model’sfree parameters. By carefully selecting these coupled stim-uli such that they follow the distinct topography of the model,researchers can overcome some limitations of existing meth-ods. Finally, we provide a proof-of-concept example of how touse our approach to assess a model of multi-alternative, multi-attribute choice.

We explore structural issues with parameter estimation fornon-linear cognitive models: Some parameter values are eas-ier to recover than others, and the recoverability of differentparameters interacts in systematic ways. We propose methodsfor researchers to anticipate and visualize and these issues, andthe systematic ways they differ across experimental designs.Our approach consists of assessing how changes in parame-ter values translate into changes in behavioral predictions, anddevelop measurements of the relative responsiveness of predic-tions to parameter values. We demonstrate application of ourapproach to cumulative prospect theory (CPT), a widely-usedmodel of risky decision-making.

Effective problem solving requires both exploration and ex-ploitation. We analyze data from a group problem-solving taskto gain insight into how people use information from past expe-riences and from others to achieve explore-exploit trade-offs incomplex environments. The behavior we observe is consistentwith the use of simple, reinforcement-based heuristics. Partic-ipants increase exploration immediately after experiencing alow payoff, and decrease exploration immediately after expe-riencing a high or improved payoff. We suggest that whetheran outcome is perceived as “high” or “low” is a dynamic func-tion of the outcome information available to participants. Thedegree to which the distribution of observed information re-flects the true range of possible outcomes plays an importantrole in determining whether or not this heuristic is adaptive ina given environment.