Search

Procrastination is often characterized as minimal progress initially, with a significant increase in progress shortly before the deadlines. Yet, the cognitive mechanisms underlying this intriguing dynamic feature of procrastination—the time course of progress—remain poorly understood. We investigated this through an experiment where participants worked on a self-paced, week-long online reading task consisting of numerous work units (N = 611). We proposed two models that fit each individual's time course of progress. Both models consider the time course of progress as the output of sequential decision-making: whether to work now (and, if so, how much) or later. The first, a normative model, calculates the value of making progress using the Bellman equation; the second, a roll-out model, estimates this value by simulating future work progress. We found that the rollout model fit the data much better, suggesting some evidence against people behaving rationally and some evidence for people simulating future work progress.

How do humans get better at planning? Previous work postulated that the improvement of cognitive strategies occurs through feedback-based metacognitive reinforcement learning (MCRL). However, it is not clear whether and, if so, how people can learn planning strategies without reinforcement. To answer these questions, we experimentally investigated the effect of frequency of feedback on people's ability to learn adaptive planning strategies. We found that participants receiving feedback only 25\% of the time nonetheless learned about as well as participants receiving constant feedback. Quantitative modelling of the data revealed that state-of-the-art MCRL models cannot explain this finding. However, extending these models by a mechanism generating an additional learning signal through self-evaluation of plan quality can account for people's ability to learn planning strategies without feedback. The findings of this research have implications for the design of learning environments and enabling people and machines to self-sufficiently improve their strategies in naturalistic settings.

How is it that humans can solve complex planning tasks so efficiently despite limited cognitive resources? One reason is its ability to know how to use its limited computational resources to make clever choices. We postulate that people learn this ability from trial and error (metacognitive reinforcement learning). In this work, we systematize models of the underlying learning mechanisms and enhance them with more sophisticated additional mechanisms. We fit the resulting 86 models to human data collected in previous experiments where different phenomena of metacognitive learning were demonstrated and performed Bayesian model selection. Our results suggest that a gradient ascent through the space of cognitive strategies can explain most of the observed qualitative phenomena, and is, therefore, a promising candidate for explaining the mechanism underlying metacognitive learning.

How do people learn when to allocate how much cognitive control to which task? According to the Learned Value of Control (LVOC) model, people learn to predict the value of alternative control allocations from features of a situation. This suggests that people may generalize the value of control learned in one situation to others with shared features, even when demands for control are different. This makes the intriguing prediction that what a person learned in one setting could cause them to misestimate the need for, and potentially overexert, control in another setting, even if this harms their performance. To test this prediction, we had participants perform a novel variant of the Stroop task in which, on each trial, they could choose to either name the color (more control-demanding) or read the word (more automatic). Only one of these tasks was rewarded each trial and could be predicted by one or more stimulus features (the color and/or word). Participants first learned colors and then words that predicted the rewarded task. Then, we tested how these learned feature associations transferred to novel stimuli with some overlapping features. The stimulus-task-reward associations were designed so that for certain combinations of stimuli, transfer of learned feature associations would incorrectly predict that more highly rewarded task would be color-naming, even though the actually rewarded task was word-reading and therefore did not require engaging control. Our results demonstrated that participants overexerted control for these stimuli, providing support for the feature-based learning mechanism described by the LVOC model.

People often fall victim to decision-making biases, e.g. short-sightedness, that lead to unfavorable outcomes in their lives. It is possible to overcome these biases by teaching people better decision-making strategies. Finding effective interventions is an open problem, with a key challenge being the lack of transfer to the real world. Here, we tested a new approach to improving human decision-making that leverages Artificial Intelligence to discover procedural descriptions of effective planning strategies. Our benchmark problem regarded improving far-sightedness. We found our intervention elicits transfer to a similar task in a different domain, but its effects in more naturalistic financial decisions were not statistically significant. Even though the tested intervention is on par with conventional approaches, which also struggle in far-transfer, further improvements are required to help people make better decisions in real life. We conclude that future work should focus on training decision-making in more naturalistic scenarios.

Which information is worth considering depends on how much effort it would take to acquire and process it. Fromthis perspective peoples tendency to neglect considering the long-term consequences of their actions (present bias) mightreflect that looking further into the future becomes increasingly more effortful. In this work, we introduce and validatethe use of Bayesian Inverse Reinforcement Learning (BIRL) for measuring individual differences in the subjective costsof planning. We extend the resource-rational model of human planning introduced by Callaway, Lieder, et al. (2018) byparameterizing the cost of planning. Using BIRL, we show that increased subjective cost for considering future outcomesmay be associated with both the present bias and acting without planning. Our results highlight testing the causal effectsof the cost of planning on both present bias and mental effort avoidance as a promising direction for future work.

To stay focused on their chosen tasks, people have to inhibit distractions. The underlying attention control skills canimprove through reinforcement learning, which can be accelerated by giving feedback. We applied the theory of metacog-nitive reinforcement learning to develop a training app that gives people optimal feedback on their attention control whilethey are working or studying. In an eight-day field experiment with 99 participants, we investigated the effect of this train-ing on peoples productivity, sustained attention, and self-control. Compared to a control condition without feedback, wefound that participants receiving optimal feedback learned to focus increasingly better (f = .08, p ¡ .01) and achieved higherproductivity scores (f = .19, p ¡ .01) during the training. In addition, they evaluated their productivity more accurately (r =.12, p ¡.01). However, due to asymmetric attrition problems, these findings need to be taken with a grain of salt.

Teaching clever heuristics is a promising approach to improvedecision-making. We can leverage machine learning to dis-cover clever strategies automatically. Current methods requirean accurate model of the decision problems people face inreal life. But most models are misspecified because of lim-ited information and cognitive biases. To address this prob-lem we develop strategy discovery methods that are robustto model misspecification. Robustness is achieved by model-ing model-misspecification and handling uncertainty about thereal-world according to Bayesian inference. We translate ourmethods into an intelligent tutor that automatically discoversand teaches robust planning strategies. Our robust cognitivetutor significantly improved human decision-making when themodel was so biased that conventional cognitive tutors were nolonger effective. These findings highlight that our robust strat-egy discovery methods are a significant step towards leverag-ing artificial intelligence to improve human decision-makingin the real world.

How can people learn to make better decisions and be-come more far-sighted? To make the underlying learningmechanisms more accessible to scientific inquiry, we developa computational method for measuring the time course ofexperience-dependent changes in people’s planning strategies.We validated our method on simulated and empirical data: onsimulated data its inferences were significantly more accuratethan simpler approaches, and when evaluated on human datait correctly detected the plasticity-enhancing effect of perfor-mance feedback. Having validated our method, we illustratehow it can be used to gain new insights into the time courseand nature of cognitive plasticity. Future work will leverageour method to i) reverse-engineer the learning mechanismsenabling people to acquire complex cognitive skills such asplanning and problem-solving and ii) measure individual dif-ferences in cognitive plasticity.

Although process data indicate that people often rely on sim-plifying processes when choosing between risky options, cur-rent models of heuristics cannot predict people’s choices veryaccurately. To address this apparent paradox, it has been pro-posed that people might adaptively choose from a toolboxof simple strategies. But which strategies are contained inthis toolbox? And how do people decide when to use whichdecision strategy? Here, we develop a model according towhich the decision maker selects a decision strategy for a givenchoice problem rationally from a toolbox of strategies; the con-tent of the toolbox is estimated for each individual decisionmaker. Using cross-validation on an empirical data set, we findthat this model of strategy selection from a personal adaptivetoolbox predicts people’s choices better than any single strat-egy (even when it is allowed to vary across participants) andbetter than previously proposed toolbox models. Our modelcomparisons show that both inferring the content of the tool-box and rational strategy selection are critical for accuratelypredicting people’s risky choices. Furthermore, our analysisreveals considerable individual differences in the set of strate-gies people are equipped with and how they choose amongthem; these individual differences could partly explain whysome people make better choices than others. These findingsrepresent an important step towards a complete formalizationof the notion that people select their cognitive strategies froma personal adaptive toolbox.

There are plenty of opportunities for life-long learning but peo-ple rarely seize them. Game elements are an increasingly pop-ular tool to keep students engaged in learning. But gamifica-tion only works when it is done properly. Here, we introducethe first principled approach to gamifying learning environ-ments. Our feedback mechanism rewards students’ efforts andstudy choices according to how beneficial they are in the longrun. The rewards are conveyed by game elements that we call“optimal brain points”. In our experiment, these optimal brainpoints significantly increased the proportion of participantswho attempted to learn a difficult skill, persisted through fail-ure, and succeeded to master it. Our method provides a princi-pled approach to designing incentive structures and feedbackmechanisms for both educational games and online courses.We are optimistic that this can help people overcome the moti-vational obstacles to self-directed life-long learning.

The human brain has the impressive capacity to adapt how it processes information to high-level goals. While it is known that these cognitive control skills are malleable and can be improved through training, the underlying plasticity mechanisms are not well understood. Here, we develop and evaluate a model of how people learn when to exert cognitive control, which controlled process to use, and how much effort to exert. We derive this model from a general theory according to which the function of cognitive control is to select and configure neural pathways so as to make optimal use of finite time and limited computational resources. The central idea of our Learned Value of Control model is that people use reinforcement learning to predict the value of candidate control signals of different types and intensities based on stimulus features. This model correctly predicts the learning and transfer effects underlying the adaptive control-demanding behavior observed in an experiment on visual attention and four experiments on interference control in Stroop and Flanker paradigms. Moreover, our model explained these findings significantly better than an associative learning model and a Win-Stay Lose-Shift model. Our findings elucidate how learning and experience might shape peoples ability and propensity to adaptively control their minds and behavior. We conclude by predicting under which circumstances these learning mechanisms might lead to self-control failure.

People's estimates of numerical quantities are systematically biased towards their initial guess. This anchoring bias is usually interpreted as sign of human irrationality, but it has recently been suggested that the anchoring bias instead results from people's rational use of their finite time and limited cognitive resources. If this were true, then adjustment should decrease with the relative cost of time. To test this hypothesis, we designed a new numerical estimation paradigm that controls people's knowledge and varies the cost of time and error independently while allowing people to invest as much or as little time and effort into refining their estimate as they wish. Two experiments confirmed the prediction that adjustment decreases with time cost but increases with error cost regardless of whether the anchor was self-generated or provided. These results support the hypothesis that people rationally adapt their number of adjustments to achieve a near-optimal speed-accuracy tradeoff. This suggests that the anchoring bias might be a signature of the rational use of finite time and limited cognitive resources rather than a sign of human irrationality.

Cognitive biases, such as the anchoring bias, pose a serious challenge to rational accounts of human cognition. We investigate whether rational theories can meet this challenge by taking into account the mind's bounded cognitive resources. We asked what reasoning under uncertainty would look like if people made rational use of their finite time and limited cognitive resources. To answer this question, we applied a mathematical theory of bounded rationality to the problem of numerical estimation. Our analysis led to a rational process model that can be interpreted in terms of anchoring-and-adjustment. This model provided a unifying explanation for ten anchoring phenomena including the differential effect of accuracy motivation on the bias towards provided versus self-generated anchors. Our results illustrate the potential of resource-rational analysis to provide formal theories that can unify a wide range of empirical results and reconcile the impressive capacities of the human mind with its apparently irrational cognitive biases.

People’s cognitive strategies are jointly shaped by function andcomputational constraints. Resource-rational analysis lever-ages these constraints to derive rational models of people’scognitive strategies from the assumption that people makerational use of limited cognitive resources. We present aresource-rational analysis of planning and evaluate its predic-tions in a newly developed process tracing paradigm. In Ex-periment 1, we find that a resource-rational planning strategypredicts the process by which people plan more accurately thanprevious models of planning. Furthermore, in Experiment 2,we find that it also captures how people’s planning strategiesadapt to the structure of the environment. In addition, our ap-proach allows us to quantify for the first time how close peo-ple’s planning strategies are to being resource-rational and tocharacterize in which ways they conform to and deviate fromoptimal planning.

People's decisions and judgments are disproportionately swayed by improbable but extreme eventualities, such as terrorism, that come to mind easily. This article explores whether such availability biases can be reconciled with rational information processing by taking into account the fact that decision makers value their time and have limited cognitive resources. Our analysis suggests that to make optimal use of their finite time decision makers should overrepresent the most important potential consequences relative to less important, put potentially more probable, outcomes. To evaluate this account, we derive and test a model we call utility-weighted sampling. Utility-weighted sampling estimates the expected utility of potential actions by simulating their outcomes. Critically, outcomes with more extreme utilities have a higher probability of being simulated. We demonstrate that this model can explain not only people's availability bias in judging the frequency of extreme events but also a wide range of cognitive biases in decisions from experience, decisions from description, and memory recall. (PsycINFO Database Record

How do we learn to think better, and what can we do to pro-mote such metacognitive learning? Here, we propose that cog-nitive growth proceeds through metacognitive reinforcementlearning. We apply this theory to model how people learn howfar to plan ahead and test its predictions about the speed ofmetacognitive learning in two experiments. In the first experi-ment, we find that our model can discern a reward structure thatpromotes metacognitive reinforcement learning from one thathinders it. In the second experiment, we show that our modelcan be used to design a feedback mechanism that enhancesmetacognitive reinforcement learning in an environment thathinders learning. Our results suggest that modeling metacog-nitive learning is a promising step towards promoting cognitivegrowth.

What is the optimal way to make a decision given that yourtime is limited and your cognitive resources are bounded? Toanswer this question, we formalized the bounded optimal de-cision process as the solution to a meta-level Markov deci-sion process whose actions are costly computations. We ap-proximated the optimal solution and evaluated its predictionsagainst human choice behavior in the Mouselab paradigm,which is widely used to study decision strategies. Our compu-tational method rediscovered well-known heuristic strategiesand the conditions under which they are used, as well as novelheuristics. A Mouselab experiment confirmed our model’smain predictions. These findings are a proof-of-concept thatoptimal cognitive strategies can be automatically derived as therational use of finite time and bounded cognitive resources.

Game elements like points and levels are a popular tool tonudge and engage students and customers. Yet, no theory cantell us which incentive structures work and how to design them.Here we connect the practice of gamification to the theory ofreward shaping in reinforcement learning. We leverage thisconnection to develop a method for designing effective incen-tive structures and delineating when gamification will succeedfrom when it will fail. We evaluate our method in two behav-ioral experiments. The results of the first experiment demon-strate that incentive structures designed by our method helppeople make better, less short-sighted decisions and avoid thepitfalls of less principled approaches. The results of the sec-ond experiment illustrate that such incentive structures can beeffectively implemented using game elements like points andbadges. These results suggest that our method provides a prin-cipled way to leverage gamification to help people make betterdecisions.

The human mind appears to be equipped with a toolbox fullof cognitive strategies, but how do people decide when to usewhich strategy? We leverage rational metareasoning to derivea rational solution to this problem and apply it to decision makingunder uncertainty. The resulting theory reconciles the twopoles of the debate about human rationality by proposing thatpeople gradually learn to make rational use of fallible heuristics.We evaluate this theory against empirical data and existingaccounts of strategy selection (i.e. SSL and RELACS). Ourresults suggest that while SSL and RELACS can explain people’sability to adapt to homogeneous environments in whichall decision problems are of the same type, rational metareasoningcan additionally explain people’s ability to adapt to heterogeneousenvironments and flexibly switch strategies fromone decision to the next.

Adults and children rely heavily on other people’s testimony.However, domains of knowledge where there is no consensuson the truth are likely to result in conflicting testimonies. Previousresearch has demonstrated that in these cases, learnerslook towards the majority opinion to make decisions. However,it remains unclear how learners evaluate social information,given that considering either the overall valence, or thenumber of testimonies, or both may lead to different conclusions.We therefore formalized several social learning strategiesand compared them to the performance of adults and children.We find that children use different strategies than adults.This suggests that the development of social learning may involvethe acquisition of cognitive strategies.