Search

Colombian drug lord Pablo Escobar, while on the run, purportedly burned two million dollars in banknotes to keep his daughter warm. A stark reminder that, in life, circumstances and goals can quickly change, forcing us to reassess and modify our values on-the-fly. Studies in decision-making and neuroeconomics have often implicitly equated value to reward, emphasising the hedonic and automatic aspect of the value computation, while overlooking its functional (concept-like) nature. Here we outline the computational and biological principles that enable the brain to compute the usefulness of an option or action by creating abstractions that flexibly adapt to changing goals. We present different algorithmic architectures, comparing ideas from artificial intelligence (AI) and cognitive neuroscience with psychological theories and, when possible, drawing parallels.

The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals – the ventromedial prefrontal cortex – prioritised and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.

Confirmation bias—the tendency to overweight information that matches prior beliefs or choices—has been shown to manifest even in simple reinforcement learning. In line with recent work, we find that participants learned significantly more from choice-confirming outcomes in a reward-learning task. What is less clear is whether asymmetric learning rates somehow benefit the learner. Here, we combine data from human participants and artificial agents to examine how confirmation-biased learning might improve performance by counteracting decisional and environmental noise. We evaluate one potential mechanism for such noise reduction: visual attention—a demonstrated driver of both value-based choice and predictive learning. Surprisingly, visual attention showed the opposite pattern to confirmation bias, as participants were most likely to fixate on “missed opportunities”, slightly dampening the effects of the confirmation bias we observed. Several million simulated experiments with artificial agents showed this bias to be a reward-maximizing strategy compared to several alternatives, but only if disconfirming feedback is not completely ignored—a condition that visual attention may help to enforce.

The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules linking simple visual features. Mixture-of-experts reinforcement learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participants’ behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals - the ventromedial prefrontal cortex – prioritized and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such coding scheme predicts a causal role for valuation: in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex as a mechanism of abstraction. Tagging the neural representation of a task’s feature with rewards evoked abstraction-based decisions. Together, these findings provide a new interpretation of value as a goal-dependent, key factor in forging abstract representations.

Uncertainty is ubiquitous in cognitive processing, which is why agents require a precise handle on how to deal with the noise inherent in their mental operations. Previous research suggests that people possess a remarkable ability to track and report uncertainty, often in the form of confidence judgments. Here, we argue that humans use uncertainty inherent in their representations of value beliefs to arbitrate between exploration and exploitation. Such uncertainty is reflected in explicit confidence judgments. Using a novel variant of a multi-armed bandit paradigm, we studied how beliefs were formed and how uncertainty in the encoding of these value beliefs (belief confidence) evolved over time. We found that people used uncertainty to arbitrate between exploration and exploitation, reflected in a higher tendency towards exploration when their confidence in their value representations was low. We furthermore found that value uncertainty can be linked to frameworks of metacognition in decision making in two ways. First, belief confidence drives decision confidence—that is people’s evaluation of their own choices. Second, individuals with higher metacognitive insight into their choices were also better at tracing the uncertainty in their environment. Together, these findings argue that such uncertainty representations play a key role in the context of cognitive control.

Artificial intelligence algorithms are capable of fantastic exploits, yet they are still grossly inefficient compared with the brain’s ability to learn from few exemplars or solve problems that have not been explicitly defined. What is the secret that the evolution of human intelligence has unlocked? Generalization is one answer, but there is more to it. The brain does not directly solve difficult problems, it is able to recast them into new and more tractable problems. Here, we propose a model whereby higher cognitive functions profoundly interact with reinforcement learning to drastically reduce the degrees of freedom of the search space, simplifying complex problems, and fostering more efficient learning.

One of the puzzles in neuroeconomics is the inconsistent pattern of brain response seen in the striatum during evaluation of losses. In some studies striatal responses appear to represent loss as a negative reward (BOLD deactivation), while in others as positive punishment (BOLD activation). We argue that these discrepancies can be explained by the existence of two fundamentally different types of loss: excitatory losses signaling the presence of substantive punishment, and inhibitory losses signaling cessation or omission of reward. We then map different theories of motivational opponency to loss related decision-making, and highlight five distinct underlying computational processes. We suggest that this excitatory–inhibitory model of loss provides a neurobiological framework for understanding reference dependence in behavioral economics.

How much we like something, whether it be a bottle of wine or a new film, is affected by the opinions of others. However, the social information that we receive can be contradictory and vary in its reliability. Here, we tested whether the brain incorporates these statistics when judging value and confidence. Participants provided value judgments about consumer goods in the presence of online reviews. We found that participants updated their initial value and confidence judgments in a Bayesian fashion, taking into account both the uncertainty of their initial beliefs and the reliability of the social information. Activity in dorsomedial prefrontal cortex tracked the degree of belief update. Analogous to how lower-level perceptual information is integrated, we found that the human brain integrates social information according to its reliability when judging value and confidence. SIGNIFICANCE STATEMENT The field of perceptual decision making has shown that the sensory system integrates different sources of information according to their respective reliability, as predicted by a Bayesian inference scheme. In this work, we hypothesized that a similar coding scheme is implemented by the human brain to process social signals and guide complex, value-based decisions. We provide experimental evidence that the human prefrontal cortex's activity is consistent with a Bayesian computation that integrates social information that differs in reliability and that this integration affects the neural representation of value and confidence.

Our personal preferences affect a broad array of social behaviors. This includes the way we learn the preferences of others, an ability that often relies on limited or ambiguous information. Here we report an egocentric influence on this type of social learning that is reflected in both performance and response times. Using computational models that combine inter-trial learning and intra-trial choice, we find transient effects of participants’ preferences on the learning process, through the influence of priors, and persistent effects on the choice process. A second experiment shows that these effects generalize to non-social learning, though participants in the social learning experiment appeared to additionally benefit by using their knowledge about the popularity of certain preferences. We further find that the domain-general egocentric influences we identify can yield performance advantages in uncertain environments., People often assume that other people share their preferences, but how exactly this bias manifests itself in learning and decision-making is unclear. Here, authors show that a person's own preferences influence learning in both social and non-social situations, and that this bias improves performance.

Every time we make a choice, we maintain an explicit representation of our confidence in that choice. Using eye-tracking and behavioural measures, the authors show that tracking decision uncertainty is helpful in guiding future behaviour.

Reward processing is linked to specific neuromodulatory systems with a dopaminergic contribution to reward learning and motivational drive being well established. Neuromodulatory influences on hedonic responses to actual receipt of reward, or punishment, referred to as experienced utility are less well characterized, although a link to the endogenous opioid system is suggested. Here, in a combined functional magnetic resonance imaging–psychopharmacological investigation, we used naloxone to block central opioid function while subjects performed a gambling task associated with rewards and losses of different magnitudes, in which the mean expected value was always zero. A graded influence of naloxone on reward outcome was evident in an attenuation of pleasure ratings for larger reward outcomes, an effect mirrored in attenuation of brain activity to increasing reward magnitude in rostral anterior cingulate cortex. A more striking effect was seen for losses such that under naloxone all levels of negative outcome were rated as more unpleasant. This hedonic effect was associated with enhanced activity in anterior insula and caudal anterior cingulate cortex, areas implicated in aversive processing. Our data indicate that a central opioid system contributes to both reward and loss processing in humans and directly modulates the hedonic experience of outcomes.

The emotional responses elicited by the way options are framed often results in lack of logical consistency in human decision making. In this study, we investigated subjects with autism spectrum disorder (ASD) using a financial task in which the monetary prospects were presented as either loss or gain. We report both behavioral evidence that ASD subjects show a reduced susceptibility to the framing effect and psycho-physiological evidence that they fail to incorporate emotional context into the decision-making process. On this basis, we suggest that this insensitivity to contextual frame, although enhancing choice consistency in ASD, may also underpin core deficits in this disorder. These data highlight both benefits and costs arising from multiple decision processes in human cognition.

Animal research suggests that the consolidation of fear and extinction memories depends on N-methyl D-aspartate (NMDA)- type glutamate receptors. Using a fear conditioning and extinction paradigm in healthy normal volunteers, we show that postlearning administration of the NMDA partial agonist D-cycloserine (DCS) facilitates fear memory consolidation, evidenced behaviorally by enhanced skin conductance responses, relative to placebo, for presentations of a conditioned stimulus (CS) at a memory test performed 72 h later. DCS also enhanced CS-evoked neural responses in a posterior hippocampus/collateral sulcus region and in the medial prefrontal cortex at test. Our data suggest a role for NMDA receptors in regulating fear memory consolidation in humans.

The ability to process stimuli that convey potential threat, under conditions of limited attentional resources, confers adaptive advantages. This study examined the neurobiology underpinnings of this capacity. Employing an attentional blink paradigm, in conjunction with functional magnetic resonance imaging (fMRI), we manipulated the salience of the second of two face target stimuli (T2), by varying emotionality. Behaviourally, fearful T2 faces were identified significantly more than neutral faces. Activity in fusiform face area (FFA) increased with correct identification of T2 faces. Enhanced activity in rostral anterior cingulate cortex (rACC) accounted for the benefit in detection of fearful stimuli reflected in a significant interaction between target valence and correct identification. Thus, under conditions of limited attention resources activation in rACC correlated with enhanced processing of emotional stimuli. We suggest that these data support a model in which a prefrontal “gate” mechanism controls conscious access of emotional information under conditions of limited attentional resources.

Humans have a tendency to overvalue their own ideas and creations. Understanding how these errors in judgement emerge is important for explaining suboptimal decisions, as when individuals and groups choose self-created alternatives over superior or equal ones. We show that such overvaluation is a reconstructive process that emerges when participants believe they have created an item, regardless of whether this belief is true or false. This overvaluation is observed both when false beliefs of self-creation are elicited (Experiment 1) or implanted (Experiment 2). Using brain imaging data we highlight the brain processes mediating an interaction between value and belief of self-creation. Specifically, following the creation manipulation there is an increased functional connectivity during valuation between the right caudate nucleus, where we show BOLD activity correlated with subjective value, and the left amygdala, where we show BOLD activity is linked to subjective belief. Our study highlights psychological and neurobiological processes through which false beliefs alter human valuation and in doing so throw light on a common source of error in judgements of value.

Recent research indicating that bilingualism is associated with enhanced executive function suggests that this enhancement may operate within a broader spectrum of cognitive abilities than previously thought (e.g., Stocco & Prat, 2014). In this study, we focus on metacognition or the ability to evaluate one’s own cognitive performance (Flavell, 1979). Over the course of two experiments, we presented young healthy adult monolinguals and bilinguals with a perceptual two-alternative-forced-choice task followed by confidence judgements. Results from both experiments indicated that bilingual participants showed a disadvantage in metacognitive efficiency, determined through the calculation of Mratio (Maniscalco & Lau, 2014). Our findings provide novel insight into the potential differences in bilingual and monolingual cognition, which may indicate a bilingual disadvantage. Results are discussed with reference to the balance of advantages versus disadvantages associated with multilanguage learning.

Rather than existing in a vacuum, decisions occur within a context of past experiences and current conditions. While standard models of economic choice assume that decisions are made independent of such factors, empirical evidence indicates that choice can be strongly context-dependent. This chapter discusses behavioral examples, theoretical models, and potential neural substrates of context-dependent choice. Decisions are subject to a number of contextual factors, including the size of the choice set, array of option attributes, and framing of the decision scenario. While a useful model for many of these effects incorporates the idea of a reference point to which outcome values are compared, the neural basis of this process is unknown. However, context-dependence is well-characterized in sensory processing, where adaptation and divisive normalization explain temporal and spatial context effects. Analogous computations in the neural circuits underlying valuation and choice offer insight into the neurobiological mechanism of context-dependent decision making.

Basic psychophysics tells us that decisions are rarely perfect: even with identical stimuli choice accuracy fluctuates and errors are often made. Metacognition allows appraisal of this uncertainty and correction of errors. For more complex value-based choices, however, metacognitive processes are poorly understood. In particular, how subjective confidence and valuation of choice options interact at the level of brain and behaviour is unknown. In this chapter, we summarise and discuss the results of a study designed to investigate this relationship. Subjects were asked to choose between pairs of snack items and subsequently provide a confidence rating in their choice. As predicted by a computational model of the decision process, confidence reflected the evolution of a decision variable over time, explaining the observed relation between confidence, value, accuracy and reaction time (RT). Furthermore, fMRI signal in human ventromedial prefrontal cortex (vmPFC) reflected both value comparison and confidence in the value comparison process. In contrast, individuals’ metacognitive ability was predicted by a measure of functional connectivity between vmPFC and rostrolateral prefrontal cortex (rlPFC), a region that responded to changes in confidence but was not involved in representing the values used to guide choice. These results provide a novel link between noise in value comparison and metacognitive awareness of choice, extending the study of metacognition to value-based decision-making.

Summary The ability to infer intentions of other agents, called theory of mind (ToM), confers strong advantages for individuals in social situations. Here, we show that ToM can also be maladaptive when people interact with complex modern institutions like financial markets. We tested participants who were investing in an experimental bubble market, a situation in which the price of an asset is much higher than its underlying fundamental value. We describe a mechanism by which social signals computed in the dorsomedial prefrontal cortex affect value computations in ventromedial prefrontal cortex, thereby increasing an individual’s propensity to ‘ride’ financial bubbles and lose money. These regions compute a financial metric that signals variations in order flow intensity, prompting inference about other traders’ intentions. Our results suggest that incorporating inferences about the intentions of others when making value judgments in a complex financial market could lead to the formation of market bubbles., Highlights • Theory of mind can be maladaptive in complex institutions like financial markets • Dorsomedial prefrontal cortex biases signals in ventromedial prefrontal cortex • Both regions compute financial metric signals • Incorporating inferences about the intentions of others could lead to market bubbles, De Martino demonstrates that the ability to infer the intentions and mental states of other individuals (“theory of mind”) biases evaluation when people interact with complex modern institutions like financial markets, contributing to the formation of economics bubbles.

Decisions are never perfect, with confidence in one's choices fluctuating over time. How subjective confidence and valuation of choice options interact at the level of brain and behavior is unknown. Using a dynamic model of the decision process, we show that confidence reflects the evolution of a decision variable over time, explaining the observed relation between confidence, value, accuracy and reaction time. As predicted by our dynamic model, we show that a functional magnetic resonance imaging signal in human ventromedial prefrontal cortex (vmPFC) reflects both value comparison and confidence in the value comparison process. Crucially, individuals varied in how they related confidence to accuracy, allowing us to show that this introspective ability is predicted by a measure of functional connectivity between vmPFC and rostrolateral prefrontal cortex. Our findings provide a mechanistic link between noise in value comparison and metacognitive awareness of choice, enabling us both to want and to express knowledge of what we want. © 2013 Nature America, Inc. All rights reserved.

Summary Employers often make payment contingent on performance in order to motivate workers. We used fMRI with a novel incentivized skill task to examine the neural processes underlying behavioral responses to performance-based pay. We found that individuals' performance increased with increasing incentives; however, very high incentive levels led to the paradoxical consequence of worse performance. Between initial incentive presentation and task execution, striatal activity rapidly switched between activation and deactivation in response to increasing incentives. Critically, decrements in performance and striatal deactivations were directly predicted by an independent measure of behavioral loss aversion. These results suggest that incentives associated with successful task performance are initially encoded as a potential gain; however, when actually performing a task, individuals encode the potential loss that would arise from failure., Highlights ► Very high monetary incentives result in poor behavioral performance ► During incentive presentation, striatal activity increases with increasing incentives ► During task performance, striatal activity decreases with increasing incentives ► Performance and striatal responses are predicted by behavioral loss aversion, Chib et al. find that very large financial incentives can lead to the paradoxical response of poor work performance. Imaging results suggest that these decreases in performance are caused by individuals encoding the potential monetary loss that would arise from failure.

Publisher Summary This chapter focuses on the neural computations underlying choice behavior in situations where the contextual information is manipulated exogenously by the experimenter. This chapter also discusses how the underlying value signals are modulated by contextual information, focusing in particular on how reference-dependent and reference-independent values are represented in the orbitofrontal cortex. The task that the algorithm needs to perform is simply to multiply these two pieces of information, calculate what is called the expected value (EV) of the gamble, and choose the gamble with higher EV. It is irrelevant to the outcome of the algorithm whether the probability is entered as a number between 0 and 1 or stated as a percentage or, similarly, whether the monetary reward is expressed in dollars or euros. Exactly how the medial orbitofrontol cortex mOFC exerts a control on behavior, enabling a less context-dependent pattern of choice, has still not been fully understood. The critical manipulation was changing the default: in one condition subjects were required to hold down a button if the ball landed inside the two lines (IN) but to switch and press another button if the ball was outside (OUT).

Losses are a possibility in many risky decisions, and organisms have evolved mechanisms to evaluate and avoid them. Laboratory and field evidence suggests that people often avoid risks with losses even when they might earn a substantially larger gain, a behavioral preference termed “loss aversion.” The cautionary brake on behavior known to rely on the amygdala is a plausible candidate mechanism for loss aversion, yet evidence for this idea has so far not been found. We studied two rare individuals with focal bilateral amygdala lesions using a series of experimental economics tasks. To measure individual sensitivity to financial losses we asked participants to play a variety of monetary gambles with possible gains and losses. Although both participants retained a normal ability to respond to changes in the gambles’ expected value and risk, they showed a dramatic reduction in loss aversion compared to matched controls. The findings suggest that the amygdala plays a key role in generating loss aversion by inhibiting actions with potentially deleterious outcomes.

Humans tend to modify their attitudes to align with past action. For example, after choosing between similarly valued alternatives, people rate the selected option as better than they originally did, and the rejected option as worse. However, it is unknown whether these modifications in evaluation reflect an underlying change in the physiological representation of a stimulus' expected hedonic value and our emotional response to it. Here, we addressed this question by combining participants' estimations of the pleasure they will derive from future events, with brain imaging data recorded while they imagined those events, both before, and after, choosing between them. Participants rated the selected alternatives as better after the decision stage relative to before, whereas discarded alternatives were valued less. Our functional magnetic resonance imaging findings reveal that postchoice changes in preference are tracked in caudate nucleus activity. Specifically, the difference in blood oxygenation level-dependent (BOLD) signal associated with the selected and rejected stimuli was enhanced after a decision was taken, reflecting the choice that had just been made. This finding suggests that the physiological representation of a stimulus' expected hedonic value is altered by a commitment to it. Furthermore, before any revaluation induced by the decision process, our data show that BOLD signal in this same region reflects the choices we are likely to make at a later time.

A key focus of current research in neuroeconomics concerns how the human brain computes value. Although, value has generally been viewed as an absolute measure (e.g., expected value, reward magnitude), much evidence suggests that value is more often computed with respect to a changing reference point, rather than in isolation. Here, we present the results of a study aimed to dissociate brain regions involved in reference-independent (i.e., “absolute”) value computations, from those involved in value computations relative to a reference point. During functional magnetic resonance imaging, subjects acted as buyers and sellers during a market exchange of lottery tickets. At a behavioral level, we demonstrate that subjects systematically accorded a higher value to objects they owned relative to those they did not, an effect that results from a shift in reference point (i.e., status quo bias or endowment effect). Our results show that activity in orbitofrontal cortex and dorsal striatum track parameters such as the expected value of lottery tickets indicating the computation of reference-independent value. In contrast, activity in ventral striatum indexed the degree to which stated prices, at a within-subjects and between-subjects level, were distorted with respect to a reference point. The findings speak to the neurobiological underpinnings of reference dependency during real market value computations.

Introduction Norepinephrine (NE) has a regulatory role in human attention. Objective To examine its role in emotional modulation of attention, we used an attentional blink (AB) paradigm, in the context of psychopharmacological manipulation, where targets were either emotional or neutral items. Results and discussion We report behavioural evidence that β-adrenergic blockade with propranolol impairs attention independent of target valence. Furthermore, this effect is centrally mediated as administration of the peripheral β-adrenergic antagonist nadolol did not impair attention. By contrast, increasing NE tone, using the selective NE reuptake inhibitor reboxetine, improves detection of emotional stimuli. Conclusion In line with theoretical and animal models, these findings provide human behavioural evidence that the adrenergic system has a modulatory influence on selective attention that in some instances depends on item valence.