Search

Abstract The orbitofrontal cortex (OFC) and hippocampus (HC) both contribute to the cognitive maps that support flexible behavior. Previously, we used the dopamine neurons to measure the functional role of OFC. We recorded midbrain dopamine neurons as rats performed an odor-based choice task, in which expected rewards were manipulated across blocks. We found that ipsilateral OFC lesions degraded dopaminergic prediction errors, consistent with reduced resolution of the task states. Here we have repeated this experiment in male rats with ipsilateral HC lesions. The results show HC also shapes the task states, however unlike OFC, which provides information local to the trial, the HC is necessary for estimating upper-level hidden states that distinguish blocks. The results contrast the roles of the OFC and HC in cognitive mapping and suggest that the dopamine neurons access rich information from distributed regions regarding the environment’s structure, potentially enabling this teaching signal to support complex behaviors.

Abstract Outcome-guided behavior requires knowledge about the identity of future rewards. Previous work across species has shown that the dopaminergic midbrain responds to violations in expected reward identity and that the lateral orbitofrontal cortex (OFC) represents reward identity expectations. Here we used network-targeted transcranial magnetic stimulation (TMS) and functional magnetic resonance imaging (fMRI) during a trans-reinforcer reversal learning task to test the hypothesis that outcome expectations in the lateral OFC contribute to the computation of identity prediction errors (iPE) in the midbrain. Network-targeted TMS aiming at lateral OFC reduced the global connectedness of the lateral OFC and impaired reward identity learning in the first block of trials. Critically, TMS disrupted neural representations of expected reward identity in the OFC and modulated iPE responses in the midbrain. These results support the idea that iPE signals in the dopaminergic midbrain are computed based on outcome expectations represented in the lateral OFC.

Background: Behavioral activation is an evidence-based treatment for depression. Theoretical considerations suggest that treatment response depends on reinforcement learning mechanisms. However, which reinforcement learning mechanisms are engaged by and mediate the therapeutic effect of behavioral activation remains only partially understood, and there are no procedures to measure such mechanisms. Objective: To perform a pilot study to examine whether reinforcement learning processes measured through tasks or self-report are related to treatment response to behavioral activation. Method: The pilot study enrolled 13 outpatients (12 completers) with major depressive disorder, from July of 2018 through February of 2019, into a nine-week trial with BA. Psychiatric evaluations, decision-making tests and self-reported reward experience and anticipations were acquired before, during and after the treatment. Task and self-report data were analysed by using reinforcement-learning models. Inferred parameters were related to measures of depression severity through linear mixed effects models. Results: Treatment effects during different phases of the therapy were captured by specific decision-making processes in the task. During the weeks focusing on the active pursuit of reward, treatment effects were more pronounced amongst those individuals who showed an increase in Pavlovian appetitive influence. During the weeks focusing on the avoidance of punishments, treatment responses were more pronounced in those individuals who showed an increase in Pavlovian avoidance. Self-reported anticipation of reinforcement changed according to formal RL rules. Individual differences in the extent to which learning followed RL rules related to changes in anhedonia. Conclusions: In this pilot study both task- and self-report-derived measures of reinforcement learning captured individual differences in treatment response to behavioral activation. Appetitive and aversive Pavlovian reflexive processes appeared to be modulated by separate psychotherapeutic interventions, and the modulation strength covaried with response to specific interventions. Self-reported changes in reinforcement expectations are also related to treatment response. Trial Registry Name: Set Your Goal: Engaging in GO/No-Go Active Learning, #NCT03538535, http://www.clinicaltrials.gov.

The olfactory system—and odor perception by extension—is susceptible to state-dependent influences. This review delves into human behavioral research in this area, and also touches on mechanistic evidence and examples from animal work. The review summarizes studies on the impact of satiety state on olfaction, highlighting the robust effects of food intake on the perceived pleasantness of food odors and olfactory decision-making. The impacts of other behavioral states on olfaction are also discussed. While research in this area is more limited, preliminary evidence suggests that odor perception is altered by circadian state, sleep deprivation, and mood. The flexibility in olfactory function described here can be considered adaptive, as it serves to direct behavior toward stimuli with high state-dependent value.

Theoretical accounts distinguish between motivational (‘wanting’) and hedonic (‘liking’) dimensions of rewards. Previous animal and human research linked wanting and liking to anatomically and neurochemically distinct brain mechanisms, but it remains unknown how the different brain regions and neurotransmitter systems interact in processing distinct reward dimensions. Here, we assessed how pharmacological manipulations of opioid and dopamine receptor activation modulate the neural processing of wanting and liking in humans in a randomized, placebo-controlled, double-blind clinical trial. Reducing opioid receptor activation with naltrexone selectively reduced wanting of rewards, which on a neural level was reflected by stronger coupling between dorsolateral prefrontal cortex and the striatum under naltrexone compared with placebo. In contrast, reducing dopaminergic neurotransmission with amisulpride revealed no robust effects on behavior or neural activity. Our findings thus provide insights into how opioid receptors mediate neural connectivity related to specifically motivational, not hedonic, aspects of rewards.

Growing evidence suggests that internal factors influence how we perceive the world. However, it remains unclear whether and how motivational states, such as hunger and satiety, regulate perceptual decision-making in the olfactory domain. Here, we developed a novel behavioral task involving mixtures of food and nonfood odors (i.e., cinnamon bun and cedar; pizza and pine) to assess olfactory perceptual decision-making in humans. Participants completed the task before and after eating a meal that matched one of the food odors, allowing us to compare perception of meal-matched and non-matched odors across fasted and sated states. We found that participants were less likely to perceive meal-matched, but not non-matched, odors as food dominant in the sated state. Moreover, functional magnetic resonance imaging (fMRI) data revealed neural changes that paralleled these behavioral effects. Namely, odor-evoked fMRI responses in olfactory/limbic brain regions were altered after the meal, such that neural patterns for meal-matched odor pairs were less discriminable and less food-like than their non-matched counterparts. Our findings demonstrate that olfactory perceptual decision-making is biased by motivational state in an odor-specific manner and highlight a potential brain mechanism underlying this adaptive behavior.

Responses in the dopaminergic midbrain are known to signal prediction errors for reward value. Here, the authors show that the human midbrain also encodes errors in predicted reward identity, and that these signals update expectations of reward identity in the orbitofrontal cortex.

Generous behaviour increases happiness, but the neural underpinnings of this link are unknown. Here, authors show that promising to be generous changes the neural response in the temporo-parietal junction, and that the connection between this region and the ventral striatum was related to happiness.

Internal representations of relationships between events in the external world can be utilized to infer outcomes when direct experience is lacking. This process is thought to involve the orbitofrontal cortex (OFC) and hippocampus (HPC), but there is little evidence regarding the relative role of these areas and their interactions in inference. Here, we used a sensory preconditioning task and pattern-based neuroimaging to study this question. We found that associations among value-neutral cues were acquired in both regions during preconditioning but that value-related information was only represented in the OFC at the time of the probe test. Importantly, inference was accompanied by representations of associated cues and inferred outcomes in the OFC, as well as by increased HPC-OFC connectivity. These findings suggest that the OFC and HPC represent only partially overlapping information and that interactions between the two regions support model-based inference.

Dopamine neurons respond to errors in predicting value-neutral sensory information. These data, combined with causal evidence that dopamine transients support sensory-based associative learning, suggest that the dopamine system signals a multidimensional prediction error. Yet such complexity is not evident in the activity of individual neurons or population averages. How then do downstream areas know what to learn in response to these signals? One possibility is that information about content is contained in the pattern of firing across many dopamine neurons. Consistent with this, here we show that the pattern of firing across a small group of dopamine neurons recorded in rats signals the identity of a mis-predicted sensory event. Further, this same information is reflected in the BOLD response elicited by sensory prediction errors in human midbrain. These data provide evidence that ensembles of dopamine neurons provide highly specific teaching signals, opening new possibilities for how this system might contribute to learning.

Sleep deprivation has marked effects on food intake, shifting food choices toward energy-dense options. Here we test the hypothesis that neural processing in central olfactory circuits, in tandem with the endocannabinoid system (ECS), plays a key role in mediating this relationship. We combined a partial sleep-deprivation protocol, pattern-based olfactory neuroimaging, and ad libitum food intake to test how central olfactory mechanisms alter food intake after sleep deprivation. We found that sleep restriction increased levels of the ECS compound 2-oleoylglycerol (2-OG), enhanced encoding of food odors in piriform cortex, and shifted food choices toward energy-dense food items. Importantly, the relationship between changes in 2-OG and food choices was formally mediated by odor-evoked connectivity between the piriform cortex and insula, a region involved in integrating feeding-related signals. These findings describe a potential neurobiological pathway by which state-dependent changes in the ECS may modulate chemosensory processing to regulate food choices.

The central processing pathways of the human olfactory system are not fully understood. The olfactory bulb projects directly to a number of cortical brain structures, but the distinct networks formed by projections from each of these structures to the rest of the brain have not been well-defined. Here, we used functional magnetic resonance imaging and k-means clustering to parcellate human primary olfactory cortex into clusters based on whole-brain functional connectivity patterns. Resulting clusters accurately corresponded to anterior olfactory nucleus, olfactory tubercle, and frontal and temporal piriform cortices, suggesting dissociable whole-brain networks formed by the subregions of primary olfactory cortex. This result was replicated in an independent data set. We then characterized the unique functional connectivity profiles of each subregion, producing a map of the large-scale processing pathways of the human olfactory system. These results provide insight into the functional and anatomical organization of the human olfactory system.

The firing of dopaminergic midbrain neurons is thought to reflect prediction errors (PE) that depend on the difference between the value of expected and received rewards. However, recent work has demonstrated that unexpected changes in value-neutral outcome features, such as identity, can evoke similar responses. It remains unclear whether the magnitude of these identity PEs scales with the perceptual dissimilarity of expected and received rewards, or whether they are independent of perceptual similarity. We used a Pavlovian transreinforcer reversal task to elicit identity PEs for value-matched food odor rewards, drawn from two perceptual categories (sweet, savory). Replicating previous findings, identity PEs were correlated with fMRI activity in midbrain, OFC, piriform cortex, and amygdala. However, the magnitude of identity PE responses was independent of the perceptual distance between expected and received outcomes, suggesting that identity comparisons underlying sensory PEs may occur in an abstract state space independent of straightforward sensory percepts.

Animals often need to form specific associations between perceptually similar stimuli and the different outcomes they may predict. Howard et al. find that the human brain accomplishes this via enhanced coupling between stable codes of sensory features and flexible codes of stimulus reward value.

Slow-wave sleep is an optimal opportunity for memory consolidation: when encoding occurs in the presence of a sensory cue, delivery of that cue during sleep enhances retrieval of associated memories. Recent studies suggest that cues might promote consolidation by inducing neural reinstatement of cue-associated content during sleep, but direct evidence for such mechanisms is scant, and the relevant brain areas supporting these processes are poorly understood. Here, we address these gaps by combining a novel olfactory cueing paradigm with an object-location memory task and simultaneous EEG-fMRI recording in human subjects. Using pattern analysis of fMRI ensemble activity, we find that presentation of odor cues during sleep promotes reactivation of category-level information in ventromedial prefrontal cortex that significantly correlates with post-sleep memory performance. In identifying the potential mechanisms by which odor cues selectively modulate memory in the sleeping brain, these findings bring unique insights into elucidating how and what we remember.

The value of rewards arises from multiple hedonic and motivational dimensions. Reward-encoding brain regions such as the ventral striatum (VS) are known to process these dimensions. However, the mechanism whereby distinct reward dimensions are selected for neural processing and guiding behavior remains unclear. Here, we used functional imaging to investigate how human individuals make either hedonic (liking) or motivational (wanting) evaluations of everyday items. We found that the two types of evaluations were differently modulated depending on whether participants won or lost these items. Neural activity in the VS encoded both hedonic and motivational dimensions of reward, whereas ventromedial prefrontal activity encoded primarily motivational evaluations and central orbitofrontal activity encoded predominantly hedonic evaluations. These distinct prefrontal representations arose regardless of which judgment was currently relevant for behavior. Critically, the VS preferentially processed the reward dimension currently being evaluated and showed judgment-specific functional connectivity with the dimension-specific prefrontal areas. Thus, our data are in line with a gating mechanism by which prefrontal cortex (PFC)-VS pathways flexibly encode reward dimensions depending on their behavioral relevance. These findings provide a prototype for a generalized information selection mechanism through content-tailored frontostriatal communication.

Memorable positive and negative experiences produce different profiles of gene expression in brain areas associated with long-term memory.

The ability to generalize previously learned information to novel situations is fundamental for adaptive behavior. However, too wide or too narrow generalization is linked to neuropsychiatric disorders. Previous research suggests that interactions between the dopaminergic system and the hippocampus may play a role in generalization, but whether and how the degree of generalization can be modulated via these pathways is currently unknown. Here, we addressed this question in humans using pharmacology, functional magnetic resonance imaging, and computational modeling. Blocking dopamine D2-receptors (D2R) altered generalization behavior as revealed by an increased kurtosis of the generalization gradient, and a decreased width of model-derived generalization parameters. Moreover, D2R-blockade modulated similarity-based responses in the hippocampus and decreased midbrain-hippocampal connectivity, which in turn correlated with individual differences in generalization. These results suggest that dopaminergic activity in the hippocampus may relate to the degree of generalization and highlight a potential target for treatment.