Search

Making predictions about the rewards associated with environmental stimuli and updating those predictions through feedback is an essential aspect of adaptive behavior. Theorists have argued that dopamine encodes a reward prediction error (RPE) signal that is used in such a reinforcement learning process. Recent work with fMRI has demonstrated that the BOLD signal in dopaminergic target areas meets both necessary and sufficient conditions of an axiomatic model of the RPE hypothesis. However, there has been no direct evidence that dopamine release itself also meets necessary and sufficient criteria for encoding an RPE signal. Further, the fact that dopamine neurons have low tonic firing rates that yield a limited dynamic range for encoding negative RPEs has led to significant debate about whether positive and negative prediction errors are encoded on a similar scale. To address both of these issues, we used fast-scan cyclic voltammetry to measure reward-evoked dopamine release at carbon fiber electrodes chronically implanted in the nucleus accumbens core of rats trained on a probabilistic decision-making task. We demonstrate that dopamine concentrations transmit a bidirectional RPE signal with symmetrical encoding of positive and negative RPEs. Our findings strengthen the case that changes in dopamine concentration alone are sufficient to encode the full range of RPEs necessary for reinforcement learning.

The dorsal striatum has been implicated in reward-based decision making, but the role played by specific striatal circuits in these processes is essentially unknown. Using cell phenotype-specific viral vectors to express engineered G-protein-coupled DREADD (designer receptors exclusively activated by designer drugs) receptors, we enhanced Gi/o- or Gs-protein-mediated signaling selectively in direct-pathway (striatonigral) neurons of the dorsomedial striatum in Long–Evans rats during discrete periods of training of a high versus low reward-discrimination task. Surprisingly, these perturbations had no impact on reward preference, task performance, or improvement of performance during training. However, we found that transiently increasing Gi/o signaling during training significantly impaired the retention of task strategies used to maximize reward obtainment during subsequent preference testing, whereas increasing Gs signaling produced the opposite effect and significantly enhanced the encoding of a high-reward preference in this decision-making task. Thus, the fact that the endurance of this improved performance was significantly altered over time—long after these neurons were manipulated—indicates that it is under bidirectional control of canonical G-protein-mediated signaling in striatonigral neurons during training. These data demonstrate that cAMP-dependent signaling in direct-pathway neurons play a well-defined role in reward-related behavior; that is, they modulate the plasticity required for the retention of task-specific information that is used to improve performance on future renditions of the task.

Repeated stress releases dynorphins and causes subsequent activation of κ-opioid receptors (KORs) in limbic brain regions. The serotonergic dorsal raphe nucleus (DRN) has previously been found to be an important site of action for the dysphoric effects of dynorphin-κ-opioid receptor system activation during stress-evoked behaviors, and KOR-induced activation of p38α mitogen-activated protein kinase (MAPK) in serotonergic neurons was found to be a critical mediator of the aversive properties of stress. Yet, how dynorphins and KORs functionally regulate the excitability of serotonergic DRN neurons both in adaptive and pathological stress states is poorly understood. Here we report that acute KOR activation by the selective agonist U69,593 [(+)-(5α,7α,8β)-N-methyl-N-[7-(1-pyrrolidinyl)-1-oxaspiro[4.5]dec-8-yl]benzeneacetamide] inhibits serotonergic neuronal excitability within the DRN through both presynaptic inhibition of excitatory synaptic transmission and postsynaptic activation of G-protein-gated inwardly rectifying potassium channels (GIRKs) electrophysiologically recorded in brain slices. C57BL/6 mice subjected to repeated swim, stress sessions had significantly reduced KOR-mediated GIRK currents recorded in serotonergic neurons in DRN postsynaptically, without significantly affecting presynaptic KOR-mediated regulation of excitatory transmission. This effect was blocked by genetic excision of p38α MAPK selectively from serotonergic neurons. An increase in phospho-immunoreactivity suggests that this functional dysregulation may be a consequence of tyrosine phosphorylation of GIRK (KIR3.1) channels. These data elucidate a mechanism for stress-induced dysregulation of the excitability of neurons in the DRN and identify a functional target of stress-induced p38α MAPK activation that may underlie some of the negative effects of pathological stress exposure.

Efficient reward seeking is essential for survival and invariably requires overcoming costs, such as physical effort and delay, which are constantly changing in natural settings. Dopamine transmission has been implicated in decisions weighing the benefits and costs of obtaining a reward, but it is still unclear how dynamically changing effort and delay costs affect dopamine signaling to rewards and related stimuli. Using fast-scan cyclic voltammetry, we examined phasic dopamine release in the nucleus accumbens (NAcc) core and shell during reward-seeking behavior in rats. To manipulate the effort and time needed to earn a reward, we used instrumental tasks in which the response requirements (number of lever presses) were either fixed throughout a behavioral session [fixed ratio (FR)] or systematically increased from trial to trial [progressive ratio (PR)]. Dopamine release evoked by cues denoting reward availability was no different between these conditions, indicating insensitivity to escalating effort or delay costs. In contrast, dopamine release to reward delivery in both the NAcc core and shell increased in PR, but not in FR, sessions. This enhancement of reward-evoked dopamine signaling was also observed in sessions in which the response requirement was fixed but the delay to reward delivery increased, yoked to corresponding trials in PR sessions. These findings suggest that delay, and not effort, was principally responsible for the increased reward-evoked dopamine release in PR sessions. Together, these data demonstrate that NAcc dopamine release to rewards and their predictors are dissociable and differentially regulated by the delays conferred under escalating costs.

Activity changes in a large subset of midbrain dopamine neurons fulfill numerous assumptions of learning theory by encoding a prediction error between actual and predicted reward. This computational interpretation of dopaminergic spike activity invites the important question of how changes in spike rate are translated into changes in dopamine delivery at target neural structures. Using electrochemical detection of rapid dopamine release in the striatum of freely moving rats, we established that a single dynamic model can capture all the measured fluctuations in dopamine delivery. This model revealed three independent short-term adaptive processes acting to control dopamine release. These short-term components generalized well across animals and stimulation patterns and were preserved under anesthesia. The model has implications for the dynamic filtering interposed between changes in spike production and forebrain dopamine release.