Search

To evaluate the effectiveness of robotic rehabilitation alone, we developed a robotic rehabilitation model for rats and investigated the effects of inducing forced response-like movements on learning a choice reaction time task. The rats, while in a standing position, initially press two levers with their forepaws. An air-puff stimulation (go-signal: GS) is then applied to either their left or right forepaw. The rats respond by releasing the lever on the correct-response side (correct-RS). If the rats respond incorrectly, a lever-release motion is forcibly induced in the next repeated trial. Combining the lever activation side (correct-RS or incorrect-RS) and its timing (200 ms after GS or just before reaction time (RT)), we formed four experimental groups. The rats with a forepaw sensorimotor cerebral lesion were trained in serial reversal learning of the task, and their performance was compared using the error rate (ER) and RT. The group with incorrect-RS lever activation after GS (iRSaftGS) and the group with correct-RS lever activation before RT (cRSbfrRT) had lower ERs and RTs than the other groups. In addition, both groups showed large improvement in ER at high ER levels but only the iRSaftGS group’s improvement continued at lower ER levels. We conclude that the cRSbfrRT group facilitates only the extinction process and that the iRSaftGS group, which stretches agonistic muscles required for correct motion, is effective in both extinction and acquisition processes. Our findings provide an opportunity to reconsider the strategies for robotic rehabilitation and training. [ABSTRACT FROM AUTHOR]

Impulsive behavior during adolescence may stem from developmental imbalances between motivational and cognitive-control systems, producing greater urges to pursue reward and weakened capacities to inhibit such actions. Here, we developed a Pavlovian-instrumental transfer (PIT) protocol to assay rats' ability to suppress cue-motivated reward seeking based on changes in reward expectancy. Traditionally, PIT studies focus on how reward-predictive cues motivate instrumental reward-seeking behavior (lever pressing). However, cues signaling imminent reward delivery also elicit countervailing focal-search responses (food-port entry). We first examined how reward expectancy (cue-reward probability) influences expression of these competing behaviors. Adult male rats increased rates of lever pressing when presented with cues signaling lower probabilities of reward but focused their activity at the food cup on trials with cues that signaled higher probabilities of reward. We then compared adolescent and adult male rats in their responsivity to cues signaling different reward probabilities. In contrast to adults, adolescent rats did not flexibly adjust patterns of responding based on the expected likelihood of reward delivery but increased their rate of lever pressing for both weak and strong cues. These findings indicate that control over cue-motivated behavior is fundamentally dysregulated during adolescence, providing a model for studying neurobiological mechanisms of adolescent impulsivity.

The Pavlovian-Instrumental Transfer (PIT) paradigm examines probabilistic and reinforcement learning. Disruptions in mechanisms that mediate PIT (i.e., cues not triggering adaptive behaviors) are thought to be contributors to psychopathology, making the study of probabilistic and reinforcement learning clinically relevant. The current study evaluated an appetitive PIT effect and its relationship with symptom dimensions spanning depression and anxiety, with a particular focus on anhedonia. Forty young adults ranging in scores across dimensions of depression and anxiety symptoms completed the PIT paradigm and self-report symptom measures. The PIT paradigm consisted of three phases. The instrumental phase consisted of a contingent association in which participants squeezed a handgrip for monetary reward. The Pavlovian phase established a purely predictive association between three visual stimuli (CS + , CS-, baseline) and presence or absence of monetary reward. In the transfer phase, participants' responses allowed for examination of whether motivational characteristics of Pavlovian predictors influenced the vigor of their handgrip squeezes (instrumental action), which were formerly independent of Pavlovian associations. Analyses revealed a baseline-reward PIT effect, whereby a reward-associated Pavlovian cue enhanced instrumental responding in the transfer phase. However, there were no significant differences between CS + and CS- or CS- and baseline cues, suggesting a disrupted interaction of Pavlovian and instrumental learning. Further, the appetitive PIT effect captured in this paradigm was not associated with anhedonia, fears, or general distress. Future work should investigate the influence of mood states using more specific appetitive PIT paradigms to further understanding of the implications of disrupted reflexive and instrumental responding. [ABSTRACT FROM AUTHOR]

Predictive learning can engage a selective form of cognitive control that biases choice between actions based on information about future outcomes that the learning provides. This influence has been hypothesized to depend on a feedback circuit in the brain through which the basal ganglia modulate activity in the prefrontal cortex; however, direct evidence for this functional circuit has proven elusive. Here, using an animal model of cognitive control, we found that the influence of predictive learning on decision making is mediated by an inhibitory feedback circuit linking the medial ventral pallidum and the mediodorsal thalamus, the activation of which causes disinhibition of the orbitofrontal cortex via reduced activation of inhibitory parvalbumin interneurons during choice. Thus, we found that, for this function, the mediodorsal thalamus serves as a pallidal-cortical relay through which predictive learning controls action selection, which has important implications for understanding cognitive control and its vicissitudes in various psychiatric disorders and addiction. • Predictive learning affects choice via a ventral pallidum to MD-thalamus to OFC circuit • MD-thalamus activity inhibits the OFC via a projection onto OFC-PV neurons • The ventral pallidum releases choice by inhibiting this MD-thalamus to OFC-PV projection • Cognitive control involves a disinhibitory basal ganglia-cortical feedback circuit Cognitive control allows us and other animals to use predictive information to control choices between actions. Here, Leung et al. establish that this control process is mediated by a disinhibitory basal ganglia-cortical feedback circuit, via which the ventral pallidum inhibits the mediodorsal thalamus to disinhibit the orbitofrontal cortex and release choice. [ABSTRACT FROM AUTHOR]

One experiment with rats used Pavlovian-to-instrumental transfer (PIT) tests to explore potential competitive interactions between Pavlovian and instrumental processes during instrumental learning. Two instrumental response-outcome relations (e.g., left lever – grain pellets, right lever – sucrose pellets) were first trained in distinct contexts for one group of rats (Group Differential) or in each of two contexts for a second group (Group Non-Differential).  Both of these groups then received training with two Pavlovian stimulus-outcome relations in a third experimental context.  Selective PIT tests conducted in both the Pavlovian and instrumental contexts revealed greater selective PIT in Group Non-Differential than in Group Differential subjects.  This result is discussed in terms of the roles played by context-outcome, response-outcome, and outcome-response associations during instrumental learning. The results further help us understand the nature of Pavlovian-instrumental interactions in specific PIT tasks.

Predictive learning endows stimuli with the capacity to signal both the sensory‐specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory‐specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome‐specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta‐opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome‐specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning‐related plasticity to modify brain circuitry to prepare for future significant events. [ABSTRACT FROM AUTHOR]

Impulsive behavior during adolescence may stem from developmental imbalances between motivational and cognitive-control systems, producing greater urges to pursue reward and weakened capacities to inhibit such actions. Here, we developed a Pavlovian-instrumental transfer (PIT) protocol to assay rats’ ability to suppress cue-motivated reward seeking based on changes in reward expectancy. Traditionally, PIT studies focus on how reward-predictive cues motivate instrumental reward-seeking behavior (lever pressing). However, cues signaling imminent reward delivery also elicit countervailing focal-search responses (food-port entry). We first examined how reward expectancy (cue-reward probability) influences expression of these competing behaviors. Adult male rats increased rates of lever pressing when presented with cues signaling lower probabilities of reward but focused their activity at the food cup on trials with cues that signaled higher probabilities of reward. We then compared adolescent and adult male rats in their responsivity to cues signaling different reward probabilities. In contrast to adults, adolescent rats did not flexibly adjust patterns of responding based on the expected likelihood of reward delivery but increased their rate of lever pressing for both weak and strong cues. These findings indicate that control over cue-motivated behavior is fundamentally dysregulated during adolescence, providing a model for studying neurobiological mechanisms of adolescent impulsivity.

Inhibitory stimuli can reduce animals' reward seeking in an outcome-specific manner or outcome-general manner. However, we do not understand the factors that determine which of these effects are produced. To address this, we carried out three experiments which examined whether instrumental training with one or multiple outcomes determined the nature of subsequently observed Pavlovian-instrumental transfer (PIT). Rats underwent Pavlovian training to produce inhibitors and excitors for two outcomes using a feature-negative procedure. In Experiment 1, these stimuli were tested for their effects on a single response trained with one of those outcomes in a PIT procedure. Here, stimuli trained as inhibitors and excitors were found to produce outcome-general effects on reward seeking (in addition to an outcome-specific effect for excitors). In Experiment 2, we trained two responses, one for each of the Pavlovian outcomes, and tested the effect of the stimuli on each response individually. This design also produced outcome-general inhibitory and excitatory PIT effects. Experiment 3 followed the procedure of Experiment 2, except for implementation of a shorter Pavlovian training phase and an additional choice test, where both responses were concurrently available. This procedure produced putative inhibitory effects that were also outcome-general. However, outcome-specific excitatory effects were observed, indicating that the general inhibitory results may not be attributable to the duration of Pavlovian training. Overall, this study suggests that variations in the number of response–outcome contingencies experienced by animals do not readily determine the specificity of putative inhibitors. [ABSTRACT FROM AUTHOR]

Background: Drug cue-reactivity can be measured by the well-established cue-elicited craving model, or by the more recently developed Pavlovian to instrumental transfer (PIT) procedure, which quantifies the impact of drug cues on drug-seeking behaviour. It remains unclear whether these two models produce similar cue reactive effects. Method: To test this, 38 young adult beer drinkers completed an alcohol cue-elicited craving procedure followed by a specific PIT procedure with alcohol cues. Results: There was a significant effect of alcohol cues on craving (p =.007) and on alcohol-seeking behaviour in the PIT procedure (p <.001). Contrary to expectations, these two indices of cue-reactivity were not correlated (r = −.08, p =.66). However, analysis indicated that the alcohol PIT effect was correlated with the self-reported belief that alcohol cues signalled greater effectiveness of the alcohol-seeking response (r =.44, p =.008). Conclusions: These findings suggest that different measures of cue-reactivity might tap into different responses within an individual. Future research is necessary to consider whether this variance is due to which aspect of cue reactivity is being assessed and whether different types of cue-reactivity are differentially influenced by variables such as outcome expectancy. [ABSTRACT FROM AUTHOR]

The influence of Pavlovian conditioned stimuli on ongoing behavior may contribute to explaining how alcohol cues stimulate drug seeking and intake. Using a Pavlovian-instrumental transfer task, we investigated the effects of alcohol-related cues on approach behavior (i.e., instrumental response behavior) and its neural correlates, and related both to the relapse after detoxification in alcohol-dependent patients. Thirty-one recently detoxified alcohol-dependent patients and 24 healthy controls underwent instrumental training, where approach or non-approach towards initially neutral stimuli was reinforced by monetary incentives. Approach behavior was tested during extinction with either alcohol-related or neutral stimuli (as Pavlovian cues) presented in the background during functional magnetic resonance imaging (fMRI). Patients were subsequently followed up for 6 months. We observed that alcohol-related background stimuli inhibited the approach behavior in detoxified alcohol-dependent patients (t = − 3.86, p <.001), but not in healthy controls (t = − 0.92, p =.36). This behavioral inhibition was associated with neural activation in the nucleus accumbens (NAcc) (t(30) = 2.06, p <.05). Interestingly, both the effects were only present in subsequent abstainers, but not relapsers and in those with mild but not severe dependence. Our data show that alcohol-related cues can acquire inhibitory behavioral features typical of aversive stimuli despite being accompanied by a stronger NAcc activation, suggesting salience attribution. The fact that these findings are restricted to abstinence and milder illness suggests that they may be potential resilience factors. Clinical trial: LeAD study, http://www.lead-studie.de, NCT01679145. [ABSTRACT FROM AUTHOR]

Highlights • The medial orbitofrontal cortex (mOFC) of the rat is functionally heterogeneous. • Anterior vs. posterior mOFC has stronger connections with the accumbens core. • Anterior vs. posterior mOFC is critical for inferring unobservable action outcomes. • Anterior vs. posterior mOFC is more directly involved in goal-directed action. Abstract Although studies examining orbitofrontal cortex (OFC) often treat it as though it were functionally homogeneous, recent evidence has questioned this assumption. Not only are the various subregions of OFC (lateral, ventral, and medial) hetereogeneous, but there is further evidence of heterogeneity within those subregions. For example, several studies in both humans and monkeys have revealed a functional subdivision along the anterior-posterior gradient of the medial OFC (mOFC). Given our previous findings suggesting that, in rats, the mOFC is responsible for inferring the likelihood of unobservable action outcomes (Bradfield, Dezfouli, van Holstein, Chieng, & Balleine, 2015), and given the anterior nature of the placements of our prior manipulations, we decided to assess whether the rat mOFC also differs in connection and function along its anteroposterior axis. We first used retrograde tracing to compare the density of efferents from mOFC to several structures known to contribute to goal-directed action: the mediodorsal thalamus, basolateral amygdala, posterior dorsomedial striatum, nucleus accumbens core and ventral tegmental area. We then compared the functional effects of anterior versus posterior mOFC excitotoxic lesions on tests of Pavlovian-instrumental transfer, instrumental outcome devaluation and outcome-specific reinstatement. We found evidence that the anterior mOFC had greater connectivity with the accumbens core and greater functional involvement in goal-directed action than the posterior mOFC. Consistent with previous findings across species, therefore, these results suggest that the anterior and posterior mOFC of the rat are indeed functionally distinct, and that it is the anterior mOFC that is particularly critical for inferring unobservable action outcomes. [ABSTRACT FROM AUTHOR]

Animals can readily learn that stimuli predict the absence of specific appetitive outcomes; however, the neural substrates underlying such outcome-specific conditioned inhibition remain largely unexplored. Here, using female and male rats as subjects, we examined the involvement of the lateral habenula (LHb) and of its inputs onto the rostromedial tegmental nucleus (RMTg) in inhibitory learning. In these experiments, we used backward conditioning and contingency reversal to establish outcome-specific conditioned inhibitors for two distinct appetitive outcomes. Then, using the Pavlovian-instrumental transfer paradigm, we assessed the effects of manipulations of the LHb and the LHb-RMTg pathway on that inhibitory encoding. In control animals, we found that an outcome-specific conditioned inhibitor biased choice away from actions delivering that outcome and toward actions earning other outcomes. Importantly, this bias was abolished by both electrolytic lesions of the LHb and selective ablation of LHb neurons using Cre-dependent Caspase3 expression in Cre-expressing neurons projecting to the RMTg. This deficit was specific to conditioned inhibition; an excitatory predictor of a specific outcome-biased choice toward actions delivering the same outcome to a similar degree whether the LHb or the LHb-RMTg network was intact or not. LHb lesions also disrupted the ability of animals to inhibit previously encoded stimulus-outcome contingencies after their reversal, pointing to a critical role of the LHb and of its inputs onto the RMTg in outcome-specific conditioned inhibition in appetitive settings. These findings are consistent with the developing view that the LHb promotes a negative reward prediction error in Pavlovian conditioning. [ABSTRACT FROM AUTHOR]

We review quantitative accounts of behavioral momentum theory (BMT), its application to clinical treatment, and its extension to post-intervention relapse of target behavior. We suggest that its extension can account for relapse using reinstatement and renewal models, but that its application to resurgence is flawed both conceptually and in its failure to account for recent data. We propose that the enhanced persistence of target behavior engendered by alternative reinforcers is limited to their concurrent availability within a distinctive stimulus context. However, a failure to find effects of stimulus-correlated reinforcer rates in a Pavlovian-to-Instrumental Transfer (PIT) paradigm challenges even a straightforward Pavlovian account of alternative reinforcer effects. BMT has been valuable in understanding basic research findings and in guiding clinical applications and accounting for their data, but alternatives are needed that can account more effectively for resurgence while encompassing basic data on resistance to change as well as other forms of relapse. [ABSTRACT FROM AUTHOR]

The present multimodal MRI study advances our understanding of the corticostriatal circuits underlying goal-directed vs. cue-driven, habitual food seeking. To this end, we employed a computerized Pavlovian-instrumental transfer paradigm. During the test phase, participants were free to perform learned instrumental responses (left and right key presses) for popcorn and Smarties outcomes. Importantly, prior to this test half of the participants had been sated on popcorn and the other half on Smarties - resulting in a reduced desirability of those outcomes. Furthermore, during a proportion of the test trials, food-associated Pavlovian cues were presented in the background. In line with previous studies, we found that participants were able to perform in a goal-directed manner in the absence of Pavlovian cues, meaning that specific satiation selectively reduced responding for that food. However, presentation of Pavlovian cues biased choice toward the associated food reward regardless of satiation. Functional MRI analyses revealed that, in the absence of Pavlovian cues, posterior ventromedial prefrontal cortex tracked outcome value. In contrast, during cued trials, the BOLD signal in the posterior putamen differentiated between responses compatible and incompatible with the cue-associated outcome. Furthermore, we identified a region in ventral amygdala showing relatively strong functional connectivity with posterior putamen during the cued trials. Structural MRI analyses provided converging evidence for the involvement of corticostriatal circuits: diffusion tensor imaging data revealed that connectivity of caudate-seeded white-matter tracts to the ventromedial prefrontal cortex predicted responding for still-valuable outcomes; and gray matter integrity in the premotor cortex predicted individual Pavlovian cueing effects. [ABSTRACT FROM AUTHOR]

Previous work has demonstrated that goal-directed control of alcohol seeking and other drug-related behaviors is reduced following extended self-administration and drug exposure. Here we examined how the magnitude of stimulus influences on responding changes across similar training and drug exposure. Rats self-administered alcohol or sucrose for two or eight weeks. Previous work has shown that eight, but not two weeks of self-administration produces habitual alcohol seeking. Next, all animals received equivalent Pavlovian conditioning sessions where a discrete stimulus predicted the delivery of alcohol or sucrose. Finally, the impact of the stimuli on ongoing instrumental responding was examined in a Pavlovian-instrumental transfer (PIT) test. While a significant PIT effect was observed following two weeks of either alcohol or sucrose self-administration, the magnitude of this effect was greater following eight weeks of training. The specificity of the PIT effect appeared unchanged by extended training. While it is well established that evaluation of the outcome of responding contributes less to behavioral control following extended training and/or drug exposure, our data indicate that reward-predictive stimuli have a stronger contribution to responding after extended training. Together, these findings provide insight into the factors that control behavior after extended drug use which will be important for developing effective methods for controlling and ideally reducing these behaviors.

Summary Extinction involves altering a previously established predictive relationship between a cue and its outcome by repeatedly presenting that cue alone. Although it is widely accepted that extinction generates some form of inhibitory learning [ 1–4 ], direct evidence for this claim has been lacking, and the nature of the associative changes induced by extinction have, therefore, remained a matter of debate [ 5–8 ]. In the current experiments, we used a novel behavioral approach that we recently developed and that provides a direct measure of conditioned inhibition [ 9 ] to compare the influence of extinguished and non-extinguished cues on choice between goal-directed actions. Using this approach, we provide direct evidence that extinction generates outcome-specific conditioned inhibition. Furthermore, we demonstrate that this inhibitory learning is controlled by the infralimbic cortex (IL); inactivation of the IL using M4 DREADDs abolished outcome-specific inhibition and rendered the cue excitatory. Importantly, we found that context modulated this inhibition. Outside its extinction context, the cue was excitatory and functioned as a specific predictor of its previously associated outcome, biasing choice toward actions earning the same outcome. In its extinction context, however, the cue acted as a specific inhibitor and biased choice toward actions earning different outcomes. Context modulation of these excitatory and inhibitory memories was mediated by the dorsal hippocampus (HPC), suggesting that the HPC and IL act in concert to control the influence of conditioned inhibitors on choice. These findings demonstrate for the first time that extinction turns a cue into a net inhibitor that can influence choice via counterfactual action-outcome associations. [ABSTRACT FROM AUTHOR]

Pavlovian stimuli predictive of food can markedly amplify instrumental responding for food. This effect is termed Pavlovian-instrumental transfer (PIT). The ventral tegmental area (VTA) plays a key role in mediating PIT, however, it is yet unknown whether N-methyl- d-aspartate (NMDA)-type glutamate receptors in the VTA are involved in PIT. Here, we examined the effects of an NMDA-receptor blockade in the VTA on PIT. Immediately prior to PIT testing, rats were subjected to intra-VTA infusions of vehicle or of the NMDA-receptor antagonist 2-amino-5-phosphonopentanoic acid (AP-5) (1, 5 µg/side). In rats that received AP-5 at the lower dose, the PIT effect was intact, i.e. presentation of the Pavlovian stimulus enhanced instrumental responding. By contrast, in rats that received AP-5 at the higher dose, the PIT effect was blocked. The data suggest that NMDA receptors in the VTA mediate the activating effects of Pavlovian stimuli on instrumental responding. [ABSTRACT FROM AUTHOR]

Reward cues can contribute to overconsumption of food and drugs and can relapse. The failure of exposure therapies to reduce overconsumption and relapse is generally attributed to the context-specificity of extinction. However, no previous study has examined whether cue-elicited reward-seeking (as opposed to cue-reactivity) is sensitive to context renewal. We tested this possibility in 160 healthy volunteers using a Pavlovian-instrumental transfer (PIT) design involving voluntary responding for a high value natural reward (chocolate). One reward cue underwent Pavlovian extinction in the same (Group AAA) or different context (Group ABA) to all other phases. This cue was compared with a second non-extinguished reward cue and an unpaired control cue. There was a significant overall PIT effect with both reward cues eliciting reward-seeking on test relative to the unpaired cue. Pavlovian extinction substantially reduced this effect, with the extinguished reward cue eliciting less reward-seeking than the non-extinguished reward cue. Most interestingly, extinction of cue-elicited reward-seeking was sensitive to renewal, with extinction less effective for reducing PIT when conducted in a different context. These findings have important implications for extinction-based interventions for reducing maladaptive reward-seeking in practice. [ABSTRACT FROM AUTHOR]

Previous work has demonstrated that goal-directed control of alcohol-seeking and other drug-related behaviors is reduced following extended self-administration and drug exposure. Here, we examined how the magnitude of stimulus influences on responding changes across similar training and drug exposure. Rats self-administered alcohol or sucrose for 2 or 8 weeks. Previous work has shown that 8 weeks, but not 2 weeks of self-administration produces habitual alcohol seeking. Next, all animals received equivalent Pavlovian conditioning sessions where a discrete stimulus predicted the delivery of alcohol or sucrose. Finally, the impact of the stimuli on ongoing instrumental responding was examined in a Pavlovian-instrumental transfer (PIT) test. While a significant PIT effect was observed following 2 weeks of either alcohol or sucrose self-administration, the magnitude of this effect was greater following 8 weeks of training. The specificity of the PIT effect appeared unchanged by extended training. While it is well established that evaluation of the outcome of responding contributes less to behavioral control following extended training and/or drug exposure, our data indicate that reward-predictive stimuli have a stronger contribution to responding after extended training. Together, these findings provide insight into the factors that control behavior after extended drug use, which will be important for developing effective methods for controlling and ideally reducing these behaviors. [ABSTRACT FROM AUTHOR]

Pavlovian stimuli predictive of food are able to amplify instrumental responding for food. This phenomenon termed Pavlovian-instrumental transfer (PIT) critically depends on intact VTA function and mesoaccumbens dopamine transmission. Considerable evidence suggests that food-predictive stimuli can enhance the release of ghrelin, an orexigen hormone that promotes food-directed responding. The ventral tegmental area (VTA) appears to be a key region through which stimulation of ghrelin receptors (GHS-R1A) invigorates food-directed responding, in part by activating the mesoaccumbens dopamine system. Thus, it is conceivable that stimulation of GHS-R1A in the VTA can amplify PIT, i.e. stimulus-elicited increase in lever pressing for food. Here we examined in rats the effects of VTA ghrelin microinfusion on PIT. Our results demonstrate that ghrelin microinfusion into the VTA failed to enhance PIT suggesting that VTA GHS-R1A stimulation was unable to enhance the motivational significance of food-predictive stimuli. Consistent with previous studies, our results further indicate that intra-VTA ghrelin microinfusion invigorated instrumental responding under a progressive ratio schedule. These data provide support to the notion that VTA GHS-R1A stimulation increases the tendency to work for food. [ABSTRACT FROM AUTHOR]

Loss of flexible control over alcohol use may contribute to the development of alcohol use disorders. An increased contribution of response habits to alcohol-related behaviors may help explain this loss of control. Focusing on data from outcome devaluation and Pavlovian-instrumental transfer procedures, we review evidence for loss of goal-directed control over alcohol seeking and consumption drawing from both preclinical findings and clinical data where they exist. Over the course of extended alcohol self-administration and exposure, the performance of alcohol-seeking responses becomes less sensitive to reduction in the value of alcohol and more vulnerable to the influences of alcohol-predictive stimuli. These behavioral changes are accompanied by a shift in the corticostriatal circuits that control responding from circuits centered on the dorsomedial to those centered on the dorsolateral striatum. These changes in behavioral and neural control could help explain failures to abstain from alcohol despite intention to do so. Understanding and ultimately ameliorating these changes will aid development of more effective treatment interventions. [ABSTRACT FROM AUTHOR]

BackgroundChanges in reflexive emotional responses are hallmarks of depression, but how emotional reflexes make an impact on adaptive decision-making in depression has not been examined formally. Using a Pavlovian-instrumental transfer (PIT) task, we compared the influence of affectively valenced stimuli on decision-making in depression and generalized anxiety disorder compared with healthy controls; and related this to the longitudinal course of the illness.MethodA total of 40 subjects with a current DSM-IV-TR diagnosis of major depressive disorder, dysthymia, generalized anxiety disorder, or a combination thereof, and 40 matched healthy controls performed a PIT task that assesses how instrumental approach and withdrawal behaviours are influenced by appetitive and aversive Pavlovian conditioned stimuli (CSs). Patients were followed up after 4–6 months. Analyses focused on patients with depression alone (n = 25).ResultsIn healthy controls, Pavlovian CSs exerted action-specific effects, with appetitive CSs boosting active approach and aversive CSs active withdrawal. This action-specificity was absent in currently depressed subjects. Greater action-specificity in patients was associated with better recovery over the follow-up period.ConclusionsDepression is associated with an abnormal influence of emotional reactions on decision-making in a way that may predict recovery. [ABSTRACT FROM PUBLISHER]

A recent focus of addiction research has been on the effect of drug exposure on the neural processes that mediate the acquisition and performance of goal-directed instrumental actions. Deficits in goal-directed control and a consequent dysregulation of habit learning processes have been described as resulting in compulsive drug seeking. Similarly, considerable research has focussed on the motivational and emotional changes that drugs produce and that result in changes in the incentive processes that modulate goal-directed performance. Although these areas have developed independently, we argue that the effects they described are likely not independent. Here we hypothesize that these changes result from a core deficit in the way the learning and performance factors that support goal-directed action are integrated at a neural level to maintain behavioural control. A dorsal basal ganglia stream mediating goal-directed learning and a ventral stream mediating various performance factors find several points of integration in the cortical basal ganglia system, most notably in the thalamocortical network linking basal ganglia output to a variety of cortical control centres. Recent research in humans and other animals is reviewed suggesting that learning and performance factors are integrated in a network centred on the mediodorsal thalamus and that disintegration in this network may provide the basis for a ‘switch’ from recreational to dysregulated drug seeking resulting in the well documented changes associated with addiction. This article is part of a Special Issue entitled SI:Addiction circuits . [ABSTRACT FROM AUTHOR]

Pavlovian conditioned stimuli can influence instrumental responding, an effect called Pavlovian-instrumental transfer (PIT).During the last decade, PIT has been subdivided into two types: specific PIT and general PIT, each having its own neural substrates.Specific PIT happens when a conditioned stimulus (CS) associated with a reward enhances an instrumental response directed to the same reward.Under general PIT instead, the CS enhances a response directed to a different reward.While important progress has been made into identifying the neural substrates, the function of specific and general PIT and how they interact with instrumental responses, are still not clear.In the experimental paradigm that distinguishes specific and general PIT an effect of PIT inhibition has also been observed and is waiting for an explanation.Here we propose an hypothesis that links these three PIT effects (specific PIT, general PIT and PIT inhibition) to three aspects of action evaluation.These three aspects, which we call "principles of action" are: context, efficacy, and utility.In goal-directed behavior, an agent has to evaluate if the context is suitable to accomplish the goal, the efficacy of his action in getting the goal and the utility of the goal itself:we suggest that each of the three PIT effects is related to one of these aspects of action evaluation.In particular, we link specific PIT with the estimation of efficacy, general PIT with the evaluation of utility and PIT inhibition with the adequacy of context.We also provide a latent cause Bayesian computational model that exemplifies this hypothesis.This hypothesis and the model provide a new framework and new predictions to advance knowledge about PIT functioning and its role in animal adaptation.

Background: Contemporary learning theories of drug addiction ascribe a key role to Pavlovian learning mechanisms in the development, maintenance, and relapse of addiction. In fact, cue-reactivity research has demonstrated the power of alcohol-associated cues to activate the brain's reward system, which has been linked to craving and subsequent relapse. However, whether de novo Pavlovian conditioning is altered in alcohol use disorder (AUD) has rarely been investigated., Methods: To characterize de novo Pavlovian conditioning in AUD, 62 detoxified patients with AUD and 63 matched healthy control participants completed a Pavlovian learning task as part of a Pavlovian-to-instrumental transfer paradigm during a functional magnetic resonance imaging session. Patients were followed up for 12 months to assess drinking behavior and relapse status., Results: While patients and healthy controls did not differ in their ability to explicitly acquire the contingencies between conditioned and unconditioned stimuli, patients with AUD displayed significantly stronger amygdala responses toward Pavlovian cues, an effect primarily driven by stronger blood oxygen level-dependent differentiation during learning from reward compared with punishment. Moreover, in patients compared with controls, differential amygdala responses during conditioning were positively related to the ability of Pavlovian stimuli to influence ongoing instrumental choice behavior measured during a subsequent Pavlovian-to-instrumental transfer test. Finally, patients who relapsed within the 12-month follow-up period showed an inverse association between amygdala activity during conditioning and relapse latency., Conclusions: We provide evidence of altered neural correlates of de novo Pavlovian conditioning in patients with AUD, especially for appetitive stimuli. Thus, heightened processing of Pavlovian cues might constitute a behaviorally relevant mechanism in alcohol addiction., (© 2023 The Authors.)

Background and Purpose: Stimuli that predict rewarding events can control choice between future actions, and this control could be mediated by δ-opioid receptors in the nucleus accumbens shell (NAc-S). Stimuli predicting the absence of important events can also guide choice, although it remains unknown whether they do so via changes in an accumbal δ-opioid receptor-related process.Experimental Approach: δ-opioid receptor-eGFP mice were trained to perform two instrumental actions that delivered different food outcomes. Choice between the two actions was then tested in the presence of stimuli paired with either the delivery or the non-delivery of each of the two outcomes. Bilateral infusions of the δ-opioid receptor antagonist naltrindole into the NAc-S were used to determine the role of these receptors at the time of choice and δ-opioid receptor expression in the NAc-S used to assess functional activity.Key Results: A stimulus predicting a specific outcome biased choice performance towards the action previously earning that same outcome. In contrast, a stimulus signalling the absence of that outcome biased performance away from the action that delivered that outcome towards actions associated with the absence of that outcome. Both effects were associated with increased δ-opioid receptor expression on the membrane of cholinergic interneurons within the NAc-S. Furthermore, both effects were blocked by naltrindole infused into the NAc-S.Conclusions and Implications: These findings suggest that δ-opioid receptors in the NAc-S were involved in the effects of predictive learning on choice between actions, whether those predictions involve the presence or absence of specific rewarding events.Linked Articles: This article is part of a themed section on Opioids: New Pathways to Functional Selectivity. To view the other articles in this section visit http://dx.doi.org/10.1111/bph.2015.172.issue-2. [ABSTRACT FROM AUTHOR]

Learning may play an important role in over-eating. One example is Pavlovian-to-instrumental transfer (PIT), whereby reward cues facilitate responding to obtain that reward. Whilst there is increasing research indicating PIT for food in humans, these studies have exclusively tested PIT under instrumental extinction (i.e. when the food is no longer available), which may reduce their ecological validity. To address this, we conducted two experiments exploring PIT for food in humans when tested under instrumental reinforcement. Participants first underwent Pavlovian discrimination training with an auditory cue paired with a chocolate reward (CS+) and another auditory cue unpaired (CS−). In instrumental training participants learnt to press a button to receive the chocolate reward on a VR10 schedule. In the test phase, each CS was presented whilst participants maintained the opportunity to press the button to receive chocolate. In Experiment 1, the PIT test was implemented after up to 20 min of instrumental training (satiation) whereas in Experiment 2 it was implemented after only 4 min of instrumental training. In both experiments there was evidence for differential PIT, but the pattern differed according to the rate of responding at the time of the PIT test. In low baseline responders the CS+ facilitated both button press responding and consumption, whereas in high baseline responders the CS− suppressed responding. These findings suggest that both excitatory and inhibitory associations may be learnt during PIT training and that the expression of these associations depends on motivation levels at the time the cues are encountered. Particularly concerning is that a food-paired cue can elicit increased motivation to obtain and consume food even when the participant is highly satiated and no longer actively seeking food, as this may be one mechanism by which over-consumption is maintained. [ABSTRACT FROM AUTHOR]

Initially neutral conditioned stimuli paired with food often acquire motivating properties, including serving as secondary reinforcers, enhancing instrumental responding in Pavlovian-instrumental transfer procedures, and potentiating food consumption under conditions of food satiation. Interestingly, cues associated with the cancellation of food and food cues may also potentiate food consumption (e.g., Galarce and Holland, ), despite their apparent negative correlations with food delivery. In three experiments with rats, we investigated conditions under which potentiation of feeding by such 'interruption stimuIi' (ISs) develops, and some aspects of the content of that learning. Although in all three experiments ISs enhanced food consumption beyond control levels, they were found to act as conditioned inhibitors for anticipatory food cup entry (Experiment 1), to serve as conditioned punishers of instrumental responding (Experiment 2), and to suppress instrumental lever press responding in a Pavlovian instrumental transfer procedure (Experiment 3). Furthermore, when given concurrent choice between different foods, an IS enhanced consumption of the food whose interruption it had previously signaled, but when given a choice between performing two instrumental responses, the IS shifted rats' choice away from the response that had previously yielded the food whose interruption had been signaled by IS (Experiment 3). Thus, the effects of an IS on appetitive responses were opposite to its effects on consummatory responding. Implications for our understanding of learned incentive motivation and the control of overeating are discussed. [ABSTRACT FROM AUTHOR]

Decisions can arise in different ways, such as a gut feeling, doing what worked last time, or planful deliberation. Different decision-making systems are dissociable behaviorally, map onto distinct brain systems, and require different computational demands. For instance, ’model-free’ decision strategies use prediction errors to estimate scalar action values from previous experience, while ’model-based’ strategies leverage internal forward models to generate and evaluate potentially rich outcome expectancies. Animal learning studies indicate that expectancies may arise from different sources, including not only forward models but also Pavlovian associations, and the flexibility with which such representations impact behavior may depend on how they are generated. In the light of these considerations, we review the results of van der Meer and Redish (2009a), who found that ventral striatal neurons that respond to reward delivery can also be activated at other points, notably at a decision point where hippocampal forward representations were also observed. These data suggest the possibility that ventral striatal reward representations contribute to model-based expectancies used in deliberative decision-making.

Initially neutral conditioned stimuli (CSs) paired with food often acquire motivating properties. For example, CS presentations may enhance the rate of instrumental responding that normally earns that food reward (Pavlovian-instrumental transfer), or potentiate consumption of that food when the animal is food-sated. Recent evidence suggests that cues associated with the withdrawal of food and food cues (interruption stimuli or ISs) may also potentiate feeding, despite exhibiting some characteristics of conditioned inhibition. Here, we compared the ability of CSs and ISs to modulate both eating food and working for it. If CSs and ISs potentiate eating food by controlling a similar incentive state, both types of cues might also be expected to enhance instrumental responding for food. Although we found substantial potentiation of feeding by both CSs and ISs. and powerful enhancement of instrumental responding by a CS, we found no evidence for such instrumental enhancement by an IS. Furthermore, although an IS produced more FOS expression in the amygdala central nucleus (CeA) than either a previously reinforced CS or a control stimulus after a test for potentiated feeding, an intact CeA was unnecessary for potentiation of feeding by either a CS or an IS. Nevertheless, as in previous studies, CeA was critical to the ability of a CS to enhance instrumental responding. Implications for understanding the nature and basis for incentive learning are discussed. [ABSTRACT FROM AUTHOR]

Pavlovian conditioned stimuli can influence instrumental responding, an effect called Pavlovian-instrumental transfer (PIT). During the last decade, PIT has been subdivided into two types: specific PIT and general PIT, each having its own neural substrates. Specific PIT happens when a conditioned stimulus (CS) associated with a reward enhances an instrumental response directed to the same reward. Under general PIT, instead, the CS enhances a response directed to a different reward. While important progress has been made into identifying the neural substrates, the function of specific and general PIT and how they interact with instrumental responses are still not clear. In the experimental paradigm that distinguishes specific and general PIT an effect of PIT inhibition has also been observed and is waiting for an explanation. Here we propose an hypothesis that links these three PIT effects (specific PIT, general PIT and PIT inhibition) to three aspects of action evaluation. These three aspects, which we call "principles of action", are: context, efficacy, and utility. In goal-directed behavior, an agent has to evaluate if the context is suitable to accomplish the goal, the efficacy of his action in getting the goal, and the utility of the goal itself: we suggest that each of the three PIT effects is related to one of these aspects of action evaluation. In particular, we link specific PIT with the estimation of efficacy, general PIT with the evaluation of utility, and PIT inhibition with the adequacy of context. We also provide a latent cause Bayesian computational model that exemplifies this hypothesis. This hypothesis and the model provide a new framework and new predictions to advance knowledge about PIT functioning and its role in animal adaptation. [ABSTRACT FROM AUTHOR]

Reward-associated cues are known to influence motivation to approach both natural and man-made rewards, such as food and drugs. However, the mechanisms underlying these effects are not well understood. To model these processes in the laboratory with humans, we developed an appetitive Pavlovian-instrumental transfer procedure with a chocolate reward. We used a single unconstrained response that led to an actual rather than symbolic reward to assess the strength of reward motivation. Presentation of a chocolate-paired cue, but not an unpaired cue, markedly enhanced instrumental responding over a 30-s period. The same pattern was observed with 10-s and 30-s cues, showing that close cue-reward contiguity is not necessary for facilitation of reward-directed action. The results confirm that reward- related cues can instigate voluntary action to obtain that reward. The effectiveness of long-duration cues suggests that in clinical settings, attention should be directed to both proximal and distal cues for reward. [ABSTRACT FROM AUTHOR]