Search

Background: In substance-dependent individuals, drug deprivation and drug use trigger divergent behavioral responses to environmental cues. These divergent responses are consonant with data showing that short- and long-term adaptations in dopamine signaling are similarly sensitive to state of drug use. The literature suggests a drug state–dependent role of learning in maintaining substance use; evidence linking dopamine to both reinforcement learning and addiction provides a framework to test this possibility. Methods: In a randomized crossover design, 22 participants with current cocaine use disorder completed a probabilistic loss-learning task during functional magnetic resonance imaging while on and off cocaine (44 sessions). Another 54 participants without Axis I psychopathology served as a secondary reference group. Within-drug state and paired-subjects’ learning effects were assessed with computational model–derived individual learning parameters. Model-based neuroimaging analyses evaluated effects of drug use state on neural learning signals. Relationships among model-derived behavioral learning rates (α+, α−), neural prediction error signals (δ+, δ−), cocaine use, and desire to use were assessed. Results: During cocaine deprivation, cocaine-dependent individuals exhibited heightened positive learning rates (α+), heightened neural positive prediction error (δ+) responses, and heightened association of α+ with neural δ+ responses. The deprivation-enhanced neural learning signals were specific to successful loss avoidance, comparable to participants without psychiatric conditions, and mediated a relationship between chronicity of drug use and desire to use cocaine. Conclusions: Neurocomputational learning signals are sensitive to drug use status and suggest that heightened reinforcement by successful avoidance of negative outcomes may contribute to drug seeking during deprivation. More generally, attention to drug use state is important for delineating substrates of addiction. © 2018 This work was supported in part by the National Institutes of Health (Grant Nos. R01MH091872 and R21DA042274 [to PHC], Grant No. R01DA036017 to [BK-C], and Grant Nos. RC1DA028387 and R01DA023624 [to RDLG]). PHC, BK-C, RDLG, and TN designed the experiments. JMW analyzed the data with input from LZ, VMB, PHC, and BK-C. PHC, BK-C, RDLG, and TN supervised this work. JMW and PHC drafted the manuscript with input from all authors. All authors edited and approved the final version. We acknowledge the technical assistance of George Christopoulos, Dongil Chung, Jacob Lee, James Mahoney, Dharol Tankersley, Katherine McCurry, Nina Lauharatanahirun, and members of the Chiu, De La Garza, King-Casas, and Newton Labs. The authors report no biomedical financial interests or potential conflicts of interest.

Disproportionate reactions to unexpected stimuli in the environment are a cardinal symptom of posttraumatic stress disorder (PTSD). Here, we test whether these heightened responses are associated with disruptions in distinct components of reinforcement learning. Specifically, using functional neuroimaging, a loss-learning task, and a computational model-based approach, we assessed the mechanistic hypothesis that overreactions to stimuli in PTSD arise from anomalous gating of attention during learning (i.e., associability). Behavioral choices of combat- deployed veterans with and without PTSD were fit to a reinforcement learning model, generating trial-by-trial prediction errors (signaling unexpected outcomes) and associability values (signaling attention allocation to the unexpected outcomes). Neural substrates of associability value and behavioral parameter estimates of associability updating, but not prediction error, increased with PTSD during loss learning. Moreover, the interaction of PTSD severity with neural markers of associability value predicted behavioral choices. These results indicate that increased attention- based learning may underlie aspects of PTSD and suggest potential neuromechanistic treatment targets. This work was supported in part by the National Institutes of Health (MH087692, MH106756 to PC), the Department of Veteran Affairs (D7030R to BKC), and the National Natural Science Foundation of China (31671171, 31630034 to LZ). We acknowledge Read Montague and the assistance of Rizwan Ali, George Christopoulos, Dongil Chung, Alec Solway, Jessica Eiseman, Katherine Gardner, David Graham, Jacob Lee, Katherine McCurry, Robert McNamara, Cari Rosoff, Dharol Tankersley, and Wright Williams.

Pearl Chiu, researcher at the Virginia Tech Carilion Research Institute, describes the use of behavioral economic games and functional magnetic resonance imaging in the exploration of social symptoms of autism and lists desired collaborations that may become possible with a Center for Autism Research.

The integrity of decision-making under emotionally evocative circumstances is critical to navigating complex environments, and dysfunctions in these processes may play an important role in the emergence and maintenance of various psychopathologies. The goal of the present study was to examine the spatial and temporal dynamics of neural responses to emotional stimuli and emotion-modulated response inhibition. High-density event-related brain potentials (ERPs) were measured as participants (N=25) performed an emotional Go/NoGo task that required button presses to words of a "target" emotional valence (i.e., positive, negative, neutral) and response inhibition to words of a different "distractor" valence. Using scalp ERP analyses in conjunction with source-localization techniques, we identified distinct neural responses associated with affective salience and affect- modulated response inhibition, respectively. Both earlier (similar to 300 ms) and later (similar to 700 ms) ERP components were enhanced with successful response inhibition to emotional distractors. Only ERPs to target stimuli differentiated affective from neutral cues. Moreover, Source localization analyses revealed right ventral lateral prefrontal cortex (VLPFC) activation in affective response inhibition regardless of emotional valence, whereas rostral anterior cingulate activation (rACC) was potentiated by emotional valence but was not modulated by response inhibition. This dissociation was supported by a significant Region x Trial Type x Emotion interaction, confirming that distinct regional dynamics characterize neural responses to affective valence and affective response-inhibition. The results are discussed in the context of an emerging affective neuroscience literature and implications for understanding psychiatric pathologies characterized by a detrimental susceptibility to emotional cues, with an emphasis on major depressive disorder., Psychology

Impaired social functioning conferred by mental illnesses has been constantly implicated in previous literatures. However, studies of social abnormalities in psychiatric conditions are often challenged by the difficulties of formalizing dynamic social exchanges and quantifying their neurocognitive underpinnings. Recently, the rapid growth of computational psychiatry as a new field along with the development of multiplayer economic paradigms provide powerful tools to parameterize complex interpersonal processes and identify quantitative indicators of social impairments. By utilizing these methodologies, the current set of studies aimed to examine social decision making during multiplayer economic games in participants diagnosed with depression (study 1) and combat-related post-traumatic stress disorder (PTSD, study 2), as well as an online population with elevated symptoms of borderline personality disorder (BPD, study 3). We then quantified and disentangled the impacts of multiple latent decision-making components, mainly social valuation and social learning, on maladaptive social behavior via explanatory modeling. Different underlying alterations were revealed across diagnoses. Atypical social exchange in depression and BPD were found attributed to altered social valuation and social learning respectively, whereas both social valuation and social learning contributed to interpersonal dysfunction in PTSD. Additionally, model-derived indices of social abnormalities positively correlated with levels of symptom severity (study 1 and 2) and exhibited a longitudinal association with symptom change (study 1). Our findings provided mechanistic insights into interpersonal difficulties in psychiatric illnesses, and highlighted the importance of a computational understanding of social function which holds potential clinical implications in differential diagnosis and precise treatment. Doctor of Philosophy People with psychiatric conditions often suffer from impaired social relationships due to an inability to engage in everyday social interactions. As different illnesses can sometimes produce the same symptoms, social impairment can also have different causes. For example, individuals who constantly avoid social activities may find them less interesting or attempt to avoid potential negative experiences. While those who display elevated aggression may have a strong desire for social dominance or falsely believe that others are also aggressive. However, it is hard to infer what drives these alterations by just observing the behavior. To address this question, we enrolled people with three different kinds of psychopathology to play an interactive game together with another player and mathematically modeled their latent decision-making processes. By comparing their model parameters to those of the control population, we were able to infer how people with psychopathology made the decisions and which part of the decision-making processes went wrong that led to disrupted social interactions. We found altered model parameters differed among people with major depression, post-traumatic stress disorder and borderline personality disorder, suggesting different causes underlying impaired social behavior observed in the game, the extent of which also positively correlated with their psychiatric symptom severity. Understanding the reasons behind social dysfunctions associated with psychiatric illnesses can help us better differentiate people with different diagnoses and design more effective treatments to restore interpersonal relationships.

Disrupted emotion processes are central features of posttraumatic stress disorder (PTSD) and major depressive disorder (MDD), which are linked to altered neural response patterns. However, inconsistent results have led to questions about the reliability of such findings. Heterogeneous clinical presentations across individuals with PTSD and MDD are likely to be associated with heterogeneous neurobehavioral changes which may differ depending on the emotion process studied. Similarly, neurobehavioral signatures of treatment response prediction may vary based on the task or context probed. In these studies, we examined how neuroimaging of emotion processes may shed light on mechanisms underlying symptom heterogeneity in PTSD (Study 1) and how similar neuroimaging signatures may be useful for predicting response to MDD treatment (Study 2). Results showed re-experiencing and hyperarousal symptoms had opponent effects on neural habituation to negative images, such that while increasing severity of hyperarousal symptoms was related to diminished habituation, increasing severity of re-experiencing symptoms was associated with enhanced habituation. Additionally, across MDD studies, two regions of the brain, the right anterior insula and the subgenual anterior cingulate cortex, exhibited pretreatment responses to negative emotional stimuli that were predictive of clinical response to treatment. Considered together, this work demonstrates the translational utility of neuroimaging of negative emotion processes to enhance our understanding of symptomatology and treatment prediction in PTSD and MDD. Ph.D. People who have posttraumatic stress disorder (PTSD) or depression often notice changes in the intensity and range of emotions they experience. These changes are thought to be related to differences in how the brain processes emotional information. Using neuroimaging to visualize changes that occur in the brains of individuals with PTSD or depression when they are experiencing negative emotions, we may gain a better understanding of how their symptoms are impacting them and how they may respond to different types of treatments. In these studies, I used brain imaging to measure responses to emotional images of people with and without PTSD. I found that certain PTSD symptoms affected the way people's brains responded over time to negative and neutral images. More several arousal symptoms were linked to less decreases of brain responses over time or less habituation. More severe symptoms of intrusive memories or distress when exposed to reminders of trauma were associated with greater decreases of brain responses to negative images. In a second study, I found that across studies of people with depression, two regions of the brain that are involved in emotion processing and stress responsivity, show pretreatment responses to negative emotional stimuli that are related to how they are likely to respond to treatment for depression. Overall, my research demonstrates how brain responses to negative emotions may be useful for understanding symptoms of mental health disorders and may help with predicting how individuals will respond to treatment.

It has been suggested that reward processing and related neural substrates are disrupted in some common mental disorders such as depression, addiction, and anxiety. An increasing number of psychiatric studies have been applying reinforcement learning (RL) models to examine these disruptions in self-regarding learning (learning about rewards delivered to the learners themselves). A review of RL alterations associated with mental disorders in extant studies will be beneficial for uncovering the mechanisms of these health problems. Although impaired social reward processing is common in some mental disorders [e.g., post-traumatic stress disorder (PTSD), social anxiety and autism], RL has not been widely used to detect the potentially disrupted social reward learning, especially for other-regarding learning (learning about rewards delivered to others). Meanwhile, it has not been clear whether some drugs, e.g., oxytocin (OT), can alter other-regarding learning, so they may serve as a therapeutic intervention when related deficits occur. In the present set of studies, we summarized common and distinct features in terms of self-regarding RL disturbances among depression, addiction and anxiety disorders based on previous findings (Paper I), tested whether behavioral and neural self- and other-regarding RL were impaired in PTSD with and without comorbid depression (Paper II), and investigated OT's behavioral and neural effects on self- and other-regarding RL in healthy males (Paper III). The results of our literature review showed that the commonalities in all three mental disorders were inflexibility and inconsistent choices, and the differences included decreased learning rates in depression, a higher weight to rewards versus punishments in addiction, and hypersensitivity to punishments in anxiety. The results of the PTSD study demonstrated impaired behavioral other-regarding learning in PTSD patients with and without depression, supposedly due to their hypervigilance to unexpected outcomes for others, as evidenced by the heightened responses in their inferior parietal lobule. The OT study detected OT's effects of attenuating behavioral other-regarding learning, as well as the neural coding of unexpected outcomes for others in the anterior cingulate cortex. These findings provide new evidence of self- and other-regarding RL alterations in mental disorders, reveal potential targets for their treatments, and bring caution for using OT as a therapeutic intervention. Doctor of Philosophy People learn to make choices to gain rewards and to avoid punishments delivered to themselves. As social animals, people also take account of outcomes delivered to others when learning. With the help of computational modeling, previous studies have found abnormal reward learning for oneself in people with mental health problems. To better understand mental illnesses, we summarized the similarities and differences of the learning abnormalities reported in previous studies about depression, addiction, and anxiety. We have found that people with these mental illnesses all tend to be inflexible and make more random choices when learning. As for the differences, people with depression tend to learn slower; people with addiction tend to see gaining rewards as more important than avoiding punishments; and people with anxiety tend to be oversensitive to punishments. Using computational modeling and imaging of brain function, we also tested whether learning for other was abnormal in post-traumatic stress disorder (PTSD), and found that, compared to healthy people, PTSD patients had slower learning for others' rewards, and the inferior parietal lobule, a brain region for processing social information, showed higher responses to unexpected outcomes for others. In another study, we examined whether oxytocin (OT), a neuropeptide that has been reported to change people's social functions, could influence reward learning for others in healthy males. The results showed that OT slowed down people's learning for others, and also decreased the neural learning signals in the anterior cingulate cortex, a region involved in processing other's outcomes. Our findings provide new information about how reward learning for oneself and others are changed in mental illnesses, reveal potential targets for their treatments, and bring caution for using OT as a therapeutic intervention.

Social Anxiety Disorder (SAD) is characterized by excessive concern or fear of negative evaluation in one or more social situations and ranks as one of the most common psychiatric disorders. SAD has also been characterized by significant deficits in social motivation and a lack of reactivity to pleasurable stimuli (i.e., positive affect; [PA]), particularly within social contexts. Recent neuroimaging work has shifted towards examining positively-valenced motivational systems in SAD focused on reward responses to social and nonsocial stimuli. These studies have revealed aberrant reward processing during social reward tasks in individuals with SAD. However, not all individuals with SAD exhibit reward circuitry dysfunction. Therefore, the current study aimed to examine if functional patterns of connectivity in the brain underlie heterogeneity in PA differences in individuals with SAD. Results revealed several functional connectivity strength differences between SAD and control groups within reward regions. Additionally, associations between regions of interest (ROIs)-couplings (i.e., OFC and insula, OFC and subgenual cingulate, insula and cingulate, and cingulate and subgenual cingulate) and diminished PA were present in individuals with SAD, but not controls. Lastly, results demonstrated that individuals with SAD had higher variability in their reward connectivity strength presentations and reports of PA as compared to controls. These results hold significance for the development of interventions for SAD that focus on the enhancement of PA to bolster social reward responsivity. M.S. Social Anxiety Disorder (SAD) is a common disorder where individuals experience persistent excessive fear of one or more social situations. Individuals with SAD also tend to show lower social motivation and a lack of reactivity to pleasurable activities/events (referred to broadly as positive affect; [PA]), particularly within social situations. Current work has focused on areas within the brain that are responsible for reward responses, and have indicated that individuals with SAD show different types of reward processing during social reward situations. However, not all individuals with SAD show these same patterns. Therefore, the current study aimed to examine if connections between reward regions in the brain underlie differences in PA differences in individuals with SAD. Results showed several differences between SAD and control groups within reward regions of the brain. Additionally, specific associations between brain regions of interest and low PA were present in individuals with SAD, but not controls. Lastly, results demonstrated that individuals with SAD had higher variability in their connections between reward regions and reports of PA as compared to controls. These results can help inform the development of treatments for SAD that focus on the improving PA in an attempt to increase responsiveness to social rewards.

Anxiety is characterized by apprehensive expectation regarding the forecasted outcomes of choice. Decision science and in particular reinforcement learning models provide a quantitative framework to explain how the likelihood and value of such outcomes are estimated, thus allowing the measurement of parameters of decision-making that may differ between high- and low- anxiety groups. However, the role of anxiety in choice allocation is not sufficiently understood, particularly regarding the influence of transient threat on current decisions. The presence of threat appears to alter choice behavior and may differentially influence quantitatively derived parameters of learning among anxious individuals. Regarding the neurobiology of reinforcement learning, the dorsolateral prefrontal cortex (dlPFC) has been suggested to play a role in temporally integrating experienced outcomes, as well as in coordinating an overall choice action plan, both of which can be described computationally by learning rate and exploration, respectively. Accordingly, it was hypothesized that high trait anxiety would be associated with a lower reward learning rate, a higher loss learning rate, and diminished exploration of available options, and furthermore that threat would increase the magnitude of these parameters in the high anxiety group. We also hypothesized that the magnitude of neural activation (measured by functional near-infrared spectroscopy; FNIRS) across dissociable regions of the left and right dlPFC would be associated with model parameters, and that threat would further increase the magnitude of activation to model parameters. Finally, it was hypothesized that reward and loss outcomes could be differentiated based on FNIRS channel activation, and that a distinct set of channels would differentiate outcomes in high relative to low anxiety groups. To test these hypotheses, a temporal difference learning model was applied to a decision-making (bandit) task to establish differences in learning parameter magnitudes among individuals high (N=26) and low (N=20) in trait anxiety, as well as the impact of threat on learning parameters. Results indicated a positive association between anxiety and both the reward and loss learning rate parameters. However, threat was not found to impact model parameters. Imaging results indicated a positive association between exploration and the left dlPFC. Reward and loss outcomes were successfully differentiated in the high, but not low anxiety group. Results add to a growing literature suggesting anxiety is characterized by differential sensitivity to both losses and rewards in reinforcement learning contexts, and further suggests that the dlPFC plays a role in modulating exploration-based choice strategies. Doctor of Philosophy Anxiety is characterized by worry about possible future negative outcomes. Mathematical models in the area of learning theory allow the representation and measurement of individual differences in decision-making tendencies that contribute to negative future apprehension. Currently, the role of anxiety in the allocation of choices, and particularly the influence of threat on decision-making is poorly understood. Threat may influence learning and alter choice behavior, collectively causing negative future apprehension. With regards to how related decision-making is computed in the brain, the dorsolateral prefrontal cortex (dlPFC) has been suggested to play a role tracking and integrating current and past experienced outcomes, in order to coordinate an overall action plan. Outcome tracking and action plan coordination can be represented mathematically within a learning theory framework by learning rate and exploration parameters, respectively. It was hypothesized that high anxiety would be associated with a lower reward learning rate, a higher loss learning rate, and diminished exploration, and furthermore that threat would increase the magnitude of these tendencies in anxious individuals. We also hypothesized that brain activation in the dlPFC would be associated with these tendencies, and that threat would further increase activation in these brain areas. It was also hypothesized that reward and loss outcomes could be differentiated based on brain activation in the dlPFC. To test these hypotheses, a mathematical model was applied to establish differences in learning within high and low anxiety individuals, as well as to test the impact of threat on these learning tendencies. Results indicated a positive association between anxiety and the rate of learning to reward and loss outcomes. Threat was not found to impact these learning rates. A positive association was found between activation in the dlPFC and the tendency to explore. Reward and loss outcomes were successfully differentiated based on brain activation in high, but not low anxiety individuals. Results add to a growing literature suggesting that anxiety is characterized by differential sensitivity to both losses and rewards, and further adds to our understanding of how the brain computes exploration-based choice strategies.

Major depressive disorder is a common, impairing disease, but current treatments are only moderately effective. Understanding how processes such as reward and punishment learning are disrupted in depression and how these disruptions are remediated through treatment is vital to improving outcomes for people with this disorder. In the present set of studies, computational reinforcement learning models and neuroimaging were used to understand how symptom clusters of depression (anhedonia and negative affect) were related to neural and behavioral measures of learning (Study 1, in Paper 1), how these alterations changed with improvement in symptoms after cognitive behavioral therapy (Study 2, in Paper 1), and how learning parameters could be directly altered in a learning retraining paradigm (Study 3, in Paper 2). Results showed that anhedonia and negative affect were uniquely related to changes in learning and that improvement in these symptoms correlated with changes in learning parameters; these parameters could also be changed through targeted queries based on reinforcement learning theory. These findings add important information to how learning is disrupted in depression and how current and novel treatments can remediate learning and improve symptoms. Ph. D.

Adolescence is a critical developmental period characterized by neurobiological changes and exposure to novel experiences. According to the Center for Disease Control, approximately 70% of adolescent deaths in the United States are due to risky behaviors such as reckless driving and risky sexual behavior (Kann et al., 2016). In order to better understand what drives adolescent risk-taking, the current studies utilized an interdisciplinary approach, which combined behavioral economic models and functional magnetic resonance imaging (fMRI) to understand neurobehavioral mechanisms of risky choice. The focus of the current studies is to investigate the extent to which neurobehavioral mechanisms of risky choice change across adolescence, and to identify individual differences that explain real-world risky behavior. In Study 1, we show that behavioral sensitivity to risk and neural correlates of risk processing change across a critical period of adolescence. Importantly, our results indicate that individual differences in neural, not behavioral risk sensitivity are predictive of future engagement in health risk behaviors. In Study 2, we examined the relation between inter-individual differences in adolescent expectations of valued rewards and self-reported risky behavior using an adapted behavioral economic model. Implications and future directions for adolescent risky decision-making are discussed. Ph. D.

Chronic nicotine use has been linked to increased sensitivity to nondrug rewards as well as improvement in mood among individuals with depression, and these effects have been hypothesized to be mediated through alternations in striatal dopamine activity. Similarly, chronic nicotine use is hypothesized to influence the mechanisms by which healthy and depressed individuals learn about rewards in their environment. However, the specific behavioral and neural mechanisms by which nicotine influences the learning process is poorly understood. Here, we use a probabilistic learning task, functional magnetic resonance imaging and neurocomputational analyses, to show that chronic smoking is associated with higher reward sensitivity, along with lower learning rate and striatal prediction error signal. Further, we show that these effects do not differ between individuals with and without major depressive disorder (MDD). In addition, a negative correlation between reward sensitivity and striatal prediction error signal was found among smokers, consistent with the suggestion that enhanced tonic dopamine associated with increased reward sensitivity leads to an attenuation of phasic dopamine activity necessary for updating of reward value during learning. Master of Science