Search

Forward planning is crucial to maximize outcome in complex sequential decision-making scenarios. In this cross-sectional study, we were particularly interested in age-related differences of forward planning. We presumed that especially older individuals would show a shorter planning depth to keep the costs of model-based decision-making within limits. To test this hypothesis, we developed a sequential decision-making task to assess forward planning in younger (age < 40 years; n = 25) and older (age > 60 years; n = 27) adults. By using reinforcement learning modelling, we inferred planning depths from participants' choices. Our results showed significantly shorter planning depths and higher response noise for older adults. Age differences in planning depth were only partially explained by well-known cognitive covariates such as working memory and processing speed. Consistent with previous findings, this indicates age-related shifts away from model-based behaviour in older adults. In addition to a shorter planning depth, our findings suggest that older adults also apply a variety of heuristical low-cost strategies., (© 2023. The Author(s).)

Human behavior consists in large parts of action sequences that are often repeated in mostly the same way. Through extensive repetition, sequential responses become automatic or habitual, but our environment often confronts us with events to which we have to react flexibly and in a goal-directed manner. To assess how implicitly learned action sequences interfere with goal-directed control, we developed a novel behavioral paradigm in which we combined action sequence learning through repetition with a goal-directed task component. So-called dual-target trials require the goal-directed selection of the response with the highest reward probability in a fast succession of trials with short response deadlines. Importantly, the response primed by the learned action sequence is sometimes different from that required by the goal-directed task. As expected, we found that participants learned the action sequence through repetition, as evidenced by reduced reaction times (RT) and error rates (ER), while still acting in a goal-directed manner in dual-target trials. Specifically, we found that the learned action sequence biased choices in the goal-directed task toward the sequential response, and this effect was more pronounced the better individuals had learned the sequence. Our novel task may help shed light on the acquisition of automatic behavioral patterns and habits through extensive repetition, allows to assess positive features of habitual behavior (e.g., increased response speed and reduced error rates), and importantly also the interaction of habitual and goal-directed behaviors under time pressure., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Frölich, Esmeyer, Endrass, Smolka and Kiebel.)

Spontaneous correlated activity is a universal hallmark of immature neural circuits. However, the cellular dynamics and intrinsic mechanisms underlying network burstiness in the intact developing brain are largely unknown. Here, we use two-photon Ca 2+ imaging to comprehensively map the developmental trajectories of spontaneous network activity in the hippocampal area CA1 of mice in vivo. We unexpectedly find that network burstiness peaks after the developmental emergence of effective synaptic inhibition in the second postnatal week. We demonstrate that the enhanced network burstiness reflects an increased functional coupling of individual neurons to local population activity. However, pairwise neuronal correlations are low, and network bursts (NBs) recruit CA1 pyramidal cells in a virtually random manner. Using a dynamic systems modeling approach, we reconcile these experimental findings and identify network bi-stability as a potential regime underlying network burstiness at this age. Our analyses reveal an important role of synaptic input characteristics and network instability dynamics for NB generation. Collectively, our data suggest a mechanism, whereby developing CA1 performs extensive input-discrimination learning prior to the onset of environmental exploration., Competing Interests: JG, VR, MM, OW, CG, SK, KH, KK No competing interests declared, (© 2022, Graf, Rahmati et al.)

Precisely timed behavior and accurate time perception plays a critical role in our everyday lives, as our wellbeing and even survival can depend on well-timed decisions. Although the temporal structure of the world around us is essential for human decision making, we know surprisingly little about how representation of temporal structure of our everyday environment impacts decision making. How does the representation of temporal structure affect our ability to generate well-timed decisions? Here we address this question by using a well-established dynamic probabilistic learning task. Using computational modeling, we found that human subjects' beliefs about temporal structure are reflected in their choices to either exploit their current knowledge or to explore novel options. The model-based analysis illustrates a large within-group and within-subject heterogeneity. To explain these results, we propose a normative model for how temporal structure is used in decision making, based on the semi-Markov formalism in the active inference framework. We discuss potential key applications of the presented approach to the fields of cognitive phenotyping and computational psychiatry., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Marković, Reiter and Kiebel.)

Cognitive control and forward planning in particular is costly, and therefore must be regulated such that the amount of cognitive resources invested is adequate to the current situation. However, knowing in advance how beneficial forward planning will be in a given situation is hard. A way to know the exact value of planning would be to actually do it, which would ab initio defeat the purpose of regulating planning, i.e. the reduction of computational and time costs. One possible solution to this dilemma is that planning is regulated by learned associations between stimuli and the expected demand for planning. Such learning might be based on generalisation processes that cluster together stimulus states with similar control relevant properties into more general control contexts. In this way, the brain could infer the demand for planning, based on previous experience with situations that share some structural properties with the current situation. Here, we used a novel sequential task to test the hypothesis that people use control contexts to efficiently regulate their forward planning, using behavioural and functional magnetic resonance imaging data. Consistent with our hypothesis, reaction times increased with trial-by-trial conflict, where this increase was more pronounced in a context with a learned high demand for planning. Similarly, we found that fMRI activity in the dorsal anterior cingulate cortex (dACC) increased with conflict, and this increase was more pronounced in a context with generally high demand for planning. Taken together, the results indicate that the dACC integrates representations of planning demand at different levels of abstraction to regulate planning in an efficient and situation-appropriate way., Competing Interests: Declaration of interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022. Published by Elsevier Inc.)

In the study of perceptual decision making, it has been widely assumed that random fluctuations of motion stimuli are irrelevant for a participant's choice. Recently, evidence was presented that these random fluctuations have a measurable effect on the relationship between neuronal and behavioral variability, the so-called choice probability. Here, we test, in a behavioral experiment, whether stochastic motion stimuli influence the choices of human participants. Our results show that for specific stochastic motion stimuli, participants indeed make biased choices, where the bias is consistent over participants. Using a computational model, we show that this consistent choice bias is caused by subtle motion information contained in the motion noise. We discuss the implications of this finding for future studies of perceptual decision making. Specifically, we suggest that future experiments should be complemented with a stimulus-informed modeling approach to control for the effects of apparent decision evidence in random stimuli., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Fard, Bitzer, Pannasch and Kiebel.)

Alcohol use disorder (AUD) is characterized by a combination of symptoms including excessive craving, loss of control, and progressive neglect of alternative pleasures. A mechanistic understanding of what drives these symptoms is needed to improve diagnostic stratification and to develop new treatment and prevention strategies for AUD. To date, there is no consensus regarding a unifying mechanistic framework that accounts for the different symptoms of AUD. Reinforcement learning (RL) and economic choice theories may be key to elucidating the underlying processes of symptom development and maintenance in AUD. These algorithms may account for the different behavioral and physiological phenomena and are suited to dissect mechanisms linked to different symptoms of AUD. We here review different RL and economic choice models and how they map onto three symptoms of AUD: (1) cue-induced craving, (2) neglect of alternative rewards, and (3) consumption despite adverse consequences. For each symptom and theory, we describe findings from animal and human studies. In humans, we focus on empirical studies that investigated RL models in the context of treatment outcome in AUD. The review indicates important gaps to be addressed in the future by highlighting the challenges in transferring findings from RL and economic choice studies to clinical application. We also critically evaluate the potential and pitfalls of a symptom-oriented approach and highlight the importance of elucidating the role of learning and decision-making processes across diagnostic boundaries., (© 2022 S. Karger AG, Basel.)

A key feature of sequential decision making under uncertainty is a need to balance between exploiting-choosing the best action according to the current knowledge, and exploring-obtaining information about values of other actions. The multi-armed bandit problem, a classical task that captures this trade-off, served as a vehicle in machine learning for developing bandit algorithms that proved to be useful in numerous industrial applications. The active inference framework, an approach to sequential decision making recently developed in neuroscience for understanding human and animal behaviour, is distinguished by its sophisticated strategy for resolving the exploration-exploitation trade-off. This makes active inference an exciting alternative to already established bandit algorithms. Here we derive an efficient and scalable approximate active inference algorithm and compare it to two state-of-the-art bandit algorithms: Bayesian upper confidence bound and optimistic Thompson sampling. This comparison is done on two types of bandit problems: a stationary and a dynamic switching bandit. Our empirical evaluation shows that the active inference algorithm does not produce efficient long-term behaviour in stationary bandits. However, in the more challenging switching bandit problem active inference performs substantially better than the two state-of-the-art bandit algorithms. The results open exciting venues for further research in theoretical and applied machine learning, as well as lend additional credibility to active inference as a general framework for studying human and animal behaviour., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2021 The Author(s). Published by Elsevier Ltd.. All rights reserved.)

Cognitive control is typically understood as a set of mechanisms that enable humans to reach goals that require integrating the consequences of actions over longer time scales. Importantly, using routine behaviour or making choices beneficial only at short time scales would prevent one from attaining these goals. During the past two decades, researchers have proposed various computational cognitive models that successfully account for behaviour related to cognitive control in a wide range of laboratory tasks. As humans operate in a dynamic and uncertain environment, making elaborate plans and integrating experience over multiple time scales is computationally expensive. Importantly, it remains poorly understood how uncertain consequences at different time scales are integrated into adaptive decisions. Here, we pursue the idea that cognitive control can be cast as active inference over a hierarchy of time scales, where inference, i.e., planning, at higher levels of the hierarchy controls inference at lower levels. We introduce the novel concept of meta-control states, which link higher-level beliefs with lower-level policy inference. Specifically, we conceptualize cognitive control as inference over these meta-control states, where solutions to cognitive control dilemmas emerge through surprisal minimisation at different hierarchy levels. We illustrate this concept using the exploration-exploitation dilemma based on a variant of a restless multi-armed bandit task. We demonstrate that beliefs about contexts and meta-control states at a higher level dynamically modulate the balance of exploration and exploitation at the lower level of a single action. Finally, we discuss the generalisation of this meta-control concept to other control dilemmas.

Various imaging and electrophysiological studies in a number of different species and brain regions have revealed that neuronal dynamics associated with diverse behavioral patterns and cognitive tasks take on a sequence-like structure, even when encoding stationary concepts. These neuronal sequences are characterized by robust and reproducible spatiotemporal activation patterns. This suggests that the role of neuronal sequences may be much more fundamental for brain function than is commonly believed. Furthermore, the idea that the brain is not simply a passive observer but an active predictor of its sensory input, is supported by an enormous amount of evidence in fields as diverse as human ethology and physiology, besides neuroscience. Hence, a central aspect of this review is to illustrate how neuronal sequences can be understood as critical for probabilistic predictive information processing, and what dynamical principles can be used as generators of neuronal sequences. Moreover, since different lines of evidence from neuroscience and computational modeling suggest that the brain is organized in a functional hierarchy of time scales, we will also review how models based on sequence-generating principles can be embedded in such a hierarchy, to form a generative model for recognition and prediction of sensory input. We shortly introduce the Bayesian brain hypothesis as a prominent mathematical description of how online, i.e., fast, recognition, and predictions may be computed by the brain. Finally, we briefly discuss some recent advances in machine learning, where spatiotemporally structured methods (akin to neuronal sequences) and hierarchical networks have independently been developed for a wide range of tasks. We conclude that the investigation of specific dynamical and structural principles of sequential brain activity not only helps us understand how the brain processes information and generates predictions, but also informs us about neuroscientific principles potentially useful for designing more efficient artificial neuronal networks for machine learning tasks., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Frölich, Marković and Kiebel.)

NKCC1 is the primary transporter mediating chloride uptake in immature principal neurons, but its role in the development of in vivo network dynamics and cognitive abilities remains unknown. Here, we address the function of NKCC1 in developing mice using electrophysiological, optical, and behavioral approaches. We report that NKCC1 deletion from telencephalic glutamatergic neurons decreases in vitro excitatory actions of γ-aminobutyric acid (GABA) and impairs neuronal synchrony in neonatal hippocampal brain slices. In vivo, it has a minor impact on correlated spontaneous activity in the hippocampus and does not affect network activity in the intact visual cortex. Moreover, long-term effects of the developmental NKCC1 deletion on synaptic maturation, network dynamics, and behavioral performance are subtle. Our data reveal a neural network function of NKCC1 in hippocampal glutamatergic neurons in vivo, but challenge the hypothesis that NKCC1 is essential for major aspects of hippocampal development., Competing Interests: The authors declare no competing interest.

Background: Individuals tend to avoid effortful tasks, regardless of whether they are physical or mental in nature. Recent experimental evidence is suggestive of individual differences in the dispositional willingness to invest cognitive effort in goal-directed behavior. The traits need for cognition (NFC) and self-control are related to behavioral measures of cognitive effort discounting and demand avoidance, respectively. Given that these traits are only moderately related, the question arises whether they reflect a common core factor underlying cognitive effort investment. If so, the common core of both traits might be related to behavioral measures of effort discounting in a more systematic fashion. To address this question, we aimed at specifying a core construct of cognitive effort investment that reflects dispositional differences in the willingness and tendency to exert effortful control., Methods: We conducted two studies (N = 613 and N = 244) with questionnaires related to cognitive motivation and effort investment including assessment of NFC, intellect, self-control and effortful control. We first calculated Pearson correlations followed by two mediation models regarding intellect and its separate aspects, seek and conquer, as mediators. Next, we performed a confirmatory factor analysis of a hierarchical model of cognitive effort investment as second-order latent variable. First-order latent variables were cognitive motivation reflecting NFC and intellect, and effortful self-control reflecting self-control and effortful control. Finally, we calculated Pearson correlations between factor scores of the latent variables and general self-efficacy as well as traits of the Five Factor Model of Personality for validation purposes., Results: Our findings support the hypothesized correlations between the assessed traits, where the relationship of NFC and self-control is specifically mediated via goal-directedness. We established and replicated a hierarchical factor model of cognitive motivation and effortful self-control that explains the shared variance of the first-order factors by a second-order factor of cognitive effort investment., Conclusions: Taken together, our results integrate disparate literatures on cognitive motivation and self-control and provide a basis for further experimental research on the role of dispositional individual differences in goal-directed behavior and cost-benefit-models.

Individuals tend to avoid cognitive demand, yet, individual differences appear to exist. Recent evidence from two studies suggests that individuals high in the personality traits Self-Control and Need for Cognition that are related to the broader construct Cognitive Effort Investment are less prone to avoid cognitive demand and show less effort discounting. These findings suggest that cost-benefit models of decision-making that integrate the costs due to effort should consider individual differences in the willingness to exert mental effort. However, to date, there are almost no replication attempts of the above findings. For the present conceptual replication, we concentrated on the avoidance of cognitive demand and used a longitudinal design and latent state-trait modeling. This approach enabled us to separate the trait-specific variance in our measures of Cognitive Effort Investment and Demand Avoidance that is due to stable, individual differences from the variance that is due to the measurement occasion, the methods used, and measurement error. Doing so allowed us to test the assumption that self-reported Cognitive Effort Investment is related to behavioral Demand Avoidance more directly by relating their trait-like features to each other. In a sample of N = 217 participants, we observed both self-reported Cognitive Effort Investment and behavioral Demand Avoidance to exhibit considerable portions of trait variance. However, these trait variances were not significantly related to each other. Thus, our results call into question previous findings of a relationship between self-reported effort investment and demand avoidance. We suggest that novel paradigms are needed to emulate real-world effortful situations and enable better mapping between self-reported measures and behavioral markers of the willingness to exert cognitive effort., Competing Interests: The authors have declared that no competing interests exist.

Most rewards in our lives require effort to obtain them. It is known that effort is seen by humans as carrying an intrinsic disutility which devalues the obtainable reward. Established models for effort discounting account for this by using participant-specific discounting parameters inferred from experiments. These parameters offer only a static glance into the bigger picture of effort exertion. The mechanism underlying the dynamic changes in a participant's willingness to exert effort is still unclear and an active topic of research. Here, we modeled dynamic effort exertion as a consequence of effort- and probability-discounting mechanisms during goal reaching, sequential behavior. To do this, we developed a novel sequential decision-making task in which participants made binary choices to reach a minimum number of points. Importantly, the time points and circumstances of effort allocation were decided by participants according to their own preferences and not imposed directly by the task. Using the computational model to analyze participants' choices, we show that the dynamics of effort exertion arise from a combination of changing task needs and forward planning. In other words, the interplay between a participant's inferred discounting parameters is sufficient to explain the dynamic allocation of effort during goal reaching. Using formal model comparison, we also inferred the forward-planning strategy used by participants. The model allowed us to characterize a participant's effort exertion in terms of only a few parameters. Moreover, the model can be adapted to a number of tasks used in establishing the neural underpinnings of forward-planning behavior and meta-control, allowing for the characterization of behavior in terms of model parameters., (Copyright © 2020 Cuevas Rivera, Strobel, Goschke and Kiebel.)

Selecting goals and successfully pursuing them in an uncertain and dynamic environment is an important aspect of human behaviour. In order to decide which goal to pursue at what point in time, one has to evaluate the consequences of one's actions over future time steps by forward planning. However, when the goal is still temporally distant, detailed forward planning can be prohibitively costly. One way to select actions at minimal computational costs is to use heuristics. It is an open question how humans mix heuristics with forward planning to balance computational costs with goal reaching performance. To test a hypothesis about dynamic mixing of heuristics with forward planning, we used a novel stochastic sequential two-goal task. Comparing participants' decisions with an optimal full planning agent, we found that at the early stages of goal-reaching sequences, in which both goals are temporally distant and planning complexity is high, on average 42% (SD = 19%) of participants' choices deviated from the agent's optimal choices. Only towards the end of the sequence, participant's behaviour converged to near optimal performance. Subsequent model-based analyses showed that participants used heuristic preferences when the goal was temporally distant and switched to forward planning when the goal was close., Competing Interests: The authors have declared that no competing interests exist.

In perceptual decision making the brain extracts and accumulates decision evidence from a stimulus over time and eventually makes a decision based on the accumulated evidence. Several characteristics of this process have been observed in human electrophysiological experiments, especially an average build-up of motor-related signals supposedly reflecting accumulated evidence, when averaged across trials. Another recently established approach to investigate the representation of decision evidence in brain signals is to correlate the within-trial fluctuations of decision evidence with the measured signals. We here report results of this approach for a two-alternative forced choice reaction time experiment measured using magnetoencephalography (MEG) recordings. Our results show: (1) that decision evidence is most strongly represented in the MEG signals in three consecutive phases and (2) that posterior cingulate cortex is involved most consistently, among all brain areas, in all three of the identified phases. As most previous work on perceptual decision making in the brain has focused on parietal and motor areas, our findings therefore suggest that the role of the posterior cingulate cortex in perceptual decision making may be currently underestimated., (Copyright © 2020 Bitzer, Park, Maess, von Kriegstein and Kiebel.)

Synchronized activity is a universal characteristic of immature neural circuits that is essential for their developmental refinement and strongly depends on GABAergic neurotransmission. A major subpopulation of GABA-releasing interneurons (INs) expresses somatostatin (SOM) and proved critical for rhythm generation in adulthood. Here, we report a mechanism whereby SOM INs promote neuronal synchrony in the neonatal CA1 region. Combining imaging and electrophysiological approaches, we demonstrate that SOM INs and pyramidal cells (PCs) coactivate during spontaneous activity. Bidirectional optogenetic manipulations reveal excitatory GABAergic outputs to PCs that evoke correlated network events in an NKCC1-dependent manner and contribute to spontaneous synchrony. Using a dynamic systems modeling approach, we show that SOM INs affect network dynamics through a modulation of network instability and amplification threshold. Our study identifies a network function of SOM INs with implications for the activity-dependent construction of developing brain circuits., (Copyright © 2019 The Author(s). Published by Elsevier Inc. All rights reserved.)

Neuronal computations rely upon local interactions across synapses. For a neuronal network to perform inference, it must integrate information from locally computed messages that are propagated among elements of that network. We review the form of two popular (Bayesian) message passing schemes and consider their plausibility as descriptions of inference in biological networks. These are variational message passing and belief propagation - each of which is derived from a free energy functional that relies upon different approximations (mean-field and Bethe respectively). We begin with an overview of these schemes and illustrate the form of the messages required to perform inference using Hidden Markov Models as generative models. Throughout, we use factor graphs to show the form of the generative models and of the messages they entail. We consider how these messages might manifest neuronally and simulate the inferences they perform. While variational message passing offers a simple and neuronally plausible architecture, it falls short of the inferential performance of belief propagation. In contrast, belief propagation allows exact computation of marginal posteriors at the expense of the architectural simplicity of variational message passing. As a compromise between these two extremes, we offer a third approach - marginal message passing - that features a simple architecture, while approximating the performance of belief propagation. Finally, we link formal considerations to accounts of neurological and psychiatric syndromes in terms of aberrant message passing.

In our daily lives timing of our actions plays an essential role when we navigate the complex everyday environment. It is an open question though how the representations of the temporal structure of the world influence our behavior. Here we propose a probabilistic model with an explicit representation of state durations which may provide novel insights in how the brain predicts upcoming changes. We illustrate several properties of the behavioral model using a standard reversal learning design and compare its task performance to standard reinforcement learning models. Furthermore, using experimental data, we demonstrate how the model can be applied to identify participants' beliefs about the latent temporal task structure. We found that roughly one quarter of participants seem to have learned the latent temporal structure and used it to anticipate changes, whereas the remaining participants' behavior did not show signs of anticipatory responses, suggesting a lack of precise temporal expectations. We expect that the introduced behavioral model will allow, in future studies, for a systematic investigation of how participants learn the underlying temporal structure of task environments and how these representations shape behavior., Competing Interests: The authors have declared that no competing interests exist.

Most research on risk aversion in behavioral science with human subjects has focused on a component of risk aversion that does not adapt itself to context. More recently, studies have explored risk aversion adaptation to changing circumstances in sequential decision-making tasks. It is an open question whether one can identify evidence, at the single subject level, for such risk aversion adaptation. We conducted a behavioral experiment on human subjects, using a sequential decision making task. We developed a model-based approach for estimating the adaptation of risk-taking behavior with single-trial resolution by modeling a subject's goals and internal representation of task contingencies. Using this model-based approach, we estimated the subject-specific adaptation of risk aversion depending on the current task context. We found striking inter-subject variations in the adaptation of risk-taking behavior. We show that these differences can be explained by differences in subjects' internal representations of task contingencies and goals. We discuss that the proposed approach can be adapted to a wide range of experimental paradigms and be used to analyze behavioral measures other than risk aversion.

Calcium imaging provides an indirect observation of the underlying neural dynamics and enables the functional analysis of neuronal populations. However, the recorded fluorescence traces are temporally smeared, thus making the reconstruction of exact spiking activity challenging. Most of the established methods to tackle this issue are limited in dealing with issues such as the variability in the kinetics of fluorescence transients, fast processing of long-term data, high firing rates, and measurement noise. We propose a novel, heuristic reconstruction method to overcome these limitations. By using both synthetic and experimental data, we demonstrate the four main features of this method: 1) it accurately reconstructs both isolated spikes and within-burst spikes, and the spike count per fluorescence transient, from a given noisy fluorescence trace; 2) it performs the reconstruction of a trace extracted from 1,000,000 frames in less than 2 s; 3) it adapts to transients with different rise and decay kinetics or amplitudes, both within and across single neurons; and 4) it has only one key parameter, which we will show can be set in a nearly automatic way to an approximately optimal value. Furthermore, we demonstrate the ability of the method to effectively correct for fast and rather complex, slowly varying drifts as frequently observed in in vivo data. NEW & NOTEWORTHY Reconstruction of spiking activities from calcium imaging data remains challenging. Most of the established reconstruction methods not only have limitations in adapting to systematic variations in the data and fast processing of large amounts of data, but their results also depend on the user's experience. To overcome these limitations, we present a novel, heuristic model-free-type method that enables an ultra-fast, accurate, near-automatic reconstruction from data recorded under a wide range of experimental conditions.

During neocortical development, network activity undergoes a dramatic transition from largely synchronized, so-called cluster activity, to a relatively sparse pattern around the time of eye-opening in rodents. Biophysical mechanisms underlying this sparsification phenomenon remain poorly understood. Here, we present a dynamic systems modeling study of a developing neural network that provides the first mechanistic insights into sparsification. We find that the rest state of immature networks is strongly affected by the dynamics of a transient, unstable state hidden in their firing activities, allowing these networks to either be silent or generate large cluster activity. We address how, and which, specific developmental changes in neuronal and synaptic parameters drive sparsification. We also reveal how these changes refine the information processing capabilities of an in vivo developing network, mainly by showing a developmental reduction in the instability of network's firing activity, an effective availability of inhibition-stabilized states, and an emergence of spontaneous attractors and state transition mechanisms. Furthermore, we demonstrate the key role of GABAergic transmission and depressing glutamatergic synapses in governing the spatiotemporal evolution of cluster activity. These results, by providing a strong link between experimental observations and model behavior, suggest how adult sparse coding networks may emerge developmentally.

Perceptual decision making can be described as a process of accumulating evidence to a bound which has been formalized within drift-diffusion models (DDMs). Recently, an equivalent Bayesian model has been proposed. In contrast to standard DDMs, this Bayesian model directly links information in the stimulus to the decision process. Here, we extend this Bayesian model further and allow inter-trial variability of two parameters following the extended version of the DDM. We derive parameter distributions for the Bayesian model and show that they lead to predictions that are qualitatively equivalent to those made by the extended drift-diffusion model (eDDM). Further, we demonstrate the usefulness of the extended Bayesian model (eBM) for the analysis of concrete behavioral data. Specifically, using Bayesian model selection, we find evidence that including additional inter-trial parameter variability provides for a better model, when the model is constrained by trial-wise stimulus features. This result is remarkable because it was derived using just 200 trials per condition, which is typically thought to be insufficient for identifying variability parameters in DDMs. In sum, we present a Bayesian analysis, which provides for a novel and promising analysis of perceptual decision making experiments.

Probabilistic models of decision making under various forms of uncertainty have been applied in recent years to numerous behavioral and model-based fMRI studies. These studies were highly successful in enabling a better understanding of behavior and delineating the functional properties of brain areas involved in decision making under uncertainty. However, as different studies considered different models of decision making under uncertainty, it is unclear which of these computational models provides the best account of the observed behavioral and neuroimaging data. This is an important issue, as not performing model comparison may tempt researchers to over-interpret results based on a single model. Here we describe how in practice one can compare different behavioral models and test the accuracy of model comparison and parameter estimation of Bayesian and maximum-likelihood based methods. We focus our analysis on two well-established hierarchical probabilistic models that aim at capturing the evolution of beliefs in changing environments: Hierarchical Gaussian Filters and Change Point Models. To our knowledge, these two, well-established models have never been compared on the same data. We demonstrate, using simulated behavioral experiments, that one can accurately disambiguate between these two models, and accurately infer free model parameters and hidden belief trajectories (e.g., posterior expectations, posterior uncertainties, and prediction errors) even when using noisy and highly correlated behavioral measurements. Importantly, we found several advantages of Bayesian inference and Bayesian model comparison compared to often-used Maximum-Likelihood schemes combined with the Bayesian Information Criterion. These results stress the relevance of Bayesian data analysis for model-based neuroimaging studies that investigate human decision making under uncertainty.

Calcium imaging has been used as a promising technique to monitor the dynamic activity of neuronal populations. However, the calcium trace is temporally smeared which restricts the extraction of quantities of interest such as spike trains of individual neurons. To address this issue, spike reconstruction algorithms have been introduced. One limitation of such reconstructions is that the underlying models are not informed about the biophysics of spike and burst generations. Such existing prior knowledge might be useful for constraining the possible solutions of spikes. Here we describe, in a novel Bayesian approach, how principled knowledge about neuronal dynamics can be employed to infer biophysical variables and parameters from fluorescence traces. By using both synthetic and in vitro recorded fluorescence traces, we demonstrate that the new approach is able to reconstruct different repetitive spiking and/or bursting patterns with accurate single spike resolution. Furthermore, we show that the high inference precision of the new approach is preserved even if the fluorescence trace is rather noisy or if the fluorescence transients show slow rise kinetics lasting several hundred milliseconds, and inhomogeneous rise and decay times. In addition, we discuss the use of the new approach for inferring parameter changes, e.g. due to a pharmacological intervention, as well as for inferring complex characteristics of immature neuronal circuits.

Decisions in everyday life are prone to error. Standard models typically assume that errors during perceptual decisions are due to noise. However, it is unclear how noise in the sensory input affects the decision. Here we show that there are experimental tasks for which one can analyse the exact spatio-temporal details of a dynamic sensory noise and better understand variability in human perceptual decisions. Using a new experimental visual tracking task and a novel Bayesian decision making model, we found that the spatio-temporal noise fluctuations in the input of single trials explain a significant part of the observed responses. Our results show that modelling the precise internal representations of human participants helps predict when perceptual decisions go wrong. Furthermore, by modelling precisely the stimuli at the single-trial level, we were able to identify the underlying mechanism of perceptual decision making in more detail than standard models.

For making decisions in everyday life we often have first to infer the set of environmental features that are relevant for the current task. Here we investigated the computational mechanisms underlying the evolution of beliefs about the relevance of environmental features in a dynamical and noisy environment. For this purpose we designed a probabilistic Wisconsin card sorting task (WCST) with belief solicitation, in which subjects were presented with stimuli composed of multiple visual features. At each moment in time a particular feature was relevant for obtaining reward, and participants had to infer which feature was relevant and report their beliefs accordingly. To test the hypothesis that attentional focus modulates the belief update process, we derived and fitted several probabilistic and non-probabilistic behavioral models, which either incorporate a dynamical model of attentional focus, in the form of a hierarchical winner-take-all neuronal network, or a diffusive model, without attention-like features. We used Bayesian model selection to identify the most likely generative model of subjects' behavior and found that attention-like features in the behavioral model are essential for explaining subjects' responses. Furthermore, we demonstrate a method for integrating both connectionist and Bayesian models of decision making within a single framework that allowed us to infer hidden belief processes of human subjects.

The olfactory information that is received by the insect brain is encoded in the form of spatiotemporal patterns in the projection neurons of the antennal lobe. These dense and overlapping patterns are transformed into a sparse code in Kenyon cells in the mushroom body. Although it is clear that this sparse code is the basis for rapid categorization of odors, it is yet unclear how the sparse code in Kenyon cells is computed and what information it represents. Here we show that this computation can be modeled by sequential firing rate patterns using Lotka-Volterra equations and Bayesian online inference. This new model can be understood as an 'intelligent coincidence detector', which robustly and dynamically encodes the presence of specific odor features. We found that the model is able to qualitatively reproduce experimentally observed activity in both the projection neurons and the Kenyon cells. In particular, the model explains mechanistically how sparse activity in the Kenyon cells arises from the dense code in the projection neurons. The odor classification performance of the model proved to be robust against noise and time jitter in the observed input sequences. As in recent experimental results, we found that recognition of an odor happened very early during stimulus presentation in the model. Critically, by using the model, we found surprising but simple computational explanations for several experimental phenomena.

Even for simple perceptual decisions, the mechanisms that the brain employs are still under debate. Although current consensus states that the brain accumulates evidence extracted from noisy sensory information, open questions remain about how this simple model relates to other perceptual phenomena such as flexibility in decisions, decision-dependent modulation of sensory gain, or confidence about a decision. We propose a novel approach of how perceptual decisions are made by combining two influential formalisms into a new model. Specifically, we embed an attractor model of decision making into a probabilistic framework that models decision making as Bayesian inference. We show that the new model can explain decision making behaviour by fitting it to experimental data. In addition, the new model combines for the first time three important features: First, the model can update decisions in response to switches in the underlying stimulus. Second, the probabilistic formulation accounts for top-down effects that may explain recent experimental findings of decision-related gain modulation of sensory neurons. Finally, the model computes an explicit measure of confidence which we relate to recent experimental evidence for confidence computations in perceptual decision tasks.

It is commonly assumed that the recruitment of visual areas during audition is not relevant for performing auditory tasks ('auditory-only view'). According to an alternative view, however, the recruitment of visual cortices is thought to optimize auditory-only task performance ('auditory-visual view'). This alternative view is based on functional magnetic resonance imaging (fMRI) studies. These studies have shown, for example, that even if there is only auditory input available, face-movement sensitive areas within the posterior superior temporal sulcus (pSTS) are involved in understanding what is said (auditory-only speech recognition). This is particularly the case when speakers are known audio-visually, that is, after brief voice-face learning. Here we tested whether the left pSTS involvement is causally related to performance in auditory-only speech recognition when speakers are known by face. To test this hypothesis, we applied cathodal transcranial direct current stimulation (tDCS) to the pSTS during (i) visual-only speech recognition of a speaker known only visually to participants and (ii) auditory-only speech recognition of speakers they learned by voice and face. We defined the cathode as active electrode to down-regulate cortical excitability by hyperpolarization of neurons. tDCS to the pSTS interfered with visual-only speech recognition performance compared to a control group without pSTS stimulation (tDCS to BA6/44 or sham). Critically, compared to controls, pSTS stimulation additionally decreased auditory-only speech recognition performance selectively for voice-face learned speakers. These results are important in two ways. First, they provide direct evidence that the pSTS is causally involved in visual-only speech recognition; this confirms a long-standing prediction of current face-processing models. Secondly, they show that visual face-sensitive pSTS is causally involved in optimizing auditory-only speech recognition. These results are in line with the 'auditory-visual view' of auditory speech perception, which assumes that auditory speech recognition is optimized by using predictions from previously encoded speaker-specific audio-visual internal models., (Copyright © 2015 Elsevier Ltd. All rights reserved.)

In sport events like Olympic Games or World Championships competitive athletes keep pushing the boundaries of human performance. Compared to team sports, high achievements in many athletic disciplines depend solely on the individual's performance. Contrasting previous research looking for expertise-related differences in brain anatomy at the group level, we aim to demonstrate changes in individual top athlete's brain, which would be averaged out in a group analysis. We compared structural magnetic resonance images (MRI) of three professional track-and-field athletes to age-, gender- and education-matched control subjects. To determine brain features specific to these top athletes, we tested for significant deviations in structural grey matter density between each of the three top athletes and a carefully matched control sample. While total brain volumes were comparable between athletes and controls, we show regional grey matter differences in striatum and thalamus. The demonstrated brain anatomy patterns remained stable and were detected after 2 years with Olympic Games in between. We also found differences in the fusiform gyrus in two top long jumpers. We interpret our findings in reward-related areas as correlates of top athletes' persistency to reach top-level skill performance over years.

The human voice is the primary carrier of speech but also a fingerprint for person identity. Previous neuroimaging studies have revealed that speech and identity recognition is accomplished by partially different neural pathways, despite the perceptual unity of the vocal sound. Importantly, the right STS has been implicated in voice processing, with different contributions of its posterior and anterior parts. However, the time point at which vocal and speech processing diverge is currently unknown. Also, the exact role of the right STS during voice processing is so far unclear because its behavioral relevance has not yet been established. Here, we used the high temporal resolution of magnetoencephalography and a speech task control to pinpoint transient behavioral correlates: we found, at 200 msec after stimulus onset, that activity in right anterior STS predicted behavioral voice recognition performance. At the same time point, the posterior right STS showed increased activity during voice identity recognition in contrast to speech recognition whereas the left mid STS showed the reverse pattern. In contrast to the highly speech-sensitive left STS, the current results highlight the right STS as a key area for voice identity recognition and show that its anatomical-functional division emerges around 200 msec after stimulus onset. We suggest that this time point marks the speech-independent processing of vocal sounds in the posterior STS and their successful mapping to vocal identities in the anterior STS.

Recognizing the identity of other individuals across different sensory modalities is critical for successful social interaction. In the human brain, face- and voice-sensitive areas are separate, but structurally connected. What kind of information is exchanged between these specialized areas during cross-modal recognition of other individuals is currently unclear. For faces, specific areas are sensitive to identity and to physical properties. It is an open question whether voices activate representations of face identity or physical facial properties in these areas. To address this question, we used functional magnetic resonance imaging in humans and a voice-face priming design. In this design, familiar voices were followed by morphed faces that matched or mismatched with respect to identity or physical properties. The results showed that responses in face-sensitive regions were modulated when face identity or physical properties did not match to the preceding voice. The strength of this mismatch signal depended on the level of certainty the participant had about the voice identity. This suggests that both identity and physical property information was provided by the voice to face areas. The activity and connectivity profiles differed between face-sensitive areas: (i) the occipital face area seemed to receive information about both physical properties and identity, (ii) the fusiform face area seemed to receive identity, and (iii) the anterior temporal lobe seemed to receive predominantly identity information from the voice. We interpret these results within a prediction coding scheme in which both identity and physical property information is used across sensory modalities to recognize individuals., (© 2014 Wiley Periodicals, Inc.)

Network studies of large-scale brain connectivity have demonstrated that highly connected areas, or "hubs," are a key feature of human functional and structural brain organization. We use resting-state functional MRI data and connectivity clustering to identify multi-network hubs and show that while hubs can belong to multiple networks their degree of integration into these different networks varies dynamically over time. The extent of the network variation was related to the connectedness of the hub. In addition, we found that these network dynamics were inversely related to positive self-generated thoughts reported by individuals and were further decreased with older age. Moreover, the left caudate varied its degree of participation between a default mode subnetwork and a limbic network. This variation was predictive of individual differences in the reports of past-related thoughts. These results support an association between ongoing thought processes and network dynamics and offer a new approach to investigate the brain dynamics underlying mental experience.

Behavioral data obtained with perceptual decision making experiments are typically analyzed with the drift-diffusion model. This parsimonious model accumulates noisy pieces of evidence toward a decision bound to explain the accuracy and reaction times of subjects. Recently, Bayesian models have been proposed to explain how the brain extracts information from noisy input as typically presented in perceptual decision making tasks. It has long been known that the drift-diffusion model is tightly linked with such functional Bayesian models but the precise relationship of the two mechanisms was never made explicit. Using a Bayesian model, we derived the equations which relate parameter values between these models. In practice we show that this equivalence is useful when fitting multi-subject data. We further show that the Bayesian model suggests different decision variables which all predict equal responses and discuss how these may be discriminated based on neural correlates of accumulated evidence. In addition, we discuss extensions to the Bayesian model which would be difficult to derive for the drift-diffusion model. We suggest that these and other extensions may be highly useful for deriving new experiments which test novel hypotheses.

How do we recognize people that are familiar to us? There is overwhelming evidence that our brains process voice and face in a combined fashion to optimally recognize both who is speaking and what is said. Surprisingly, this combined processing of voice and face seems to occur even if one stream of information is missing. For example, if subjects only hear someone who is familiar to them talking, without seeing their face, visual face-processing areas are active. One reason for this crossmodal activation might be that it is instrumental for early sensory processing of voices-a hypothesis that is contrary to current models of unisensory perception. Here, we test this hypothesis by harnessing a temporally highly resolved method, i.e., magnetoencephalography (MEG), to identify the temporal response profile of the fusiform face area in response to auditory-only voice recognition. Participants briefly learned a set of voices audio-visually, i.e., together with a talking face. After learning, we measured subjects' MEG signals in response to the auditory-only, now familiar, voices. The results revealed three key mechanisms that characterize the sensory processing of familiar speakers' voices: (i) activation in the face-sensitive fusiform gyrus at very early auditory processing stages, i.e., only 100ms after auditory onset, (ii) a temporal facilitation of auditory processing (M200), and (iii) a correlation of this temporal facilitation with recognition performance. These findings suggest that a neural representation of face information is evoked before the identity of the voice is even recognized and that the brain uses this visual representation to facilitate early sensory processing of auditory-only voices., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Our knowledge about the computational mechanisms underlying human learning and recognition of sound sequences, especially speech, is still very limited. One difficulty in deciphering the exact means by which humans recognize speech is that there are scarce experimental findings at a neuronal, microscopic level. Here, we show that our neuronal-computational understanding of speech learning and recognition may be vastly improved by looking at an animal model, i.e., the songbird, which faces the same challenge as humans: to learn and decode complex auditory input, in an online fashion. Motivated by striking similarities between the human and songbird neural recognition systems at the macroscopic level, we assumed that the human brain uses the same computational principles at a microscopic level and translated a birdsong model into a novel human sound learning and recognition model with an emphasis on speech. We show that the resulting Bayesian model with a hierarchy of nonlinear dynamical systems can learn speech samples such as words rapidly and recognize them robustly, even in adverse conditions. In addition, we show that recognition can be performed even when words are spoken by different speakers and with different accents-an everyday situation in which current state-of-the-art speech recognition models often fail. The model can also be used to qualitatively explain behavioral data on human speech learning and derive predictions for future experiments.

In sentence processing, it is still unclear how the neural language network successfully establishes argument-verb dependencies in its spatiotemporal neuronal dynamics. Previous work has suggested that the establishment of subject-verb and object-verb dependencies requires argument retrieval from working memory, and that dependency establishment in object-first sentences additionally necessitates argument reordering. We examine the spatiotemporal neuronal dynamics of the brain regions that subserve these sub-processes by crossing an argument reordering factor (i.e., subject-first versus object-first sentences) with an argument retrieval factor (i.e., short versus long argument-verb dependencies) in German. Using functional magnetic resonance imaging (fMRI), we found that reordering demands focally activate the left pars opercularis (Broca's area), while storage and retrieval demands activated left temporo-parietal (TP) regions. In addition, when analyzing the time course of fMRI-informed equivalent current dipole sources in the EEG at the subcategorizing verb, we found that activity in the TP-region occurs relatively early (40-180 ms), followed by activity in Broca's area (300-500 ms). These findings were matched by topographical correlation analyses of fMRI activations in EEG sensor space, showing that, in the scalp potential, TP-region activity surfaces as an early positivity and IFG activity as a later positivity in the scalp potential. These results provide fine-grained evidence for spatiotemporally separable sub-processes of argument retrieval and reordering in sentence processing.

An echo state network (ESN) consists of a large, randomly connected neural network, the reservoir, which is driven by an input signal and projects to output units. During training, only the connections from the reservoir to these output units are learned. A key requisite for output-only training is the echo state property (ESP), which means that the effect of initial conditions should vanish as time passes. In this paper, we use analytical examples to show that a widely used criterion for the ESP, the spectral radius of the weight matrix being smaller than unity, is not sufficient to satisfy the echo state property. We obtain these examples by investigating local bifurcation properties of the standard ESNs. Moreover, we provide new sufficient conditions for the echo state property of standard sigmoid and leaky integrator ESNs. We furthermore suggest an improved technical definition of the echo state property, and discuss what practicians should (and should not) observe when they optimize their reservoirs for specific tasks., (Copyright © 2012 Elsevier Ltd. All rights reserved.)

Developmental dyslexia, a severe and persistent reading and spelling impairment, is characterized by difficulties in processing speech sounds (i.e., phonemes). Here, we test the hypothesis that these phonological difficulties are associated with a dysfunction of the auditory sensory thalamus, the medial geniculate body (MGB). By using functional MRI, we found that, in dyslexic adults, the MGB responded abnormally when the task required attending to phonemes compared with other speech features. No other structure in the auditory pathway showed distinct functional neural patterns between the two tasks for dyslexic and control participants. Furthermore, MGB activity correlated with dyslexia diagnostic scores, indicating that the task modulation of the MGB is critical for performance in dyslexics. These results suggest that deficits in dyslexia are associated with a failure of the neural mechanism that dynamically tunes MGB according to predictions from cortical areas to optimize speech processing. This view on task-related MGB dysfunction in dyslexics has the potential to reconcile influential theories of dyslexia within a predictive coding framework of brain function.

Accumulating empirical evidence suggests a role of Bayesian inference and learning for shaping neural responses in auditory and visual perception. However, its relevance for somatosensory processing is unclear. In the present study we test the hypothesis that cortical somatosensory processing exhibits dynamics that are consistent with Bayesian accounts of brain function. Specifically, we investigate the cortical encoding of Bayesian surprise, a recently proposed marker of Bayesian perceptual learning, using EEG data recorded from 15 subjects. Capitalizing on a somatosensory mismatch roving paradigm, we performed computational single-trial modeling of evoked somatosensory potentials for the entire peri-stimulus time period in source space. By means of Bayesian model selection, we find that, at 140 ms post-stimulus onset, secondary somatosensory cortex represents Bayesian surprise rather than stimulus change, which is the conventional marker of EEG mismatch responses. In contrast, at 250 ms, right inferior frontal cortex indexes stimulus change. Finally, at 360 ms, our analyses indicate additional perceptual learning attributable to medial cingulate cortex. In summary, the present study provides novel evidence for anatomical-temporal/functional segregation in human somatosensory processing that is consistent with the Bayesian brain hypothesis., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Recurrent neural networks (RNNs) are widely used in computational neuroscience and machine learning applications. In an RNN, each neuron computes its output as a nonlinear function of its integrated input. While the importance of RNNs, especially as models of brain processing, is undisputed, it is also widely acknowledged that the computations in standard RNN models may be an over-simplification of what real neuronal networks compute. Here, we suggest that the RNN approach may be made computationally more powerful by its fusion with Bayesian inference techniques for nonlinear dynamical systems. In this scheme, we use an RNN as a generative model of dynamic input caused by the environment, e.g. of speech or kinematics. Given this generative RNN model, we derive Bayesian update equations that can decode its output. Critically, these updates define a 'recognizing RNN' (rRNN), in which neurons compute and exchange prediction and prediction error messages. The rRNN has several desirable features that a conventional RNN does not have, e.g. fast decoding of dynamic stimuli and robustness to initial conditions and noise. Furthermore, it implements a predictive coding scheme for dynamic inputs. We suggest that the Bayesian inversion of RNNs may be useful both as a model of brain function and as a machine learning tool. We illustrate the use of the rRNN by an application to the online decoding (i.e. recognition) of human kinematics.

Neuronal responses exhibit two stimulus or task-related components: evoked and induced. The functional role of induced responses has been ascribed to 'top-down' modulation through backward connections and lateral interactions; as opposed to the bottom-up driving processes that may predominate in evoked components. The implication is that evoked and induced components may reflect different neuronal processes. The conventional way of separating evoked and induced responses assumes that they can be decomposed linearly; in that induced responses are the average of the power minus the power of the average (the evoked component). However, this decomposition may not hold if both components are generated by nonlinear processes. In this work, we propose a Dynamic Causal Model that models evoked and induced responses at the same time. This allows us to explain both components in terms of shared mechanisms (coupling) and changes in coupling that are necessary to explain any induced components. To establish the face validity of our approach, we used Bayesian Model Selection to show that the scheme can disambiguate between models of synthetic data that did and did not contain induced components. We then repeated the analysis using MEG data during a hand grip task to ask whether induced responses in motor control circuits are mediated by 'top-down' or backward connections. Our result provides empirical evidence that induced responses are more likely to reflect backward message passing in the brain, while evoked and induced components share certain characteristics and mechanisms., (Copyright © 2011 Elsevier Inc. All rights reserved.)

The neuronal system underlying learning, generation and recognition of song in birds is one of the best-studied systems in the neurosciences. Here, we use these experimental findings to derive a neurobiologically plausible, dynamic, hierarchical model of birdsong generation and transform it into a functional model of birdsong recognition. The generation model consists of neuronal rate models and includes critical anatomical components like the premotor song-control nucleus HVC (proper name), the premotor nucleus RA (robust nucleus of the arcopallium), and a model of the syringeal and respiratory organs. We use Bayesian inference of this dynamical system to derive a possible mechanism for how birds can efficiently and robustly recognize the songs of their conspecifics in an online fashion. Our results indicate that the specific way birdsong is generated enables a listening bird to robustly and rapidly perceive embedded information at multiple time scales of a song. The resulting mechanism can be useful for investigating the functional roles of auditory recognition areas and providing predictions for future birdsong experiments.

In this paper, we pursue recent observations that, through selective dendritic filtering, single neurons respond to specific sequences of presynaptic inputs. We try to provide a principled and mechanistic account of this selectivity by applying a recent free-energy principle to a dendrite that is immersed in its neuropil or environment. We assume that neurons self-organize to minimize a variational free-energy bound on the self-information or surprise of presynaptic inputs that are sampled. We model this as a selective pruning of dendritic spines that are expressed on a dendritic branch. This pruning occurs when postsynaptic gain falls below a threshold. Crucially, postsynaptic gain is itself optimized with respect to free energy. Pruning suppresses free energy as the dendrite selects presynaptic signals that conform to its expectations, specified by a generative model implicit in its intracellular kinetics. Not only does this provide a principled account of how neurons organize and selectively sample the myriad of potential presynaptic inputs they are exposed to, but it also connects the optimization of elemental neuronal (dendritic) processing to generic (surprise or evidence-based) schemes in statistics and machine learning, such as Bayesian model selection and automatic relevance determination.

In this paper, we present a generic approach that can be used to infer how subjects make optimal decisions under uncertainty. This approach induces a distinction between a subject's perceptual model, which underlies the representation of a hidden "state of affairs" and a response model, which predicts the ensuing behavioural (or neurophysiological) responses to those inputs. We start with the premise that subjects continuously update a probabilistic representation of the causes of their sensory inputs to optimise their behaviour. In addition, subjects have preferences or goals that guide decisions about actions given the above uncertain representation of these hidden causes or state of affairs. From a Bayesian decision theoretic perspective, uncertain representations are so-called "posterior" beliefs, which are influenced by subjective "prior" beliefs. Preferences and goals are encoded through a "loss" (or "utility") function, which measures the cost incurred by making any admissible decision for any given (hidden) state of affair. By assuming that subjects make optimal decisions on the basis of updated (posterior) beliefs and utility (loss) functions, one can evaluate the likelihood of observed behaviour. Critically, this enables one to "observe the observer", i.e. identify (context- or subject-dependent) prior beliefs and utility-functions using psychophysical or neurophysiological measures. In this paper, we describe the main theoretical components of this meta-Bayesian approach (i.e. a Bayesian treatment of Bayesian decision theoretic predictions). In a companion paper ('Observing the observer (II): deciding when to decide'), we describe a concrete implementation of it and demonstrate its utility by applying it to simulated and real reaction time data from an associative learning task.