Search

Over the last decades, several studies have demonstrated that conscious and unconscious reward incentives both affect performance in physical and cognitive tasks, suggesting that goal-pursuit can arise from an unconscious will. Whether the planning of goal-directed saccadic eye movements during an effortful task can also be affected by subliminal reward cues has not been systematically investigated. We employed a novel task where participants made several eye movements back and forth between a fixation point and a number of peripheral targets. The total number of targets visited by the eyes in a fixed amount of time determined participants' monetary gain. The magnitude of the reward at stake was briefly shown at the beginning of each trial and masked by pattern images superimposed in time so that at shorter display durations participants perceived reward incentives subliminally. We found a main effect of reward across all display durations as higher reward enhanced participants' oculomotor effort measured as the frequency and peak velocity of saccades. This effect was strongest for consciously perceived rewards but also occurred when rewards were subliminally perceived. Although we did not find a statistically significant dissociation between the reward-related modulation of different saccadic parameters, across two experiments the most robust effect of subliminal rewards was observed for the modulation of the saccadic frequency but not the peak velocity. These results suggest that multiple indices of oculomotor effort can be incentivized by subliminal rewards and that saccadic frequency may provide the most sensitive indicator of subliminal incentivization of eye movements.

Stimuli associated with high reward modulate perception and such value-driven effects have been shown to originate from the modulation of the earliest stages of sensory processing in the brain. In natural environments objects comprise multiple features (imagine a rolling soccer ball, with its black and white patches and the swishing sound made during its motion), where each feature may signal different associations with previously encountered rewards. How perception of such an object is affected by the value associations of its constituent parts is unknown. The present study compares intra- and cross-modal value-driven effects on behavioral and electrophysiological correlates of visual perception. Human participants first learned the reward associations of visual and auditory cues. Subsequently, they performed a visual orientation discrimination task in the presence of previously rewarded visual or auditory cues (intra- and cross-modal cues, respectively) that were concurrently presented with the target stimulus. During the conditioning phase, when reward associations were learned and reward cues were the target of the task, reward value of both modalities enhanced the electrophysiological correlates of sensory processing in visual cortex. During the post-conditioning phase, when reward delivery was halted and previously rewarded stimuli were task-irrelevant, cross-modal value-enhanced behavioral measures of visual sensitivity whereas intra-modal value led to a trend for suppression. A similar pattern of modulations was found in the simultaneously recorded event-related potentials (ERPs) of posterior electrodes. We found an early (90-120 ms) suppression of ERPs evoked by high-value, intra-modal stimuli. Cross-modal cues led to a later value-driven modulation, with an enhancement of response positivity for high-compared to low-value stimuli starting at the N1 window (180-250 ms) and extending to the P3 (300-600 ms) responses of the posterior electrodes. These results indicate that visual cortex is modulated by the reward value of visual as well as auditory cues. Previously rewarded, task-irrelevant cues from the same or different sensory modality have a different effect on visual perception, as intra-modal high-value cues may interfere with the target processing, whereas cross-modal high-value cues boost the perception of the target.

In everyday life, faces with emotional expressions quickly attract attention and eye-movements. To study the neural mechanisms of such emotion-driven attention by means of event-related brain potentials (ERPs), tasks that employ covert shifts of attention are commonly used, in which participants need to inhibit natural eye-movements towards stimuli. It remains, however, unclear how shifts of attention to emotional faces with and without eye-movements differ from each other. The current preregistered study aimed to investigate neural differences between covert and overt emotion-driven attention. We combined eye-tracking with measurements of ERPs to compare shifts of attention to faces with happy, angry or neutral expressions when eye-movements were either executed (Go conditions) or withheld (No-go conditions). Happy and angry faces led to larger EPN amplitudes, shorter latencies of the P1 component and faster saccades, suggesting that emotional expressions significantly affected shifts of attention. Several ERPs (N170, EPN, LPC), were augmented in amplitude when attention was shifted with an eye-movement, indicating an enhanced neural processing of faces if eye-movements had to be executed together with a reallocation of attention. However, the modulation of ERPs by facial expressions did not differ between the Go and No-go conditions, suggesting that emotional content enhances both covert and overt shifts of attention. In summary, our results indicate that overt and covert attention shifts differ but are comparably affected by emotional content.

Humans can use their previous experience in form of statistical priors to improve decisions. It is however unclear how such priors are learned and represented. Importantly, it has remained elusive whether prior learning is independent of the sensorimotor system involved in the learning process or not, as both modality-specific and modality-general learning have been reported in the past. Here, we used a saccadic eye movement task to probe the learning and representation of a spatial prior across a few trials. In this task, learning occurs in an unsupervised manner and through encountering trial-by-trial visual hints drawn from a distribution centered on the target location. Using a model-comparison approach, we found that participants’ prior knowledge is largely represented in the form of their previous motor actions, with minimal influence from the previously seen visual hints. By using two different motor contexts for response (looking either at the estimated target location, or exactly opposite to it), we could further compare whether prior experience obtained in one motor context can be transferred to the other. Although learning curves were highly similar, and participants seemed to use the same strategy for both response types, they could not transfer their knowledge between contexts, as performance and confidence ratings dropped to naïve levels after a switch of the required response. Together, our results suggest that humans preferably use the internal representations of their previous motor actions, rather than past incoming sensory information, to form statistical sensorimotor priors on the timescale of a few trials.

One of the most important tasks for the visual system is to construct an internal representation of the spatial properties of objects, including their size. Size perception includes a combination of bottom-up (retinal inputs) and top-down (e.g., expectations) information, which makes the estimates of object size malleable and susceptible to numerous contextual cues. For example, it has been shown that size perception is prone to adaptation: brief previous presentations of larger or smaller adapting stimuli at the same region of space changes the perceived size of a subsequent test stimulus. Large adapting stimuli cause the test to appear smaller than its veridical size and vice versa. Here, we investigated whether size adaptation is susceptible to attentional modulation. First, we measured the magnitude of adaptation aftereffects for a size discrimination task. Then, we compared these aftereffects (on average 15-20%) with those measured while participants were engaged, during the adaptation phase, in one of the two highly demanding central visual tasks: Multiple Object Tracking (MOT) or Rapid Serial Visual Presentation (RSVP). Our results indicate that deploying visual attention away from the adapters did not significantly affect the distortions of perceived size induced by adaptation, with accuracy and precision in the discrimination task being almost identical in all experimental conditions. Taken together, these results suggest that visual attention does not play a key role in size adaptation, in line with the idea that this phenomenon can be accounted for by local gain control mechanisms within area V1.

Adaptive agents must act in intrinsically uncertain environments with complex latent structure. Here, we elaborate a model of visual foraging – in a hierarchical context – wherein agents infer a higher-order visual pattern (a ‘scene’) by sequentially sampling ambiguous cues. Inspired by previous models of scene construction – that cast perception and action as consequences of approximate Bayesian inference – we use active inference to simulate decisions of agents categorizing a scene in a hierarchically-structured setting. Under active inference, agents develop probabilistic beliefs about their environment, while actively sampling it to maximise the evidence for their internal generative model. This approximate evidence maximization (i.e. self-evidencing) comprises drives to both maximise rewards and resolve uncertainty about hidden states. This is realised via minimization of a free energy functional of posterior beliefs about both the world as well as the actions used to sample or perturb it, corresponding to perception and action, respectively. We show that active inference, in the context of hierarchical scene construction, gives rise to many empirical evidence accumulation phenomena, such as noise-sensitive reaction times and epistemic saccades. We explain these behaviours in terms of the principled drives that constitute the expected free energy, the key quantity for evaluating policies under active inference. In addition, we report novel behaviours exhibited by these active inference agents that furnish new predictions for research on evidence accumulation and perceptual decision-making. We discuss the implications of this hierarchical active inference scheme for tasks that require planned sequences of information-gathering actions to infer compositional latent structure (such as visual scene construction and sentence comprehension). Finally, we propose experiments to contextualise active inference in relation to other formulations of evidence accumulation (e.g. drift-diffusion models) in tasks that require planning in uncertain environments with higher-order structure.

Reward value guides goal-directed behavior and modulates early sensory processing. Rewarding stimuli are often multisensory but it is not known how reward value is combined across sensory modalities. Here we show that the integration of reward value critically depends on whether the distinct sensory inputs are perceived to emanate from the same multisensory object. We systematically manipulated the congruency in monetary reward values and the relative spatial positions of co-occurring auditory and visual stimuli that served as bimodal distractors during an oculomotor task. The amount of interference induced by the distractors was used as an indicator of their perceptual salience. Our results across two experiments show that when reward value is linked to each modality separately, the value congruence between vision and audition determines the combined salience of the bimodal distractors. However, reward value of vision wins over the value of audition if visual and auditory stimuli have been experienced as belonging to the same audiovisual object prior to the learning of the reward values. The perceived spatial alignment of auditory and visual stimuli is a prerequisite for the integration of their reward values, as no effect of reward value was observed when the two modalities were perceived to be misaligned. These results show that in a task that highly relies on the processing of visual spatial information, the reward values from multiple sensory modalities are integrated with each other, each with their respective weights. This weighting depends on the congruency in reward values, exposure history, and spatial co-localization.

Effort constitutes a major part of cost-benefit calculations underlying decision making. Therefore, estimating the effort someone has spent on a task is a core dimension for evaluating own and others’ actions. It has been previously shown that self-judgments of effort are influenced by the magnitude of obtained rewards. It is unclear, however, whether the influence of reward on effort estimations is limited to self-judgments or whether reward incorporation represents a general computational principle when judging effort. Here we show that people also integrate reward magnitude when judging the effort exerted by others. Participants (N=48) performed an effortful sensorimotor task interleaved with a partner, while rating either their own or the other person’s effort. After each trial but before the effort rating, both participants were informed about the obtained reward. We found that higher rewards led to higher estimations of exerted effort, in self-as well as other-judgments, and this effect was more pronounced for other-judgments. In both types of judgment, computational modelling revealed that reward information and the perceived level of exertion were combined in a Bayes optimal manner to form effort estimates. Remarkably, the extent to which rewards influenced effort judgments was positively correlated with conservative world-views, indicating that the basic computations underlying this behavioural phenomenon might be related to more general beliefs about the association between effort and reward in the society. The integration of reward information into retrospective effort judgments underscores the convergence of multiple information sources that supports adaptive learning and decision making in social contexts.

Societal norms exert a powerful influence on our decisions. Behaviours motivated by norms, however, do not always concur with the responses mandated by decision relevant information potentially generating a conflict. To probe the interplay between normative and informational influences, we examined how prosocial norms impact on perceptual decisions subjects made in the context of a simultaneous presentation of social information. Participants displayed a bias in their perceptual decisions towards that mandated by social information. However, normative prescriptions modulated this bias bi-directionally depending on whether norms mandated a decision in accord or contrary to the contextual social information. At a neural level, the addition of a norms increased activity in prefrontal cortex and modulated functional connectivity between prefrontal and parietal areas. The bi-directional effect of our norms was captured by differential activations when participants decided against the social information. When norms indicated a decision in line with social information, non-compliance modulated lateral prefrontal cortex activity. By contrast, when norms mandated a decision against social information norm compliance increased activity in the anterior cingulate cortex. Hence, social norms changed the balance between a reliance on perceptual and social information by modulating brain activity in regions associated with response inhibition and conflict monitoring.

Summary Background Object-based attention can group image elements of spatially extended objects into coherent representations, but its mechanisms have remained unclear. The mechanisms for object-based attention may include shape-selective neurons in higher visual cortical areas that feed back to lower areas to simultaneously enhance the representation of all image elements of a relevant shape. Another, nonexclusive mechanism is the spread of attention in early visual cortex according to Gestalt rules, which could successively add new elements to a growing object representation. Results We investigated the dynamics of object-based attention in the primary visual cortex of monkeys trained to perform a contour-grouping task. The animals mentally traced a target curve through the visual field and ignored a distracting curve. Neuronal responses elicited by the target curve were enhanced relative to those elicited by distracting curve. Remarkably, the response enhancement was delayed for neurons with receptive fields farther from the start of the tracing process. We could therefore measure propagation speed and found that it was low if curves were nearby and that it increased if curves were far apart. The results are well explained by an "attentional growth-cone" model, which holds that the response enhancement can spread in multiple visual cortical areas with different receptive field sizes at a speed of approximately 50 ms per receptive field. Conclusions Our findings support an active role for early visual areas in object-based attention because neurons in these areas gradually spread enhanced activity over the representation of relevant objects.

In natural scenes, objects rarely occur in isolation but appear within a spatiotemporal context. Here, we show that the perceived size of a stimulus is significantly affected by the context of the scene: brief previous presentation of larger or smaller adapting stimuli at the same region of space changes the perceived size of a test stimulus, with larger adapting stimuli causing the test to appear smaller than veridical and vice versa. In a human fMRI study, we measured the blood oxygen level-dependent activation (BOLD) responses of the primary visual cortex (V1) to the contours of large-diameter stimuli and found that activation closely matched the perceptual rather than the retinal stimulus size: the activated area of V1 increased or decreased, depending on the size of the preceding stimulus. A model based on local inhibitory V1 mechanisms simulated the inward or outward shifts of the stimulus contours and hence the perceptual effects. Our findings suggest that area V1 is actively involved in reshaping our perception to match the short-term statistics of the visual scene.

Recently, M. Boi, H. Ogmen, J. Krummenacher, T. U. Otto, & M. H. Herzog (2009) reported a fascinating visual effect, where the direction of apparent motion was disambiguated by cues along the path of apparent motion, the Ternus-Pikler group motion, even though no actual movement occurs in this stimulus. They referred to their study as a "litmus test" to distinguish "non-retinotopic" (motion-based) from "retinotopic" (retina-based) image processing. We adapted the test to one with simple grating stimuli that could be more readily modeled and replicated their psychophysical results quantitatively with this stimulus. We then modeled our experiments in 3D (x, y, t) Fourier space and demonstrated that the observed perceptual effects are readily accounted for by integration of information within a detector that is oriented in space and time, in a similar way to previous explanations of other motion illusions. This demonstration brings the study of Boi et al. into the more general context of perception of moving objects.