Search

We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation.

In spontaneous, stimulus-evoked, and eye-movement evoked EEG, the oscillatory signal shows large scale, dynamically organized patterns of phase. We investigated eye-movement evoked patterns in free-viewing conditions. Participants viewed photographs of natural scenes in anticipation of a memory test. From 200 ms intervals following saccades, we estimated the EEG phase gradient over the entire scalp, and the wave activity, i.e. the goodness of fit of a wave model involving a phase gradient assumed to be smooth over the scalp. In frequencies centered at 6.5 Hz, large-scale phase organization occurred, peaking around 70 ms after fixation onset and taking the form of a traveling wave. According to the wave gradient, most of the times the wave spreads from the posterior-inferior to anterior-superior direction. In these directions, the gradients depended on the size and direction of the saccade. Wave propagation velocity decreased in the course of the fixation, particularly in the interval from 50 to 150 ms after fixation onset. This interval corresponds to the fixation-related lambda activity, which reflects early perceptual processes following fixation onset. We conclude that lambda activity has a prominent traveling wave component. This component consists of a short-term whole-head phase pattern of specific direction and velocity, which may reflect feedforward propagation of visual information at fixation.

Co-registration of EEG and eye movement has promise for investigating perceptual processes in free viewing conditions, provided certain methodological challenges can be addressed. Most of these arise from the self-paced character of eye movements in free viewing conditions. Successive eye movements occur within short time intervals. Their evoked activity is likely to distort the EEG signal during fixation. Due to the non-uniform distribution of fixation durations, these distortions are systematic, survive across-trials averaging, and can become a source of confounding. We illustrate this problem with effects of sequential eye movements on the evoked potentials and time-frequency components of EEG and propose a solution based on matching of eye movement characteristics between experimental conditions. The proposal leads to a discussion of which eye movement characteristics are to be matched, depending on the EEG activity of interest. We also compare segmentation of EEG into saccade-related epochs relative to saccade and fixation onsets and discuss the problem of baseline selection and its solution. Further recommendations are given for implementing EEG-eye movement co-registration in free viewing conditions. By resolving some of the methodological problems involved, we aim to facilitate the transition from the traditional stimulus-response paradigm to the study of visual perception in more naturalistic conditions., (Copyright © 2016 Elsevier Inc. All rights reserved.)

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eyemovement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. [ABSTRACT FROM AUTHOR]

• Episodic memory builds up across eye movements to event elements. • Co-registered eye-tracking and EEG reveal the underlying neural mechanisms. • Deconvolution modeling corrected for the spurious effects of saccades on EEG. • Fixation-related theta and alpha activity predicts subsequent episodic memory performance. The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior. [Display omitted] [ABSTRACT FROM AUTHOR]

Oculomotor behavior reveals, not only the acquisition of visual information at fixation, but also the accumulation of information in memory across subsequent fixations. Two candidate measures were considered as indicators of such dynamic visual memory load: fixation duration and pupil size. While recording these measures, we displayed an arrangement of 3, 4 or 5 targets among distractors. Both occurred in various orientations. Participants searched for targets and reported whether in a subsequent display one of them had changed orientation. We determined to what extent fixation duration and pupil size indicate dynamic memory load, as a function of the number of targets fixated during the search. We found that fixation duration reflects the number of targets, both when this number is within and above the limit of working memory capacity. Pupil size reflects the number of targets only when it exceeds the capacity limit. Moreover, the duration of fixations on successive targets but not on distractors increases whereas pupil size does not. The increase in fixation duration with number of targets both within and above working memory capacity suggests that in free viewing fixation duration is sensitive to actual memory load as well as to processing load, whereas pupil size is indicative of processing load only. Two alternative models relating visual attention and working memory are considered relevant to these results. We discuss the results as supportive of a model which involves a temporary buffer in the interaction of attention and working memory. [ABSTRACT FROM AUTHOR]

In scrutinizing a scene, the eyes alternate between fixations and saccades. During a fixation, two component processes can be distinguished: visual encoding and selection of the next fixation target. We aimed to distinguish the neural correlates of these processes in the electrical brain activity prior to a saccade onset. Participants viewed color photographs of natural scenes, in preparation for a change detection task. Then, for each participant and each scene we computed an image heat map, with temperature representing the duration and density of fixations. The temperature difference between the start and end points of saccades was taken as a measure of the expected task-relevance of the information concentrated in specific regions of a scene. Visual encoding was evaluated according to whether subsequent change was correctly detected. Saccades with larger temperature difference were more likely to be followed by correct detection than ones with smaller temperature differences. The amplitude of presaccadic activity over anterior brain areas was larger for correct detection than for detection failure. This difference was observed for short "scrutinizing" but not for long "explorative" saccades, suggesting that presaccadic activity reflects top-down saccade guidance. Thus, successful encoding requires local scanning of scene regions which are expected to be task-relevant. Next, we evaluated fixation target selection. Saccades "moving up" in temperature were preceded by presaccadic activity of higher amplitude than those "moving down". This finding suggests that presaccadic activity reflects attention deployed to the following fixation location. Our findings illustrate how presaccadic activity can elucidate concurrent brain processes related to the immediate goal of planning the next saccade and the larger-scale goal of constructing a robust representation of the visual scene. [ABSTRACT FROM AUTHOR]

Abstract: We considered the hypothesis that spontaneous dissociation between the direction of attention and eye movement causes encoding failure in change detection. We tested this hypothesis by analyzing eye fixation-related potentials (EFRP) at the encoding stage of a change blindness task; when participants freely inspect a scene containing an unmarked target region, in which a change will occur in a subsequent presentation. We measured EFRP amplitude prior to the execution of a saccade, depending on its starting or landing position relative to the target region. For those landings inside the target region, we found a difference in EFRP between correct detection and failure. Overall, correspondence between EFRP amplitude and the size of the saccade predicted successful detection of change; lack of correspondence was followed by change blindness. By contrast, saccade sizes and fixation durations around the target region were unrelated to subsequent change detection. Since correspondence between EFRP and eye movement indicates that overt attention was given to the target region, we concluded that overt attention is needed for successful encoding and that dissociation between eye movement and attention leads to change blindness. [Copyright &y& Elsevier]

Eye tracking research has revealed that refixations (returns to previously visited locations) serve to recover information lost or missed during scanning (Gilchrist and Harvey, 2000; Zelinsky et al., 2011). However, the neural mechanisms of human refixation control are unexplored. We combined eye tracking with EEG registration in a visual search task. EEG epochs related to ordinary fixations and refixations were analyzed for saccade planning effects in the presaccadic interval and for information acquisition effects in the postsaccadic interval. To control for overlapping brain responses in a saccade sequence, EEG epochs were matched between conditions on all relevant eye movement characteristics (Dimigen et al., 2011; Nikolaev et al., 2016). To control for the effect of time, EEG epochs were matched on fixation rank within trials. EEG in the presaccadic interval differed between refixation and ordinary fixation during the shift of attention to the next saccade target. Thus, refixation control operates on saccade planning (Nikolaev et al., 2018). In the postsaccadic interval, EEG differed between refixation and ordinary fixation only in lambda activity over occipital areas, which is associated with perception at fixation. This suggests that early information acquisition is distinct in refixations. Ordinary fixations to locations that were later revisited differed during the entire postsaccadic interval from ones that were not. This suggests attenuation not only of perception but also encoding, which results in loss of information that a subsequent refixation has to recover. [ABSTRACT FROM AUTHOR]

When EEG is co-registered with eye movements in free-viewing tasks, EEG response to cognitive processes of interest maybe confounded with that to sequential eye movements (Dimigen et al 2011; Nikolaev et al 2016). These two can be disentangled using generalized additive mixed models (GAMM) with eye movement parameters and experimental conditions as predictor variables. Since GAMM can account for non-linearity in data, it is particularly suited to modeling EEG. GAMM was used to investigate the role of visual saliency in saccade guidance in an EEG eye movement co-registration study. Participants were asked to search, during an 8s interval, for a contour formed by 7 collinear Gabor elements embedded in a field of similar but randomly oriented distractors. Participants were then asked to indicate whether the contour was present, which was the case in half of the trials. Since the interval preceding saccade onset (presaccadic interval) involves saccade planning to the next fixation location, EEG epochs in this interval were modeled using GAMM with visual saliency of the subsequent fixation location and perisaccadic eye movement parameters as predictors. Besides the effect of fixation duration and saccade size on pressaccadic EEG, low presaccadic EEG amplitude was associated with high saliency at the next fixation location in both contour-present and contour-absent conditions (van Humbeeck et al 2018). This reveals the role of bottom-up saliency in saccade guidance. Using GAMM, the effects of visual saliency on saccade guidance were successfully isolated from the effect of eye movements on EEG. [ABSTRACT FROM AUTHOR]

While viewing a scene, the eyes are attracted to salient stimuli. We set out to identify the brain signals controlling this process. In a contour integration task, in which participants searched for a collinear contour in a field of randomly oriented Gabor elements, a previously established model was applied to calculate a visual saliency value for each fixation location. We studied brain activity related to the modeled saliency values, using coregistered eye tracking and EEG. To disentangle EEG signals reflecting salience in free viewing from overlapping EEG responses to sequential eye movements, we adopted generalized additive mixed modeling (GAMM) to single epochs of saccade‐related EEG. We found that, when saliency at the next fixation location was high, amplitude of the presaccadic EEG activity was low. Since presaccadic activity reflects covert attention to the saccade target, our results indicate that larger attentional effort is needed for selecting less salient saccade targets than more salient ones. This effect was prominent in contour‐present conditions (half of the trials), but ambiguous in the contour‐absent condition. Presaccadic EEG activity may thus be indicative of bottom‐up factors in saccade guidance. The results underscore the utility of GAMM for EEG—eye movement coregistration research. [ABSTRACT FROM AUTHOR]

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning. [ABSTRACT FROM AUTHOR]

Highlights • Fixation-related potentials were studied in visual search with free eye movements. • The P300 is a neural correlate of target detection in visual search. • The amplitude of the target-related P300 depends on target probability. Abstract This study investigated the influence of target probability on the neural response to target detection in free viewing visual search. Participants were asked to indicate the number of targets (one or two) among distractors in a visual search task while EEG and eye movements were co-registered. Target probability was manipulated by varying the set size of the displays between 10, 22, and 30 items. Fixation-related potentials time-locked to first target fixations revealed a pronounced P300 at the centro-parietal cortex with larger amplitudes for set sizes 22 and 30 than for set size 10. With increasing set size, more distractor fixations preceded the detection of the target, resulting in a decreased target probability and, consequently, a larger P300. For distractors, no increase of P300 amplitude with set size was observed. The findings suggest that set size specifically affects target but not distractor processing in overt serial visual search. [ABSTRACT FROM AUTHOR]