Search

When freely viewing a scene, the eyes often return to previously visited locations. By tracking eye movements and coregistering eye movements and EEG, such refixations are shown to have multiple roles: repairing insufficient encoding from precursor fixations, supporting ongoing viewing by resampling relevant locations prioritized by precursor fixations, and aiding the construction of memory representations. All these functions of refixation behavior are understood to be underpinned by three oculomotor and cognitive systems and their associated brain structures. First, immediate saccade planning prior to refixations involves attentional selection of candidate locations to revisit. This process is likely supported by the dorsal attentional network. Second, visual working memory, involved in maintaining task-related information, is likely supported by the visual cortex. Third, higher-order relevance of scene locations, which depends on general knowledge and understanding of scene meaning, is likely supported by the hippocampal memory system. Working together, these structures bring about viewing behavior that balances exploring previously unvisited areas of a scene with exploiting visited areas through refixations. [ABSTRACT FROM AUTHOR]

We investigated visual working memory encoding across saccadic eye movements, focusing our analysis on refixation behavior. Over 10-s periods, participants performed a visual search for three, four, or five targets and remembered their orientations for a subsequent change-detection task. In 50% of the trials, one of the targets had its orientation changed. From the visual search period, we scored three types of refixations and applied measures for quantifying eye-fixation recurrence patterns. Repeated fixations on the same regions as well as repeated fixation patterns increased with memory load. Correct change detection was associated with more refixations on targets and less on distractors, with increased frequency of recurrence, and with longer intervals between refixations. The results are in accordance with the view that patterns of eye movement are an integral part of visual working memory representation. [ABSTRACT FROM AUTHOR]

In free viewing, the eyes return to previously visited locations rather frequently, even though the attentional and memory-related processes controlling eyemovement show a strong antirefixation bias. To overcome this bias, a special refixation triggering mechanism may have to be recruited. We probed the neural evidence for such a mechanism by combining eye tracking with EEG recording. A distinctive signal associated with refixation planning was observed in the EEG during the presaccadic interval: the presaccadic potential was reduced in amplitude before a refixation compared with normal fixations. The result offers direct evidence for a special refixation mechanism that operates in the saccade planning stage of eye movement control. Once the eyes have landed on the revisited location, acquisition of visual information proceeds indistinguishably from ordinary fixations. [ABSTRACT FROM AUTHOR]

• Episodic memory builds up across eye movements to event elements. • Co-registered eye-tracking and EEG reveal the underlying neural mechanisms. • Deconvolution modeling corrected for the spurious effects of saccades on EEG. • Fixation-related theta and alpha activity predicts subsequent episodic memory performance. The brain systems of episodic memory and oculomotor control are tightly linked, suggesting a crucial role of eye movements in memory. But little is known about the neural mechanisms of memory formation across eye movements in unrestricted viewing behavior. Here, we leverage simultaneous eye tracking and EEG recording to examine episodic memory formation in free viewing. Participants memorized multi-element events while their EEG and eye movements were concurrently recorded. Each event comprised elements from three categories (face, object, place), with two exemplars from each category, in different locations on the screen. A subsequent associative memory test assessed participants' memory for the between-category associations that specified each event. We used a deconvolution approach to overcome the problem of overlapping EEG responses to sequential saccades in free viewing. Brain activity was time-locked to the fixation onsets, and we examined EEG power in the theta and alpha frequency bands, the putative oscillatory correlates of episodic encoding mechanisms. Three modulations of fixation-related EEG predicted high subsequent memory performance: (1) theta increase at fixations after between-category gaze transitions, (2) theta and alpha increase at fixations after within-element gaze transitions, (3) alpha decrease at fixations after between-exemplar gaze transitions. Thus, event encoding with unrestricted viewing behavior was characterized by three neural mechanisms, manifested in fixation-locked theta and alpha EEG activity that rapidly turned on and off during the unfolding eye movement sequences. These three distinct neural mechanisms may be the essential building blocks that subserve the buildup of coherent episodic memories during unrestricted viewing behavior. [Display omitted] [ABSTRACT FROM AUTHOR]

Co-registration of EEG and eye movement has promise for investigating perceptual processes in free viewing conditions, provided certain methodological challenges can be addressed. Most of these arise from the self-paced character of eye movements in free viewing conditions. Successive eye movements occur within short time intervals. Their evoked activity is likely to distort the EEG signal during fixation. Due to the non-uniform distribution of fixation durations, these distortions are systematic, survive across-trials averaging, and can become a source of confounding. We illustrate this problem with effects of sequential eye movements on the evoked potentials and time-frequency components of EEG and propose a solution based on matching of eye movement characteristics between experimental conditions. The proposal leads to a discussion of which eye movement characteristics are to be matched, depending on the EEG activity of interest. We also compare segmentation of EEG into saccade-related epochs relative to saccade and fixation onsets and discuss the problem of baseline selection and its solution. Further recommendations are given for implementing EEG-eye movement co-registration in free viewing conditions. By resolving some of the methodological problems involved, we aim to facilitate the transition from the traditional stimulus-response paradigm to the study of visual perception in more naturalistic conditions. [ABSTRACT FROM AUTHOR]

Abstract: We considered the hypothesis that spontaneous dissociation between the direction of attention and eye movement causes encoding failure in change detection. We tested this hypothesis by analyzing eye fixation-related potentials (EFRP) at the encoding stage of a change blindness task; when participants freely inspect a scene containing an unmarked target region, in which a change will occur in a subsequent presentation. We measured EFRP amplitude prior to the execution of a saccade, depending on its starting or landing position relative to the target region. For those landings inside the target region, we found a difference in EFRP between correct detection and failure. Overall, correspondence between EFRP amplitude and the size of the saccade predicted successful detection of change; lack of correspondence was followed by change blindness. By contrast, saccade sizes and fixation durations around the target region were unrelated to subsequent change detection. Since correspondence between EFRP and eye movement indicates that overt attention was given to the target region, we concluded that overt attention is needed for successful encoding and that dissociation between eye movement and attention leads to change blindness. [Copyright &y& Elsevier]

Eye tracking research has revealed that refixations (returns to previously visited locations) serve to recover information lost or missed during scanning (Gilchrist and Harvey, 2000; Zelinsky et al., 2011). However, the neural mechanisms of human refixation control are unexplored. We combined eye tracking with EEG registration in a visual search task. EEG epochs related to ordinary fixations and refixations were analyzed for saccade planning effects in the presaccadic interval and for information acquisition effects in the postsaccadic interval. To control for overlapping brain responses in a saccade sequence, EEG epochs were matched between conditions on all relevant eye movement characteristics (Dimigen et al., 2011; Nikolaev et al., 2016). To control for the effect of time, EEG epochs were matched on fixation rank within trials. EEG in the presaccadic interval differed between refixation and ordinary fixation during the shift of attention to the next saccade target. Thus, refixation control operates on saccade planning (Nikolaev et al., 2018). In the postsaccadic interval, EEG differed between refixation and ordinary fixation only in lambda activity over occipital areas, which is associated with perception at fixation. This suggests that early information acquisition is distinct in refixations. Ordinary fixations to locations that were later revisited differed during the entire postsaccadic interval from ones that were not. This suggests attenuation not only of perception but also encoding, which results in loss of information that a subsequent refixation has to recover. [ABSTRACT FROM AUTHOR]

When EEG is co-registered with eye movements in free-viewing tasks, EEG response to cognitive processes of interest maybe confounded with that to sequential eye movements (Dimigen et al 2011; Nikolaev et al 2016). These two can be disentangled using generalized additive mixed models (GAMM) with eye movement parameters and experimental conditions as predictor variables. Since GAMM can account for non-linearity in data, it is particularly suited to modeling EEG. GAMM was used to investigate the role of visual saliency in saccade guidance in an EEG eye movement co-registration study. Participants were asked to search, during an 8s interval, for a contour formed by 7 collinear Gabor elements embedded in a field of similar but randomly oriented distractors. Participants were then asked to indicate whether the contour was present, which was the case in half of the trials. Since the interval preceding saccade onset (presaccadic interval) involves saccade planning to the next fixation location, EEG epochs in this interval were modeled using GAMM with visual saliency of the subsequent fixation location and perisaccadic eye movement parameters as predictors. Besides the effect of fixation duration and saccade size on pressaccadic EEG, low presaccadic EEG amplitude was associated with high saliency at the next fixation location in both contour-present and contour-absent conditions (van Humbeeck et al 2018). This reveals the role of bottom-up saliency in saccade guidance. Using GAMM, the effects of visual saliency on saccade guidance were successfully isolated from the effect of eye movements on EEG. [ABSTRACT FROM AUTHOR]

Eye movement research has shown that attention shifts from the currently fixated location to the next before a saccade is executed. We investigated whether the cost of the attention shift depends on higher-order processing at the time of fixation, in particular on visual working memory load differences between fixations and refixations on task-relevant items. The attention shift is reflected in EEG activity in the saccade-related potential (SRP). In a free viewing task involving visual search and memorization of multiple targets amongst distractors, we compared the SRP in first fixations versus refixations on targets and distractors. The task-relevance of targets implies that more information will be loaded in memory (e.g. both identity and location) than for distractors (e.g. location only). First fixations will involve greater memory load than refixations, since first fixations involve loading of new items, while refixations involve rehearsal of previously visited items. The SRP in the interval preceding the saccade away from a target or distractor revealed that saccade preparation is affected by task-relevance and refixation behavior. For task-relevant items only, we found longer fixation duration and higher SRP amplitudes for first fixations than for refixations over the occipital region and the opposite effect over the frontal region. Our findings provide first neurophysiological evidence that working memory loading of task-relevant information at fixation affects saccade planning. [ABSTRACT FROM AUTHOR]

Highlights • Fixation-related potentials were studied in visual search with free eye movements. • The P300 is a neural correlate of target detection in visual search. • The amplitude of the target-related P300 depends on target probability. Abstract This study investigated the influence of target probability on the neural response to target detection in free viewing visual search. Participants were asked to indicate the number of targets (one or two) among distractors in a visual search task while EEG and eye movements were co-registered. Target probability was manipulated by varying the set size of the displays between 10, 22, and 30 items. Fixation-related potentials time-locked to first target fixations revealed a pronounced P300 at the centro-parietal cortex with larger amplitudes for set sizes 22 and 30 than for set size 10. With increasing set size, more distractor fixations preceded the detection of the target, resulting in a decreased target probability and, consequently, a larger P300. For distractors, no increase of P300 amplitude with set size was observed. The findings suggest that set size specifically affects target but not distractor processing in overt serial visual search. [ABSTRACT FROM AUTHOR]

While viewing a scene, the eyes are attracted to salient stimuli. We set out to identify the brain signals controlling this process. In a contour integration task, in which participants searched for a collinear contour in a field of randomly oriented Gabor elements, a previously established model was applied to calculate a visual saliency value for each fixation location. We studied brain activity related to the modeled saliency values, using coregistered eye tracking and EEG. To disentangle EEG signals reflecting salience in free viewing from overlapping EEG responses to sequential eye movements, we adopted generalized additive mixed modeling (GAMM) to single epochs of saccade‐related EEG. We found that, when saliency at the next fixation location was high, amplitude of the presaccadic EEG activity was low. Since presaccadic activity reflects covert attention to the saccade target, our results indicate that larger attentional effort is needed for selecting less salient saccade targets than more salient ones. This effect was prominent in contour‐present conditions (half of the trials), but ambiguous in the contour‐absent condition. Presaccadic EEG activity may thus be indicative of bottom‐up factors in saccade guidance. The results underscore the utility of GAMM for EEG—eye movement coregistration research. [ABSTRACT FROM AUTHOR]