Search

Visual search and working memory (WM) are tightly linked cognitive processes. Theories of attentional selection assume that WM plays an important role in top-down guided visual search. However, computational models of visual search do not model WM. Here we show that an existing model of WM can utilize its mechanisms of rapid plasticity and pattern completion to perform visual search. In this model, a search template, like a memory item, is encoded into the network's synaptic weights forming a momentary stable attractor. During search, recurrent activation between the template and visual inputs amplifies the target and suppresses non-matching features via mutual inhibition. While the model cannot outperform models designed specifically for search, it can, "off-the-shelf", account for important characteristics. Notably, it produces search display set-size costs, repetition effects, and multiple-template search effects, qualitatively in line with empirical data. It is also informative that the model fails to produce some important aspects of visual search behaviour, such as suppression of repeated distractors. Also, without additional control structures for top-down guidance, the model lacks the ability to differentiate between encoding and searching for targets. The shared architecture bridges theories of visual search and visual WM, highlighting their common structure and their differences. [ABSTRACT FROM AUTHOR]

Can individuals look for multiple objects at the same time? A simple question, but answering it has proven difficult. In this review, we describe possible cognitive architectures and their predictions about the capacity of visual search. We broadly distinguish three stages at which limitations may occur: (1) preparation (establishing and maintaining a mental representation of a search target), (2) selection (using this mental representation to extract candidate targets from the visual input), and (3) post-selection processing (verifying that the selected information actually is a target). We then review the empirical evidence from various paradigms, together with their strengths and pitfalls. The emerging picture is that multiple target search comes with costs, but the magnitude of this cost differs depending on the processing stage. Selection appears strongly limited, while preparation of multiple search target representations in anticipation of a search is possible with relatively small costs. Finally, there is currently not sufficient information to determine the capacity limitations of post-selection processing. We hope that our review contributes to better targeted research into the mechanisms of multiple-target search. A better understanding of multiple-target search will also contribute to better design of real-life multiple-target search problems, reducing the risk of detrimental search failures. [ABSTRACT FROM AUTHOR]

Visual attention serves to select relevant visual information. However, observers often first need to find out what is relevant. Little is known about this information-seeking process and how it affects attention. We employed a cued visual search task in combination with eye tracking to investigate which oculomotor measures reflect the acquisition of information for a subsequent task. A cue indicated as to which target to look for in a following search display. Cue-target combinations were repeated several times, enabling learning of the target. We found that reductions in cue fixation times and saccade size provided stable indices of learning. Despite the learning, participants continued to attend to repeated cues. Several factors contribute to people attending to information they already know: First, the information value provided by the cue continues to drive attention. Second, even in the absence of information value, attention continues to be directed to cue features that previously signalled relevant information. Third, the decision to attend to a known cue depends on cognitive effort. We propose that this combination of information value, previous relevance, and effort is best captured within an information-seeking framework, and that oculomotor parameters provide a useful proxy for uncovering these factors and their interactions. [ABSTRACT FROM AUTHOR]

Information in visual working memory that is only prospectively relevant can nevertheless guide attention towards memory matching visual input. Previous studies demonstrated that such memory-based attentional biases can be modulated by top-down processing strategies. Here we examined whether attentional capture by memory matching distractors is also modulated by more implicit biases stemming from selection history. Observers performed a visual search task while holding a colour in memory for a subsequent task. Crucially, a coloured distractor in the search display not only matched the memory content half of the time, it also appeared on one location more often than on all other locations. Consistent with statistical learning having a strong impact on attentional priorities, attentional capture by the distractor was attenuated at high probability distractor locations. The additional slowing by memory matching distractors, however, was the same at suppressed and non-suppressed locations. We interpret this finding as evidence that memory-based feature biases are independent from learned spatial biases. [ABSTRACT FROM AUTHOR]

Many important situations require human observers to simultaneously search for more than one object. Despite a long history of research into visual search, the behavioral and neural mechanisms associated with multiple-target search are poorly understood. Here we test the novel theory that the efficiency of looking for multiple targets critically depends on the mode of cognitive control the environment affords to the observer. We used an innovative combination of electroencephalogram (EEG) and eye tracking while participants searched for two targets, within two different contexts: either both targets were present in the search display and observers were free to prioritize either one of them, thus enabling proactive control over selection; or only one of the two targets would be present in each search display, which requires reactive control to reconfigure selection when the wrong target has been prioritized. During proactive control, both univariate and multivariate signals of beta-band (15-35 Hz) power suppression before display onset predicted switches between target selections. This signal originated over midfrontal and sensorimotor regions and has previously been associated with endogenous state changes. In contrast, imposed target selections requiring reactive control elicited prefrontal power enhancements in the delta/theta band (2- 8 Hz), but only after display onset. This signal predicted individual differences in associated oculomotor switch costs, reflecting reactive reconfiguration of target selection. The results provide compelling evidence that multiple target representations are differentially prioritized during visual search, and for the first time reveal distinct neural mechanisms underlying proactive and reactive control over multiple-target search. [ABSTRACT FROM AUTHOR]

A “target template”, specifying target features, is thought to benefit visual search performance. Setting up a “negative template”, specifying distractor features, should improve distractor inhibition and also benefit target detection. In the current fMRI study, subjects were required to search for a target among distractors enclosed in coloured circles. Before search, one of three colour cues appeared: a positive cue indicating the target will appear in the same colour, a negative cue indicating only distractors will appear in the same colour, or a neutral cue indicating that the same colour will not appear in the search display. fMRI results revealed down-regulation of neural processing in large parts of visual cortex following negative compared to positive cues. We further found a general attention inhibition mechanism in SPL/precuneus for neutral cues compared to positive and negative cues. These results suggest a cortical distinction between target templates, negative templates, and task-irrelevant distractor inhibition. [ABSTRACT FROM AUTHOR]

Objects in visual working memory (VWM) that are only prospectively relevant can nevertheless affect the guidance of attention in an ongoing visual search task. Here we investigated whether learning changes the attentional status of such prospective memories. Observers performed a visual search while holding an item in memory for a later memory test. This prospective memory was then repeated for several trials. When the memory was new, it interfered with the ongoing search task. However, with repetition, memory performance increased but memory-based interference rapidly diminished, suggesting that observers learned to shield the prospective memory from the ongoing task. This contrasts with earlier findings showing stronger attentional biases from learned memories when these are immediately task-relevant. Interestingly, interference resurfaced again in anticipation of a new memory, suggesting a reactivation of VWM. These effects were sensitive to task context, indicating that the attentional status of prospective memories is flexible. [ABSTRACT FROM AUTHOR]

There is evidence that the deployment of attention can be biased by the content of visual working memory. Recently, it has been shown that focusing internal attention to a specific item in memory not only increases the accessibility of that specific item for retrieval, but also results in increased attentional guidance toward matching external stimuli by that item. Here, we investigated the time course of attentional guidance by cued memories. Following a retro-cue that prioritized one of two memory items, participants performed a visual search task. The interval between the cue and the search display was varied. Consistent with earlier findings, we observed memory-related capture when the search display contained a distractor that was related to the cued item in memory. No such effects were found for the uncued items or when none of the memory items were prioritized by a retro-cue. Results suggest that the prioritization by a retro-cue is a very rapid process that starts to affect perceptual selection within the first 100-200 ms following the cue. [ABSTRACT FROM AUTHOR]

Abstract: When observers perform a visual search task, they are assumed to adopt an attentional set for what they are looking for. The present experiment investigates the influence of long-term visual memory associations on this attentional set. On each trial, observers were asked to search a display for a grayscale version of a known traffic sign. On each trial, a distractor sign was drawn in full color. This color could either be related or unrelated to the target sign. Distractors interfered more with search when their color was related (e.g. red when the target was a stop sign), implying that long-term color associations resulted in inadvertent attentional guidance, even though color was irrelevant to the task. The results add to the growing body of evidence that long-term memory representations automatically affect attentional orienting. [Copyright &y& Elsevier]

Abstract: In the visual world paradigm as used in psycholinguistics, eye gaze (i.e. visual orienting) is measured in order to draw conclusions about linguistic processing. However, current theories are underspecified with respect to how visual attention is guided on the basis of linguistic representations. In the visual search paradigm as used within the area of visual attention research, investigators have become more and more interested in how visual orienting is affected by higher order representations, such as those involved in memory and language. Within this area more specific models of orienting on the basis of visual information exist, but they need to be extended with mechanisms that allow for language-mediated orienting. In the present paper we review the evidence from these two different – but highly related – research areas. We arrive at a model in which working memory serves as the nexus in which long-term visual as well as linguistic representations (i.e. types) are bound to specific locations (i.e. tokens or indices). The model predicts that the interaction between language and visual attention is subject to a number of conditions, such as the presence of the guiding representation in working memory, capacity limitations, and cognitive control mechanisms. [Copyright &y& Elsevier]

Abstract: Previous work has shown that distractors present in a visual search display attract attention when they match objects kept in visual working memory. It seems that maintaining an object in working memory is functionally identical to adopting an attentional set for that object. We test this conjecture by asking observers to perform a memory task as well as a visual search task (in which memory-related distractors could return), but to leave the observer uncertain as to which of these tasks would have to be completed first. This way, observers ought to more readily look for the memorized information, rather than just remember it. Memory-related distractor effects were larger than when participants knew the order of the tasks beforehand, consistent with the idea that trying to attend to something involves additional processes or representations beyond those needed for simply storing an item. [Copyright &y& Elsevier]

Abstract: Visual search for a target among distractors is often speeded when the target-defining feature is repeated from trial to trial, compared to when it changes. It has been proposed that whether this intertrial priming effect is perceptual in nature or not, and how strong it is, depends on the perceptual ambiguity of the display. Using the event-related potential (ERP) method, Experiment 1 assessed whether perceptual ambiguity indeed resulted in increased priming, by comparing target repetitions and changes in a condition with a target-resembling distractor to a condition without such a distractor. Priming effects on response times were indeed greater in the presence of a distractor. Moreover, EEG analyses revealed latency shifts and amplitude differences in the P1 and N2pc components for the same condition, consistent with a perceptual locus of intertrial priming. Experiment 2 confirmed this by varying the perceptual similarity of the distractor to the target. Priming benefits increased with increasing similarity. [Copyright &y& Elsevier]

Abstract: Previous research has revealed that we create and maintain mental representations for perceived objects on the basis of their spatiotemporal continuity. An important question is what type of information can be maintained within these so-called object files. We provide evidence that object files retain specific attentional control settings for items presented inside the object, even when it disappears from vision. The objects were entire visual search displays consisting of multiple items moving into and out of view. It was demonstrated that search was speeded when the search target position was repeated from trial to trial, but especially so when spatiotemporal continuity suggested that the entire display was the same object. We conclude that complete spatial attentional biases can be stored in an object file. [Copyright &y& Elsevier]

Searching for an object within a cluttered, continuously changing environment can be a very time-consuming process. The authors show that a simple auditory pip drastically decreases search times for a synchronized visual object that is normally very difficult to find. This effect occurs even though the pip contains no information on the location or identity of the visual object. The experiments also show that the effect is not due to general alerting (because it does not occur with visual cues), nor is it due to top-down cuing of the visual change (because it still occurs when the pip is synchronized with distractors on the majority of trials). Instead, we propose that the temporal information of the auditory signal is integrated with the visual signal, generating a relatively salient emergent feature that automatically draws attention. Phenomenally, the synchronous pip makes the visual object pop out from its complex environment, providing a direct demonstration of spatially nonspecific sounds affecting competition in spatial visual processing. [ABSTRACT FROM AUTHOR]

Perceptual grouping is crucial to distinguish objects from their background. Recent studies have shown that observers can detect an object that does not have any unique qualities other than unique temporal properties. A crucial question is whether focused attention is needed for this type of grouping. In two visual search experiments, we show that searching for an object defined by temporal grouping can occur in parallel. These findings suggest that focused attention is not needed for temporal grouping to occur. It is proposed that temporal grouping may occur because the neurons representing the changing object elements adopt firing frequencies that cause the visual system to bind these elements together without the need for focused attention. [ABSTRACT FROM AUTHOR]

Intuitively, dynamic visual stimuli, such as moving objects or flashing lights, attract attention. Visual search tasks have revealed that dynamic targets among static distractors can indeed efficiently guide attention. The present study shows that the reverse case, a static target among dynamic distractors, allows for relatively efficient selection in certain but not all cases. A static target was relatively efficiently found among distractors that featured apparent motion, corroborating earlier findings. The important new finding was that static targets were equally easily found among distractors that blinked on and off continuously, even when each individual item blinked at a random rate. However, search for a static target was less efficient when distractors abruptly varied in luminance but did not completely disappear. The authors suggest that the division into the parvocellular pathway dealing with static visual information, on the one hand, and the magnocellular pathway common to motion and new object onset detection, on the other hand, allows for efficient filtering of dynamic and static information. [ABSTRACT FROM AUTHOR]

Adaptive behaviour requires the selection of relevant, and the ignoring of irrelevant visual information. Previous work has shown that prior information about target (i.e., relevant) and non-target (i.e., irrelevant) objects facilitates such selection, presumably by enabling observers to create a working memory template for or against such objects. Using a cued visual search task, here we aimed to investigate whether, and how, target and non-target templates differ in terms of neural representation, as measured through fMRI activity patterns. Two hypotheses were tested: (1) while target representations are activated, non-target representations are suppressed prior to search, and thus these representations differ, and (2) target and non-target templates share similar initial representations but involve different control signals in anticipation of, or upon encountering the search display. Behaviourally, both target and non-target cues facilitated search compared to baseline. Using multivariate pattern and representational similarity analyses, the findings revealed little support for the suppression hypothesis. Posterior brain regions primarily coded for the category content of the template and did so similarly for target and non-target information. Frontal cortical areas, particularly the lateral prefrontal cortex and frontal eye fields, instead showed sensitivity to the status of the template, regardless of the object category it belonged to. These results are most consistent with the hypothesis that prior to selection, target and non-target information are represented similarly in terms of content, but differ in terms of the associated control signals. [ABSTRACT FROM AUTHOR]

Abstract Daily life is filled with sequences of multiple tasks, each with their own relevant perceptual input. Working memory needs to dissociate representations that drive attention towards currently relevant information from prospective representations that are needed for future tasks, but that until then should be prevented from guiding attention. Yet, little is known about how the brain initiates and controls such sequential prioritization of selection. In the current study we recorded EEG while subjects remembered a color as the target template for one of two sequential search tasks, thus making it either currently relevant (when it was the target for the first search) or prospectively relevant (when it was the target for the second search) prior to the task sequence. Using time-frequency specific linear classifiers, we were able to predict the priority status (current versus prospective) of the memory representation from multivariate patterns of delta (2–4 Hz) and non-lateralized alpha power (8–14 Hz) during both delay periods. The delta band was only transiently involved when initializing the priority status at the start of the first delay, or when switching priority during the second delay, which we interpret as reflecting the momentary top-down control over prioritization. In contrast, alpha power decoding was based on a more stable pattern of activity that generalized across time both within and between delay periods, which we interpret as reflecting a difference in the prioritized memory representations themselves. Taken together, we reveal the involvement of a complex, distributed and dynamic spatiotemporal landscape of frequency-specific oscillatory activity in controlling prioritization of information within working memory. Highlights • MVPA of EEG oscillatory signals dissociates current from future perceptual tasks • Current versus future status can be decoded during the delay prior to the tasks • Status decoding is reflected specifically in oscillatory delta and alpha band activity • Spatial pattern of activity underlying status decoding is complex and distributed • MVPA more sensitive than univariate measures to study higher order brain states [ABSTRACT FROM AUTHOR]

It is debated whether people can actively search for more than one object or whether this results in switch costs. Using a gaze-contingent eye-tracking paradigm, we revealed a crucial role for cognitive control in multiple-target search. We instructed participants to simultaneously search for two target objects presented among distractors. In one condition, both targets were available, which gave the observer free choice of what to search for and allowed for proactive control. In the other condition, only one of the two targets was available, so that the choice was imposed, and a reactive mechanism would be required. No switch costs emerged when target choice was free, but switch costs emerged reliably when targets were imposed. Bridging contradictory findings, the results are consistent with models of visual selection in which only one attentional template actively drives selection and in which the efficiency of switching targets depends on the type of cognitive control allowed for by the environment. [ABSTRACT FROM AUTHOR]

Previous research has shown that when observers search for targets defined by a particular colour, attention can be directed rapidly and independently to two target objects that appear in close temporal proximity. We investigated how such rapid attention shifts are modulated by task instructions to selectively attend versus ignore one of these objects. Two search displays that both contained a colour-defined target and a distractor in a different colour were presented in rapid succession, with a stimulus onset asynchrony (SOA) of 100 ms. In different blocks, participants were instructed to attend and respond to target-colour objects in the first display and to ignore these objects in the second display, or vice versa. N2pc components were measured to track the allocation of spatial attention to target-colour objects in these two displays. When participants responded to the second display, irrelevant target-colour objects in the first display still triggered N2pc components, demonstrating task-set contingent attentional capture while a feature-specific target template is active. Critically, when participants responded to the first display instead, no N2pc was elicited by target-colour items in the second display, indicating that they no longer rapidly captured attention. However, these items still elicited a longer-latency contralateral negativity (SPCN component), suggesting that attention was oriented towards template-matching objects in working memory. This dissociation between N2pc and SPCN components shows that rapid attentional capture and subsequent attentional selection processes within working memory can be independent. We suggest that early attentional orienting mechanisms can be inhibited when task-set matching objects are no longer task-relevant, and that this type of inhibitory control is a rapid but transient process. [ABSTRACT FROM AUTHOR]

An important question is, to what extent is visual attention driven by the semantics of individual objects, rather than by their visual appearance? This study investigates the hypothesis that timing is a crucial factor in the occurrence and strength of semantic influences on visual orienting. To assess the dynamics of such influences, the authors presented the target instruction either before or after visual stimulus onset, while eye movements were continuously recorded throughout the search. The results show a substantial but delayed bias in orienting toward semantically related objects compared with visually related objects when target instruction is presented before visual stimulus onset. However, this delay can be completely undone by presenting the visual information before the target instruction (Experiment 1). Moreover, the absence or presence of visual competition does not change the temporal dynamics of the semantic bias (Experiment 2). Visual orienting is thus driven by priority settings that dynamically shift between visual and semantic representations, with each of these types of bias operating largely independently. The findings bridge the divide between the visual attention and the psycholinguistic literature. [ABSTRACT FROM AUTHOR]

Recent studies have revealed that the deployment of attention can be biased by the content of visual working memory (VWM), but that stored memories do not always interact with attention. This has led to a model which proposes a division within VWM between a single active template that interacts with perception and multiple accessory representations that do not. The present study was designed to study whether multiple memory representations are able to bias attention. Participants performed a visual search task while maintaining a variable number of colors in VWM. Consistent with earlier findings, we observed increased attentional capture by memory related distractors when VWM was filled with a single item. However, memory related capture was no longer present for memory loads beyond a single item. The absence of memory related capture at higher VWM loads was independent of individual VWM capacity, nor was it attributable to weaker encoding, forgetting, or reduced precision of memory representations. When analyses were limited to those trials in which participants had a relatively precise memory, there was still no sign of attentional guidance at higher loads. However, when observers were cued toward a specific memory item after encoding, interference with search returned. These results are consistent with a distinction within VWM between representations that interact with perception and those that do not, and show that only a single VWM representation at a time can interact with visual attention. [ABSTRACT FROM AUTHOR]

Visual search requires the maintenance of a search template in visual working memory in order to guide attention towards the target. This raises the question whether a search template is essentially the same as a visual working memory representation used in tasks that do not require attentional guidance, or whether it is a qualitatively different representation. Two experiments tested this by comparing electrophysiological markers of visual working memory maintenance between simple recognition and search tasks. For both experiments, responses were less rapid and less accurate in search task than in simple recognition. Nevertheless, the contralateral delay activity (CDA), an index of quantity and quality of visual working memory representations, was equal across tasks. On the other hand, the late positive complex (LPC), which is sensitive to the effort invested in visual working memory maintenance, was greater for the search task than the recognition task. Additionally, when the same target cue was repeated across trials (Experiment 2), the amplitude of visual working memory markers (both CDA and LPC) decreased, demonstrating learning of the target at an equal rate for both tasks. Our results suggest that a search template is qualitatively the same as a representation used for simple recognition, but greater effort is invested in its maintenance. [ABSTRACT FROM AUTHOR]

Abstract: In dynamic cluttered environments, audition and vision may benefit from each other in determining what deserves further attention and what does not. We investigated the underlying neural mechanisms responsible for attentional guidance by audiovisual stimuli in such an environment. Event-related potentials (ERPs) were measured during visual search through dynamic displays consisting of line elements that randomly changed orientation. Search accuracy improved when a target orientation change was synchronized with an auditory signal as compared to when the auditory signal was absent or synchronized with a distractor orientation change. The ERP data show that behavioral benefits were related to an early multisensory interaction over left parieto-occipital cortex (50–60ms post-stimulus onset), which was followed by an early positive modulation (80–100ms) over occipital and temporal areas contralateral to the audiovisual event, an enhanced N2pc (210–250ms), and a contralateral negative slow wave (CNSW). The early multisensory interaction was correlated with behavioral search benefits, indicating that participants with a strong multisensory interaction benefited the most from the synchronized auditory signal. We suggest that an auditory signal enhances the neural response to a synchronized visual event, which increases the chances of selection in a multiple object environment. [Copyright &y& Elsevier]

Cognitive control can involve proactive (preparatory) and reactive (corrective) mechanisms. Using a gaze-contingent eye tracking paradigm combined with fMRI, we investigated the involvement of these different modes of control and their underlying neural networks, when switching between different targets in multiple-target search. Participants simultaneously searched for two possible targets presented among distractors, and selected one of them. In one condition, only one of the targets was available in each display, so that the choice was imposed, and reactive control would be required. In the other condition, both targets were present, giving observers free choice over target selection, and allowing for proactive control. Switch costs emerged only when targets were imposed and not when target selection was free. We found differential levels of activity in the frontoparietal control network depending on whether target switches were free or imposed. Furthermore, we observed core regions of the default mode network to be active during target repetitions, indicating reduced control on these trials. Free and imposed switches jointly activated parietal and posterior frontal cortices, while free switches additionally activated anterior frontal cortices. These findings highlight unique contributions of proactive and reactive control during visual search. • We compared free versus imposed target selection in visual search. • Imposed target switches result in performance costs unlike self-initiated switches. • A frontoparietal control network differentiates between free and imposed selection. [ABSTRACT FROM AUTHOR]