Search

The aim of this review is to highlight the idea of grounding social cognition in sensorimotor interactions shared across agents. We discuss an action-oriented account that emerges from a broader interpretation of the concept of sensorimotor contingencies. We suggest that dynamic informational and sensorimotor coupling across agents can mediate the deployment of action-effect contingencies in social contexts. We propose this concept of socializing sensorimotor contingencies (socSMCs) as a shared framework of analysis for processes within and across brains and bodies, and their physical and social environments. In doing so, we integrate insights from different fields, including neuroscience, psychology, and research on human–robot interaction. We review studies on dynamic embodied interaction and highlight empirical findings that suggest an important role of sensorimotor and informational entrainment in social contexts. Furthermore, we discuss links to closely related concepts, such as enactivism, models of coordination dynamics and others, and clarify differences to approaches that focus on mentalizing and high-level cognitive representations. Moreover, we consider conceptual implications of rethinking cognition as social sensorimotor coupling. The insight that social cognitive phenomena like joint attention, mutual trust or empathy rely heavily on the informational and sensorimotor coupling between agents may provide novel remedies for people with disturbed social cognition and for situations of disturbed social interaction. Furthermore, our proposal has potential applications in the field of human–robot interaction where socSMCs principles might lead to more natural and intuitive interfaces for human users. [ABSTRACT FROM AUTHOR]

Conventional visual BCIs, in which control channels are tagged with stimulation patterns to elicit distinguishable brain patterns, has made impressive progress in terms of the information transfer rates (ITRs). However, less development has been seen with respect to user experience and complexity of the technical setup. The requirement to tag each of targets by a unique stimulus substantially limits the flexibility of conventional visual BCI systems. A method for decoding the targets in the environment flexibly was therefore proposed in the present study. A BCI speller with thirteen symbols drawn on paper was developed. The symbols were interspersed with four flickers with distinct frequencies, but the user did not have to gaze at flickers. Rather, subjects could spell a sequence by looking at the symbols on the paper. In a cue-guided spelling task, the average offline and online accuracies reached 89.3± 7.3% and 90.3± 6.9% for 13 subjects, corresponding to ITRs of 43.0± 7.4 bit/min and 43.8± 6.8 bit/min. In an additional free-spelling task for seven out of thirteen subjects, an accuracy of 92.3± 3.1% and an ITR of 45.6± 3.3 bit/min were achieved. Analysis of a simulated online system showed the possibility to reach an average ITR of 105.8 bit/min by reducing the epoch duration from 4 to 1 second. Reliable BCI control is possible by gazing at targets in the environment instead of dedicated stimuli which encode control channels. The proposed method can drastically reduce the technical effort for visual BCIs and thereby advance their applications outside the laboratory. [ABSTRACT FROM AUTHOR]

Whereas the fundamental role of the body in social cognition seems to be generally accepted, elucidating the bodily mechanisms associated with non-verbal communication and cooperation between two or more persons is still a challenging endeavor. In this article we propose a fresh approach for investigating the function of the autonomic nervous system that is reflected in parameters of heart rate variability, respiration, and electrodermal activity in a social setting. We analyzed autonomic parameters of dyads solving a target-tracking task together with the partner or individually. A machine classifier was trained to predict the subjects' rating of performance and collaboration either from tracking error data or from the set of autonomic parameters. When subjects collaborated, this classifier could predict the subjective performance ratings better from the autonomic response than from the objective performance of the subjects. However, when they solved the task individually, predictability from autonomic parameters dropped to the level of objective performance, indicating that subjects were more rational in rating their performance in this condition. Moreover, the model captured general knowledge about the population that allows it to predict the performance ratings of an unseen subject significantly better than chance. Our results suggest that, in particular in situations that require collaboration with others, evaluation of performance is shaped by the bodily processes that are quantified by autonomic parameters. Therefore, subjective performance assessments appear to be modulated not only by the output of a rational or discriminative system that tracks the objective performance but to a significant extent also by interoceptive processes. [ABSTRACT FROM AUTHOR]

Learning and memorizing sequences of events is an important function of the human brain and the basis for forming expectations and making predictions. Learning is facilitated by repeating a sequence several times, causing rhythmic appearance of the individual sequence elements. This observation invites to consider the resulting multitude of rhythms as a spectral "fingerprint" which characterizes the respective sequence. Here we explore the implications of this perspective by developing a neurobiologically plausible computational model which captures this "fingerprint" by attuning an ensemble of neural oscillators. In our model, this attuning process is based on a number of oscillatory phenomena that have been observed in electrophysiological recordings of brain activity like synchronization, phase locking, and reset as well as cross-frequency coupling. We compare the learning properties of the model with behavioral results from a study in human participants and observe good agreement of the errors for different levels of complexity of the sequence to be memorized. Finally, we suggest an extension of the model for processing sequences that extend over several sensory modalities. [ABSTRACT FROM AUTHOR]

Objective: We present a new type of brain-computer interface (BCI) that utilizes the retinotopic mapping of motion-onset visual evoked potentials (mVEP) to accomplish four control channels using a single motion stimulus. Methods: Participants selected a BCI command by fixating one of four target locations around a centrally presented visual motion stimulus. A template-matching method was employed to recognize the users’ intention by decoding the position of the motion stimulus in the peripheral visual field, and classification performances were evaluated in an offline manner. The motion stimulus eccentricity between the targets and the visual motion stimulus varied among 5.1°, 6.7°, 9.8°, and 13.0°. Results: Distinct N200 spatial patterns were elicited when participants directed attention overtly to the target locations. A four-class classification accuracy of 72.2 ± 5.05% was achieved with a distance of 5.1° visual angle between the targets and the visual motion stimulus. The classification accuracies decreased with increasing motion stimulus eccentricities but remained separable well above the chance level at 13.0° (47.3 ± 8.54%). Conclusion: Our results support the feasibility of a single-stimulus, multitarget mVEP BCI. Significance: The proposed system can simplify the visual stimulation of mVEP BCIs, improve user experience and pave the way for simple yet efficient BCI communication. [ABSTRACT FROM AUTHOR]

When two individuals interact in a collaborative task, such as carrying a sofa or a table, usually spatiotemporal coordination of individual motor behavior will emerge. In many cases, interpersonal coordination can arise independently of verbal communication, based on the observation of the partners' movements and/or the object's movements. In this study, we investigate how social coupling between two individuals can emerge in a collaborative task under different modes of perceptual information. A visual reference condition was compared with three different conditions with new types of additional auditory feedback provided in real time: effect-based auditory feedback, performance-based auditory feedback, and combined effect/performance-based auditory feedback. We have developed a new paradigm in which the actions of both participants continuously result in a seamlessly merged effect on an object simulated by a tablet computer application. Here, participants should temporally synchronize their movements with a 90° phase difference and precisely adjust the finger dynamics in order to keep the object (a ball) accurately rotating on a given circular trajectory on the tablet. Results demonstrate that interpersonal coordination in a joint task can be altered by different kinds of additional auditory information in various ways. [ABSTRACT FROM AUTHOR]

In brain&#x2013.computer interfaces (BCIs) that use the steady-state visual evoked response (SSVEP), the user selects a control command by directing attention overtly or covertly to one out of several flicker stimuli. The different control channels are encoded in the frequency, phase, or time domain of the flicker signals. Here, we present a new type of SSVEP BCI, which uses only a single flicker stimulus and yet affords controlling multiple channels. The approach rests on the observation that the relative position between the stimulus and the foci of overt attention result in distinct topographies of the SSVEP response on the scalp. By classifying these topographies, the computer can determine at which position the user is gazing. Offline data analysis in a study on 12 healthy volunteers revealed that 9 targets can be recognized with about 95±3% accuracy, corresponding to an information transfer rate (ITR) of 40.8 ± 3.3 b/min on average. We explored how the classification accuracy is affected by the number of control channels, the trial length, and the number of EEG channels. Our findings suggest that the EEG data from five channels over parieto-occipital brain areas are sufficient for reliably classifying the topographies and that there is a large potential to improve the ITR by optimizing the trial length. The robust performance and the simple stimulation setup suggest that this approach is a prime candidate for applications on desktop and tablet computers. [ABSTRACT FROM AUTHOR]

A promising approach for brain-computer interfaces (BCIs) employs the steady-state visual evoked potential (SSVEP) for extracting control information. Main advantages of these SSVEP BCIs are a simple and low-cost setup, little effort to adjust the system parameters to the user and comparatively high information transfer rates (ITR). However, traditional frequency-coded SSVEP BCIs require the user to gaze directly at the selected flicker stimulus, which is liable to cause fatigue or even photic epileptic seizures. The spatially coded SSVEP BCI we present in this article addresses this issue. It uses a single flicker stimulus that appears always in the extrafoveal field of view, yet it allows the user to control four control channels. We demonstrate the embedding of this novel SSVEP stimulation paradigm in the user interface of an online BCI for navigating a 2-dimensional computer game. Offline analysis of the training data reveals an average classification accuracy of 96.9±1.64%, corresponding to an information transfer rate of 30.1±1.8 bits/min. In online mode, the average classification accuracy reached 87.9±11.4%, which resulted in an ITR of 23.8±6.75 bits/min. We did not observe a strong relation between a subject’s offline and online performance. Analysis of the online performance over time shows that users can reliably control the new BCI paradigm with stable performance over at least 30 minutes of continuous operation. [ABSTRACT FROM AUTHOR]

Compared to the different brain signals used in brain–computer interface (BCI) paradigms, the s teady-state visually evoked potential (SSVEP) features a high signal to noise ratio, enabling reliable and fast classification of neural activity patterns without extensive training requirements. In this paper, we present methods to further increase the information transfer rates (ITRs) of SSVEP-based BCIs. Starting with stimulus parameter optimizations methods, we develop an improved approach for the use of Canonical correlation analysis and analyze properties of the SSVEP when the user fixates a target and during transitions between targets. These transitions show a negative effect on the system's ITR which we trace back to delays and dead times of the SSVEP. Using two classifier types adapted to continuous and transient SSVEPs and two control modes (fast feedback and fast input), we present a simulated online BCI implementation which addresses the challenges introduced by transient SSVEPs. The resulting system reaches an average ITR of 181 Bits/min and peak ITR values of up to 295 Bits/min for individual users. [ABSTRACT FROM AUTHOR]

When people speak about consciousness, they distinguish various types and different levels, and they argue for different concepts of cognition. This complicates the discussion about artificial or machine consciousness. Here we take a bottom-up approach to this question by presenting a family of robot experiments that invite us to think about consciousness in the context of artificial agents. The experiments are based on a computational model of sensorimotor contingencies. It has been suggested that these regularities in the sensorimotor flow of an agent can explain raw feels and perceptual consciousness in biological agents. We discuss the validity of the model with respect to sensorimotor contingency theory and consider whether a robot that is controlled by knowledge of its sensorimotor contingencies could have any form of consciousness. We propose that consciousness does not require higher-order thought or higher-order representations. Rather, we argue that consciousness starts when (i) an agent actively (endogenously triggered) uses its knowledge of sensorimotor contingencies to issue predictions and (ii) when it deploys this capability to structure subsequent action. [ABSTRACT FROM AUTHOR]

One of the main assertions of sensorimotor contingency theory is that sensory experience is not generated by activating an internal representation of the outside world through sensory signals, but corresponds to a mode of exploration and hence is an active process. Perception and sensory awareness emerge from using the structure of changes in the sensory input resulting from these exploratory actions, called sensorimotor contingencies (SMCs), for planning, reasoning, and goal achievement. Using a previously developed computational model of SMCs we show how an artificial agent can plan ahead with SMCs and use them for action guidance. Our main assumption is that SMCs are associated with a utility for the agent, and that the agent selects actions that maximize this utility. We analyze the properties of the resulting actions in a robot that is endowed with several sensory modalities and controlled by our model in a simple environment. The results demonstrate that its actions avoid aversive events, and that it can achieve a low-level form of spatial awareness that is resilient to the complete loss of a sensory modality. [ABSTRACT FROM AUTHOR]

The feasibility of a steady-state visual evoked potential (SSVEP) brain–computer interface (BCI) with a single-flicker stimulus for multiple-target decoding has been demonstrated in a number of recent studies. The single-flicker BCIs have mainly employed the direction information for encoding the targets, i.e., different targets are placed at different spatial directions relative to the flicker stimulus. The present study explored whether visual eccentricity information can also be used to encode targets for the purpose of increasing the number of targets in the single-flicker BCIs. A total number of 16 targets were encoded, placed at eight spatial directions, and two eccentricities (2.5° and 5°) relative to a 12 Hz flicker stimulus. Whereas distinct SSVEP topographies were elicited when participants gazed at targets of different directions, targets of different eccentricities were mainly represented by different signal-to-noise ratios (SNRs). Using a canonical correlation analysis-based classification algorithm, simultaneous decoding of both direction and eccentricity information was achieved, with an offline 16-class accuracy of 66.8 ± 16.4% averaged over 12 participants and a best individual accuracy of 90.0%. Our results demonstrate a single-flicker BCI with a substantially increased target number towards practical applications. [ABSTRACT FROM AUTHOR]

Given the features of a video, recurrent neural networks can be used to automatically generate a caption for the video. Existing methods for video captioning have at least three limitations. First, semantic information has been widely applied to boost the performance of video captioning models, but existing networks often fail to provide meaningful semantic features. Second, the Teacher Forcing algorithm is often utilized to optimize video captioning models, but during training and inference, different strategies are applied to guide word generation, leading to poor performance. Third, current video captioning models are prone to generate relatively short captions that express video contents inappropriately. Toward resolving these three problems, we suggest three corresponding improvements. First of all, we propose a metric to compare the quality of semantic features, and utilize appropriate features as input for a semantic detection network (SDN) with adequate complexity in order to generate meaningful semantic features for videos. Then, we apply a scheduled sampling strategy that gradually transfers the training phase from a teacher-guided manner toward a more self-teaching manner. Finally, the ordinary logarithm probability loss function is leveraged by sentence length so that the inclination of generating short sentences is alleviated. Our model achieves better results than previous models on the YouTube2Text dataset and is competitive with the previous best model on the MSR-VTT dataset., (Copyright © 2020 Chen, Lin, Maye, Li and Hu.)

Modulation of steady-state visual evoked potential (SSVEP) by directing gaze to targets flickering at different frequencies has been utilized in many brain-computer interface (BCI) studies. However, this paradigm may not work with patients suffering from complete locked-in syndrome or other severe motor disabilities that do not allow conscious control of gaze direction. In this paper, we present a novel, independent BCI paradigm based on covert shift of non-spatial visual selective attention. Subjects viewed a display consisting of two spatially overlapping sets of randomly positioned dots. The two dot sets differed in color, motion and flickering frequency. Two types of motion, rotation and linear motion, were investigated. Both, the SSVEP amplitude and phase response were modulated by selectively attending to one of the two dot sets. Offline analysis revealed a predicted online classification accuracy of 69.3+/-10.2% for the rotating dots, and 80.7+/-10.4% for the linearly moving dots.

Brains are usually described as input/output systems: they transform sensory input into motor output. However, the motor output of brains (behavior) is notoriously variable, even under identical sensory conditions. The question of whether this behavioral variability merely reflects residual deviations due to extrinsic random noise in such otherwise deterministic systems or an intrinsic, adaptive indeterminacy trait is central for the basic understanding of brain function. Instead of random noise, we find a fractal order (resembling Lévy flights) in the temporal structure of spontaneous flight maneuvers in tethered Drosophila fruit flies. Lévy-like probabilistic behavior patterns are evolutionarily conserved, suggesting a general neural mechanism underlying spontaneous behavior. Drosophila can produce these patterns endogenously, without any external cues. The fly's behavior is controlled by brain circuits which operate as a nonlinear system with unstable dynamics far from equilibrium. These findings suggest that both general models of brain function and autonomous agents ought to include biologically relevant nonlinear, endogenous behavior-initiating mechanisms if they strive to realistically simulate biological brains or out-compete other agents.