Search

Developments in cognitive neuroscience have led to the emergence of hyperscanning, the simultaneous measurement of brain activity from multiple people. Hyperscanning is useful for investigating social cognition, including joint action, because of its ability to capture neural processes that occur within and between people as they coordinate actions toward a shared goal. Here, we provide a practical guide for researchers considering using hyperscanning to study joint action and seeking to avoid frequently raised concerns from hyperscanning skeptics. We focus specifically on EEG hyperscanning, which is widely available and optimally suited for capturing fine-grained temporal dynamics of action coordination. Our guidelines cover questions that are likely to arise when planning a hyperscanning project, ranging from whether hyperscanning is appropriate for answering one's research questions to considerations for study design, dependent variable selection, data analysis, and visualization. By following clear guidelines that facilitate careful consideration of the theoretical implications of research design choices and other methodological decisions, joint action researchers can mitigate interpretability issues and maximize the benefits of hyperscanning paradigms.

In small musical groups, performers can seem to coordinate their movements almost effortlessly in remarkable exhibits of joint action and entrainment. To achieve a common musical goal, co-performers interact and communicate using non-verbal means such as upper-body movements, and particularly head motion. Studying these phenomena in naturalistic contexts can be challenging since most techniques make use of motion capture technologies that can be intrusive and costly. To investigate an alternative method, we analyze video recordings of a professional instrumental ensemble by extracting trajectory information using pose estimation algorithms. We examine Kansei perspectives such as the analysis of non-verbal expression conveyed by bodily movements and gestures, and test for causal relationships and directed influence between performers using the Granger Causality method. We compute weighted probabilities representing the likelihood that each performer Granger Causes co-performers’ movements. Effects of different aspects of musical textures were examined and results indicated stronger directionality for homophonic textures (clear melodic leader) than polyphonic (ambiguous leadership).

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

Humans perceive and spontaneously move to one or several levels of periodic pulses (a meter, for short) when listening to musical rhythm, even when the sensory input does not provide prominent periodic cues to their temporal location. Here, we review a multi-levelled framework to understanding how external rhythmic inputs are mapped onto internally represented metric pulses. This mapping is studied using an approach to quantify and directly compare representations of metric pulses in signals corresponding to sensory inputs, neural activity and behaviour (typically body movement). Based on this approach, recent empirical evidence can be drawn together into a conceptual framework that unpacks the phenomenon of meter into four levels. Each level highlights specific functional processes that critically enable and shape the mapping from sensory input to internal meter. We discuss the nature, constraints and neural substrates of these processes, starting with fundamental mechanisms investigated in macaque monkeys that enable basic forms of mapping between simple rhythmic stimuli and internally represented metric pulse. We propose that human evolution has gradually built a robust and flexible system upon these fundamental processes, allowing more complex levels of mapping to emerge in musical behaviours. This approach opens promising avenues to understand the many facets of rhythmic behaviours across individuals and species. This article is part of the theme issue 'Synchrony and rhythm interaction: from the brain to behavioural ecology'.

People have a natural and intrinsic ability to coordinate body movements with rhythms surrounding them, known as sensorimotor synchronisation. This can be observed in daily environments, when dancing or singing along with music, or spontaneously walking, talking or applauding in synchrony with one another. However, the neurophysiological mechanisms underlying accurately synchronised movement with selected rhythms in the environment remain unclear. Here we studied real and imagined sensorimotor synchronisation with interleaved auditory and visual rhythms using cortico-muscular coherence (CMC) to better understand the processes underlying the preparation and execution of synchronised movement. Electroencephalography (EEG), electromyography (EMG) from the finger flexors, and continuous force signals were recorded in 20 participants during tapping and imagined tapping with discrete stimulus sequences consisting of alternating auditory beeps and visual flashes. The results show that the synchronisation between cortical and muscular activity in the beta (14-38 Hz) frequency band becomes time-locked to the taps executed in synchrony with the visual and auditory stimuli. Dynamic modulation in CMC also occurred when participants imagined tapping with the visual stimuli, but with lower amplitude and a different temporal profile compared to real tapping. These results suggest that CMC does not only reflect changes related to the production of the synchronised movement, but also to its preparation, which appears heightened under higher attentional demands imposed when synchronising with the visual stimuli. These findings highlight a critical role of beta band neural oscillations in the cortical-muscular coupling underlying sensorimotor synchronisation.

The ability to distinguish between an individual's own actions and those of another person is a requirement for successful joint action, particularly in domains such as group music making where precise interpersonal coordination ensures perceptual overlap in the effects of co-performers' actions. We tested the hypothesis that such coordination benefits from simultaneous integration and segregation of information about 'self' and 'other' in an experiment using a musical joint action paradigm. Sixteen pairs of individuals with little or no musical training performed a dyadic synchronization task on a pair of electronic music boxes. The relationship between pitches produced by paired participants (same vs. different) and the relationship between movement frequencies required to trigger synchronous tones (congruent vs. incongruent) were varied in a repeated measures design. The results indicate that interpersonal coordination was most accurate when sounds were different in pitch but movement frequency was congruent. Under other conditions, participants often drifted apart, resulting in poor coordination, especially with same sounds and incongruent movements across co-performers. These findings suggest that interpersonal coordination was facilitated when simultaneous self-other integration and segregation occurred across sensory modalities in an asymmetrical manner where pitch relations favoured segregation via auditory streaming while movement congruence favoured integration via visuo-motor coupling. Such self-other representational balance may enable co-performers to maintain autonomous control while attending, anticipating, and adapting to each other's timing when joint action requires precise temporal coordination.

When listening to music, people often perceive and move along with a periodic meter. However, the dynamics of mapping between meter perception and the acoustic cues to meter periodicities in the sensory input remain largely unknown. To capture these dynamics, we recorded the electroencephalography while nonmusician and musician participants listened to nonrepeating rhythmic sequences, where acoustic cues to meter frequencies either gradually decreased (from regular to degraded) or increased (from degraded to regular). The results revealed greater neural activity selectively elicited at meter frequencies when the sequence gradually changed from regular to degraded compared with the opposite. Importantly, this effect was unlikely to arise from overall gain, or low-level auditory processing, as revealed by physiological modeling. Moreover, the context effect was more pronounced in nonmusicians, who also demonstrated facilitated sensory-motor synchronization with the meter for sequences that started as regular. In contrast, musicians showed weaker effects of recent context in their neural responses and robust ability to move along with the meter irrespective of stimulus degradation. Together, our results demonstrate that brain activity elicited by rhythm does not only ref lect passive tracking of stimulus features, but represents continuous integration of sensory input with recent context.

Human movements often spontaneously fall into synchrony with auditory and visual environmental rhythms. Related behavioral studies have shown that motor responses are automatically and unintentionally coupled with external rhythmic stimuli. However, the neurophysiological processes underlying such motor entrainment remain largely unknown. Here, we investigated with electroencephalography (EEG) and electromyography (EMG) the modulation of neural and muscular activity induced by periodic audio and/or visual sequences. The sequences were presented at either 1 or 2 Hz, while participants maintained constant finger pressure on a force sensor. The results revealed that although there was no change of amplitude in participants' EMG in response to the sequences, the synchronization between EMG and EEG recorded over motor areas in the beta (12-40 Hz) frequency band was dynamically modulated, with maximal coherence occurring about 100 ms before each stimulus. These modulations in beta EEG-EMG motor coherence were found for the 2-Hz audio-visual sequences, confirming at a neurophysiological level the enhancement of motor entrainment with multimodal rhythms that fall within preferred perceptual and movement frequency ranges. Our findings identify beta band cortico-muscular coupling as a potential underlying mechanism of motor entrainment, further elucidating the nature of the link between sensory and motor systems in humans.

Human rhythmic movements spontaneously synchronize with auditory rhythms at various frequency ratios. The emergence of more complex relationships-for instance, frequency ratios of 1:2 and 1:3-is enhanced by adding a congruent accentuation pattern (binary for 1:2 and ternary for 1:3), resulting in a 1:1 movement-accentuation relationship. However, this benefit of accentuation on movement synchronization appears to be stronger for the ternary pattern than for the binary pattern. Here, we investigated whether this difference in accent-induced movement synchronization may be related to a difference in the neural tracking of these accentuation profiles. Accented and control unaccented auditory sequences were presented to participants who concurrently produced finger taps at their preferred frequency, and spontaneous movement synchronization was measured. EEG was recorded during passive listening to each auditory sequence. The results revealed that enhanced movement synchronization with ternary accentuation was accompanied by enhanced neural tracking of this pattern. Larger EEG responses at the accentuation frequency were found for the ternary pattern compared with the binary pattern. Moreover, the amplitude of accent-induced EEG responses was positively correlated with the magnitude of accent-induced movement synchronization across participants. Altogether, these findings show that the dynamics of spontaneous auditory-motor synchronization is strongly driven by the multi-time-scale sensory processing of auditory rhythms, highlighting the importance of considering neural responses to rhythmic sequences for understanding and enhancing synchronization performance.

Humans coordinate their movements with one another in a range of everyday activities and skill domains. Optimal joint performance requires the continuous anticipation of and adaptation to each other's movements, especially when actions are spontaneous rather than pre-planned. Here we employ dual-EEG and frequency-tagging techniques to investigate how the neural tracking of self- and other-generated movements supports interpersonal coordination during improvised motion. LEDs flickering at 5.7 and 7.7 Hz were attached to participants' index fingers in 28 dyads as they produced novel patterns of synchronous horizontal forearm movements. EEG responses at these frequencies revealed enhanced neural tracking of self-generated movement when leading and of other-generated movements when following. A marker of self-other integration at 13.4 Hz (inter-modulation frequency of 5.7 and 7.7 Hz) peaked when no leader was designated, and mutual adaptation and movement synchrony were maximal. Furthermore, the amplitude of EEG responses reflected differences in the capacity of dyads to synchronize their movements, offering a neurophysiologically grounded perspective for understanding perceptual-motor mechanisms underlying joint action.

The combination of frequency-tagging with electroencephalography (EEG) has recently proved fruitful for understanding the perception of beat and meter in musical rhythm, a common behavior shared by humans of all cultures. EEG frequency-tagging allows the objective measurement of input-output transforms to investigate beat perception, its modulation by exogenous and endogenous factors, development, and neural basis. Recent doubt has been raised about the validity of comparing frequency-domain representations of auditory rhythmic stimuli and corresponding EEG responses, assuming that it implies a one-to-one mapping between the envelope of the rhythmic input and the neural output, and that it neglects the sensitivity of frequency-domain representations to acoustic features making up the rhythms. Here we argue that these elements actually reinforce the strengths of the approach. The obvious fact that acoustic features influence the frequency spectrum of the sound envelope precisely justifies taking into consideration the sounds used to generate a beat percept for interpreting neural responses to auditory rhythms. Most importantly, the many-to-one relationship between rhythmic input and perceived beat actually validates an approach that objectively measures the input-output transforms underlying the perceptual categorization of rhythmic inputs. Hence, provided that a number of potential pitfalls and fallacies are avoided, EEG frequency-tagging to study input-output relationships appears valuable for understanding rhythm perception.

The spontaneous ability to entrain to meter periodicities is central to music perception and production across cultures. There is increasing evidence that this ability involves selective neural responses to meter-related frequencies. This phenomenon has been observed in the human auditory cortex, yet it could be the product of evolutionarily older lower-level properties of brainstem auditory neurons, as suggested by recent recordings from rodent midbrain. We addressed this question by taking advantage of a new method to simultaneously record human EEG activity originating from cortical and lower-level sources, in the form of slow (< 20 Hz) and fast (> 150 Hz) responses to auditory rhythms. Cortical responses showed increased amplitudes at meter-related frequencies compared to meter-unrelated frequencies, regardless of the prominence of the meter-related frequencies in the modulation spectrum of the rhythmic inputs. In contrast, frequency-following responses showed increased amplitudes at meter-related frequencies only in rhythms with prominent meter-related frequencies in the input but not for a more complex rhythm requiring more endogenous generation of the meter. This interaction with rhythm complexity suggests that the selective enhancement of meter-related frequencies does not fully rely on subcortical auditory properties, but is critically shaped at the cortical level, possibly through functional connections between the auditory cortex and other, movement-related, brain structures. This process of temporal selection would thus enable endogenous and motor entrainment to emerge with substantial flexibility and invariance with respect to the rhythmic input in humans in contrast with non-human animals.

Music makes us move, and using bass instruments to build the rhythmic foundations of music is especially effective at inducing people to dance to periodic pulse-like beats. Here, we show that this culturally widespread practice may exploit a neurophysiological mechanism whereby low-frequency sounds shape the neural representations of rhythmic input by boosting selective locking to the beat. Cortical activity was captured using electroencephalography (EEG) while participants listened to a regular rhythm or to a relatively complex syncopated rhythm conveyed either by low tones (130 Hz) or high tones (1236.8 Hz). We found that cortical activity at the frequency of the perceived beat is selectively enhanced compared with other frequencies in the EEG spectrum when rhythms are conveyed by bass sounds. This effect is unlikely to arise from early cochlear processes, as revealed by auditory physiological modeling, and was particularly pronounced for the complex rhythm requiring endogenous generation of the beat. The effect is likewise not attributable to differences in perceived loudness between low and high tones, as a control experiment manipulating sound intensity alone did not yield similar results. Finally, the privileged role of bass sounds is contingent on allocation of attentional resources to the temporal properties of the stimulus, as revealed by a further control experiment examining the role of a behavioral task. Together, our results provide a neurobiological basis for the convention of using bass instruments to carry the rhythmic foundations of music and to drive people to move to the beat.

The combination of frequency-tagging with electroencephalography (EEG) has recently proved fruitful for understanding the perception of beat and meter in musical rhythm, a common behavior shared by humans of all cultures. EEG frequency-tagging allows the objective measurement of input-output transforms to investigate beat perception, its modulation by exogenous and endogenous factors, development, and neural basis. Recent doubt has been raised about the validity of comparing frequency-domain representations of auditory rhythmic stimuli and corresponding EEG responses, assuming that it implies a one-to-one mapping between the envelope of the rhythmic input and the neural output, and that it neglects the sensitivity of frequency-domain representations to acoustic features making up the rhythms. Here we argue that these elements actually reinforce the strengths of the approach. The obvious fact that acoustic features influence the frequency spectrum of the sound envelope precisely justifies taking into consideration the sounds used to generate a beat percept for interpreting neural responses to auditory rhythms. Most importantly, the many-to-one relationship between rhythmic input and perceived beat actually validates an approach that objectively measures the input-output transforms underlying the perceptual categorization of rhythmic inputs. Hence, provided that a number of potential pitfalls and fallacies are avoided, EEG frequency-tagging to study input-output relationships appears valuable for understanding rhythm perception.

The current study aims at characterizing the mechanisms that allow humans to entrain the mind and body to incoming rhythmic sensory inputs in real time. We addressed this unresolved issue by examining the relationship between covert neural processes and overt behavior in the context of musical rhythm. We measured temporal prediction abilities, sensorimotor synchronization accuracy and neural entrainment to auditory rhythms as captured using an EEG frequency-tagging approach. Importantly, movement synchronization accuracy with a rhythmic beat could be explained by the amplitude of neural activity selectively locked with the beat period when listening to the rhythmic inputs. Furthermore, stronger endogenous neural entrainment at the beat frequency was associated with superior temporal prediction abilities. Together, these results reveal a direct link between cortical and behavioral measures of rhythmic entrainment, thus providing evidence that frequency-tagged brain activity has functional relevance for beat perception and synchronization.

To successfully navigate through and interact with a dynamic environment it is necessary to acquire and use adequate temporal information to guide behavior. Apart from several areas in the cerebral cortex and cortico-striatal networks, the cerebellum has been proposed to engage in the processing of temporal information. Damage to the cerebellum can impair precise event-based temporal processing in motor and non-motor behavior. To further substantiate cerebellar contributions to temporal processing and to explore its specific role in adapting to a dynamic environment, we investigated sensorimotor temporal processing in ten patients with cerebellar lesions and a corresponding number of healthy matched controls. Experimental tasks included simple self-paced repetitive finger-tapping (spontaneous motor tempo), temporally non-adaptive (isochronous pacing) and adaptive (tempo-changing pacing) sensorimotor synchronization with auditory sequences (synchronization-continuation tapping), and a perceptual tempo judgment. The results indicate that patients' performance diverges systematically from controls on several measures. Cerebellar patients demonstrate more variable self-paced tapping, larger negative asynchronies when synchronizing with isochronous pacing sequences, altered automatic error correction responses to tempo changes (phase correction), and decreased perceptual sensitivity to these perturbations, especially for small accelerations. These findings confirm imprecise temporal processing in cerebellar patients, and hint at a specific impairment in the tenth-of-milliseconds range preceding critical events, in line with a temporally predictive account of cerebellar function. Moreover, this cerebellar profile complements previous findings concerning dysfunctional temporal processing in basal ganglia patients assessed with the same experimental setup, suggesting structural and functional differentiation within an integrative temporal processing network.

To successfully navigate through and interact with a dynamic environment it is necessary to acquire and use adequate temporal information to guide behavior. Apart from several areas in the cerebral cortex and cortico-striatal networks, the cerebellum has been proposed to engage in the processing of temporal information. Damage to the cerebellum can impair precise event-based temporal processing in motor and non-motor behavior. To further substantiate cerebellar contributions to temporal processing and to explore its specific role in adapting to a dynamic environment, we investigated sensorimotor temporal processing in ten patients with cerebellar lesions and a corresponding number of healthy matched controls. Experimental tasks included simple self-paced repetitive finger-tapping (spontaneous motor tempo), temporally non-adaptive (isochronous pacing) and adaptive (tempo-changing pacing) sensorimotor synchronization with auditory sequences (synchronization-continuation tapping), and a perceptual tempo judgment. The results indicate that patients' performance diverges systematically from controls on several measures. Cerebellar patients demonstrate more variable self-paced tapping, larger negative asynchronies when synchronizing with isochronous pacing sequences, altered automatic error correction responses to tempo changes (phase correction), and decreased perceptual sensitivity to these perturbations, especially for small accelerations. These findings confirm imprecise temporal processing in cerebellar patients, and hint at a specific impairment in the tenth-of-milliseconds range preceding critical events, in line with a temporally predictive account of cerebellar function. Moreover, this cerebellar profile complements previous findings concerning dysfunctional temporal processing in basal ganglia patients assessed with the same experimental setup, suggesting structural and functional differentiation within an integrative temporal processing network.

Humans vary substantially in their ability to learn new motor skills. Here, we examined inter-individual differences in learning to play the piano, with the goal of identifying relations to structural properties of white matter fiber tracts relevant to audio-motor learning. Non-musicians (n = 18) learned to perform three short melodies on a piano keyboard in a pure audio-motor training condition (vision of their own fingers was occluded). Initial learning times ranged from 17 to 120 min (mean ± SD: 62 ± 29 min). Diffusion-weighted magnetic resonance imaging was used to derive the fractional anisotropy (FA), an index of white matter microstructural arrangement. A correlation analysis revealed that higher FA values were associated with faster learning of piano melodies. These effects were observed in the bilateral corticospinal tracts, bundles of axons relevant for the execution of voluntary movements, and the right superior longitudinal fasciculus, a tract important for audio-motor transformations. These results suggest that the speed with which novel complex audio-motor skills can be acquired may be determined by variability in structural properties of white matter fiber tracts connecting brain areas functionally relevant for audio-motor learning.

Synchronizing movements with a beat requires rapid compensation for timing errors. The phase-correction response (PCR) has been studied extensively in finger tapping by shifting a metronome onset and measuring the adjustment of the following tap time. How the response unfolds during the subsequent tap cycle remains unknown. Using motion capture, we examined finger kinematics during the PCR. Participants tapped with a metronome containing phase perturbations. They tapped in "legato" and "staccato" style at various tempi, which altered the timing of the constituent movement stages (dwell at the surface, extension, and flexion). After a phase perturbation, tapping kinematics changed compared with baseline, and the PCR was distributed differently across movement stages. In staccato tapping, the PCR trajectory changed primarily during finger extension across tempi. In legato tapping, at fast tempi the PCR occurred primarily during extension, whereas at slow tempi most phase correction was already completed during dwell. Across conditions, timing adjustments occurred primarily 100-250 ms into the following tap cycle. The change in movement around 100 ms represents the time to integrate information into an already planned movement and the rapidity suggests a subcortical route.

Humans vary substantially in their ability to learn new motor skills. Here, we examined inter-individual differences in learning to play the piano, with the goal of identifying relations to structural properties of white matter fiber tracts relevant to audio-motor learning. Non-musicians (n = 18) learned to perform three short melodies on a piano keyboard in a pure audio-motor training condition (vision of their own fingers was occluded). Initial learning times ranged from 17 to 120 min (mean ± SD: 62 ± 29 min). Diffusion-weighted magnetic resonance imaging was used to derive the fractional anisotropy (FA), an index of white matter microstructural arrangement. A correlation analysis revealed that higher FA values were associated with faster learning of piano melodies. These effects were observed in the bilateral corticospinal tracts, bundles of axons relevant for the execution of voluntary movements, and the right superior longitudinal fasciculus, a tract important for audio-motor transformations. These results suggest that the speed with which novel complex audio-motor skills can be acquired may be determined by variability in structural properties of white matter fiber tracts connecting brain areas functionally relevant for audio-motor learning.