Search

Stuttering is a neurodevelopmental disorder characterized by impaired production of coordinated articulatory movements needed for fluent speech. It is currently unknown whether these abnormal production characteristics reflect disruptions to brain mechanisms underlying the acquisition and/or execution of speech motor sequences. To dissociate learning and control processes, we used a motor sequence learning paradigm to examine the behavioral and neural correlates of learning to produce novel phoneme sequences in adults who stutter (AWS) and neurotypical controls. Participants intensively practiced producing pseudowords containing non-native consonant clusters (e.g., “GVAZF”) over two days. The behavioral results indicated that although the two experimental groups showed comparable learning trajectories, AWS performed significantly worse on the task prior to and after speech motor practice. Using functional magnetic resonance imaging (fMRI), the authors compared brain activity during articulation of the practiced words and a set of novel pseudowords (matched in phonetic complexity). FMRI analyses revealed no differences between AWS and controls in cortical or subcortical regions; both groups showed comparable increases in activation in left-lateralized brain areas implicated in phonological working memory and speech motor planning during production of the novel sequences compared to the practiced sequences. Moreover, activation in left-lateralized basal ganglia sites was negatively correlated with in-scanner mean disfluency in AWS. Collectively, these findings demonstrate that AWS exhibit no deficit in constructing new speech motor sequences but do show impaired execution of these sequences before and after they have been acquired and consolidated.

Effective human supervision of robots can be key for ensuring correct robot operation in a variety of potentially safety-critical scenarios. This paper takes a step towards fast and reliable human intervention in supervisory control tasks by combining two streams of human biosignals: muscle and brain activity acquired via EMG and EEG, respectively. It presents continuous classification of left and right hand-gestures using muscle signals, time-locked classification of error-related potentials using brain signals (unconsciously produced when observing an error), and a framework that combines these pipelines to detect and correct robot mistakes during multiple-choice tasks. The resulting hybrid system is evaluated in a “plug-and-play” fashion with 7 untrained subjects supervising an autonomous robot performing a target selection task. Offline analysis further explores the EMG classification performance, and investigates methods to select subsets of training data that may facilitate generalizable plug-and-play classifiers.

BACKGROUND: The Directions Into Velocities of Articulators (DIVA) model and its partner, the Gradient Order DIVA (GODIVA) model, provide neurobiologically grounded, computational accounts of speech motor control and motor sequencing, with applications for the study and treatment of neurological motor speech disorders. AIMS: In this review, we provide an overview of the DIVA and GODIVA models and how they explain the interface between phonological and motor planning systems to build on previous models and provide a mechanistic accounting of apraxia of speech (AOS), a disorder of speech motor programming. MAIN CONTRIBUTION: Combined, the DIVA and GODIVA models account for both the segmental and suprasegmental features that define AOS via damage to (i) a speech sound map, hypothesized to reside in left ventral premotor cortex, (ii) a phonological content buffer hypothesized to reside in left posterior inferior frontal sulcus, and/or (iii) the axonal projections between these regions. This account is in line with a large body of behavioural work, and it unifies several prior theoretical accounts of AOS. CONCLUSIONS: The DIVA and GODIVA models provide an integrated framework for the generation and testing of both behavioural and neuroimaging hypotheses about the underlying neural mechanisms responsible for motor programming in typical speakers and in speakers with AOS.

Purpose Adductor spasmodic dysphonia (ADSD), the most common form of spasmodic dysphonia, is a debilitating voice disorder characterized by hyperactivity and muscle spasms in the vocal folds during speech. Prior neuroimaging studies have noted excessive brain activity during speech in participants with ADSD compared to controls. Speech involves an auditory feedback control mechanism that generates motor commands aimed at eliminating disparities between desired and actual auditory signals. Thus, excessive neural activity in ADSD during speech may reflect, at least in part, increased engagement of the auditory feedback control mechanism as it attempts to correct vocal production errors detected through audition. Method To test this possibility, functional magnetic resonance imaging was used to identify differences between participants with ADSD ( n = 12) and age-matched controls ( n = 12) in (a) brain activity when producing speech under different auditory feedback conditions and (b) resting-state functional connectivity within the cortical network responsible for vocalization. Results As seen in prior studies, the ADSD group had significantly higher activity than the control group during speech with normal auditory feedback (compared to a silent baseline task) in three left-hemisphere cortical regions: ventral Rolandic (sensorimotor) cortex, anterior planum temporale, and posterior superior temporal gyrus/planum temporale. Importantly, this same pattern of hyperactivity was also found when auditory feedback control of speech was eliminated through masking noise. Furthermore, the ADSD group had significantly higher resting-state functional connectivity between sensorimotor and auditory cortical regions within the left hemisphere as well as between the left and right hemispheres. Conclusions Together, our results indicate that hyperactivation in the cortical speech network of individuals with ADSD does not result from hyperactive auditory feedback control mechanisms and rather is likely related to impairments in somatosensory feedback control and/or feedforward control mechanisms.

Vocal communication is a critical feature of social interaction across species; however, the relation between such behavior in humans and nonhumans remains unclear. To enable comparative investigation of this topic, we review the literature pertinent to interactive language use and identify the superset of cognitive operations involved in generating communicative action. We posit these functions comprise three intersecting multistep pathways: ( a) the Content Pathway, which selects the movements constituting a response; ( b) the Timing Pathway, which temporally structures responses; and ( c) the Affect Pathway, which modulates response parameters according to internal state. These processing streams form the basis of the Convergent Pathways for Interaction framework, which provides a conceptual model for investigating the cognitive and neural computations underlying vocal communication across species.

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

Purpose This study aimed to test whether (and how) somatosensory feedback signals from the vocal tract affect concurrent unimodal visual speech perception. Method Participants discriminated pairs of silent visual utterances of vowels under 3 experimental conditions: (a) normal (baseline) and while holding either (b) a bite block or (c) a lip tube in their mouths. To test the specificity of somatosensory–visual interactions during perception, we assessed discrimination of vowel contrasts optically distinguished based on their mandibular (English /ɛ/–/æ/) or labial (English /u/–French /u/) postures. In addition, we assessed perception of each contrast using dynamically articulating videos and static (single-frame) images of each gesture (at vowel midpoint). Results Engaging the jaw selectively facilitated perception of the dynamic gestures optically distinct in terms of jaw height, whereas engaging the lips selectively facilitated perception of the dynamic gestures optically distinct in terms of their degree of lip compression and protrusion. Thus, participants perceived visible speech movements in relation to the configuration and shape of their own vocal tract (and possibly their ability to produce covert vowel production–like movements). In contrast, engaging the articulators had no effect when the speaking faces did not move, suggesting that the somatosensory inputs affected perception of time-varying kinematic information rather than changes in target (movement end point) mouth shapes. Conclusions These findings suggest that orofacial somatosensory inputs associated with speech production prime premotor and somatosensory brain regions involved in the sensorimotor control of speech, thereby facilitating perception of concordant visible speech movements. Supplemental Material https://doi.org/10.23641/asha.9911846

Neuroimaging has revealed a core network of cortical regions that contribute to speech production, but the functional organization of this network remains poorly understood. Purpose We describe efforts to identify reliable boundaries around functionally homogenous regions within the cortical speech motor control network in order to improve the sensitivity of functional magnetic resonance imaging (fMRI) analyses of speech production and thus improve our understanding of the functional organization of speech production in the brain. Method We used a bottom-up, data-driven approach by pooling data from 12 previously conducted fMRI studies of speech production involving the production of monosyllabic and bisyllabic words and pseudowords that ranged from single vowels and consonant–vowel pairs to short sentences (163 scanning sessions, 136 unique participants, 39 different speech conditions). After preprocessing all data through the same pipeline and registering individual contrast maps to a common surface space, hierarchical clustering was applied to contrast maps randomly sampled from the pooled data set in order to identify consistent functional boundaries across subjects and tasks. Boundary completion was achieved by applying adaptive smoothing and watershed segmentation to the thresholded population-level boundary map. Hierarchical clustering was applied to the mean within–functional region of interest (fROI) response to identify networks of fROIs that respond similarly during speech. Results We identified highly reliable functional boundaries across the cortical areas involved in speech production. Boundary completion resulted in 117 fROIs in the left hemisphere and 109 in the right hemisphere. Clustering of the mean within-fROI response revealed a core sensorimotor network flanked by a speech motor planning network. The majority of the left inferior frontal gyrus clustered with the visual word form area and brain regions (e.g., anterior insula, dorsal anterior cingulate) associated with detecting salient sensory inputs and choosing the appropriate action. Conclusion The fROIs provide insight into the organization of the speech production network and a valuable tool for studying speech production in the brain by improving within-group and between-groups comparisons of speech-related brain activity. Supplemental Material https://doi.org/10.23641/asha.9402674

PurposeUnderstanding what limits speech development in minimally verbal (MV) children with autism spectrum disorder (ASD) is important for providing highly effective targeted therapies. This preliminary investigation explores the extent to which developmental speech deficits predicted by Directions Into Velocities of Articulators (DIVA), a computational model of speech production, exemplify real phenotypes.MethodImplementing a motor speech disorder in DIVA predicted that speech would become highly variable within and between tokens, while implementing a motor speech plus an auditory processing disorder predicted that DIVA's speech would become highly centralized (schwa-like). Acoustic analyses of DIVA's output predicted that acoustically measured phoneme distortion would be similar between the two cases, but that in the former case, speech would show more within- and between-token variability than in the latter case. We tested these predictions quantitatively on the speech of children with MV ASD. In Study 1, we tested the qualitative predictions using perceptual analysis methods. Speech pathologists blinded to the purpose of the study tallied the signs of childhood apraxia of speech that appeared in the speech of 38 MV children with ASD. K-means clustering was used to create two clusters from the group of 38, and analysis of variance was used to determine whether the clusters differed according to perceptual features corresponding to within- and between-token variability. In Study 2, we employed acoustic analyses on the speech of the child from each cluster who produced the largest number of analyzable tokens to test the predictions of differences in within-token variability, between-token variability, and vowel space area.ResultsClusters produced by k-means analysis differed by perceptual features that corresponded to within-token variability. Nonsignificant differences between clusters were found for features corresponding to between-token variability. Subsequent acoustic analyses of the selected cases revealed that the speech of the child from the high-variability cluster showed significantly more quantitative within- and between-token variability than the speech of the child from the low-variability cluster. The vowel space of the child from the low-variability cluster was more centralized than that of typical children and that of the child from the high-variability cluster.ConclusionsResults provide preliminary evidence that subphenotypes of children with MV ASD may exist, characterized by (a) comorbid motor speech disorder and (b) comorbid motor speech plus auditory processing disorder. The results motivate testable predictions about how these comorbidities affect speech.Supplemental Materialhttps://doi.org/10.23641/asha.14384432

BACKGROUND: Communication difficulties are a core deficit in many people with autism spectrum disorder (ASD). The current study evaluated neural activation in participants with ASD and neurotypical (NT) controls during a speech production task. METHODS: Neural activities of participants with ASD (N = 15, M = 16.7 years, language abilities ranged from low verbal abilities to verbally fluent) and NT controls (N = 12, M = 17.1 years) was examined using functional magnetic resonance imaging with a sparse-sampling paradigm. RESULTS: There were no differences between the ASD and NT groups in average speech activation or inter-subject run-to-run variability in speech activation. Intra-subject run-to-run neural variability was greater in the ASD group and was positively correlated with autism severity in cortical areas associated with speech. CONCLUSIONS: These findings highlight the importance of understanding intra-subject neural variability in participants with ASD.

In this cross-sectional study, we examined the relationship between cortical thickness and performance on several verbal repetition tasks in a cohort of patients with primary progressive aphasia in order to test predictions generated by theoretical accounts of phonological working memory that predict phonological content buffers in left posterior inferior frontal sulcus and supramarginal gyrus. Cortical surfaces were reconstructed from magnetic resonance imaging scans from 42 participants diagnosed with primary progressive aphasia. Cortical thickness was measured in a set of anatomical regions spanning the entire cerebral cortex. Correlation analyses were performed between cortical thickness and average score across three phonological working memory-related tasks: the Repetition sub-test from the Western Aphasia Battery, a forward digit span task, and a backward digit span task. Significant correlations were found between average working memory score across tasks and cortical thickness in left supramarginal gyrus and left posterior inferior frontal sulcus, in support of prior theoretical accounts of phonological working memory. Exploratory whole-brain correlation analyses performed for each of the three behavioural tasks individually revealed a distinct set of positively correlated regions for each task. Comparison of cortical thickness measures from different primary progressive aphasia sub-types to cortical thickness in age-matched controls further revealed unique patterns of atrophy in the different subtypes., Graphical Abstract Graphical Abstract

PurposeStuttering is characterized by intermittent speech disfluencies, which are dramatically reduced when speakers synchronize their speech with a steady beat. The goal of this study was to characterize the neural underpinnings of this phenomenon using functional magnetic resonance imaging.MethodData were collected from 16 adults who stutter and 17 adults who do not stutter while they read sentences aloud either in a normal, self-paced fashion or paced by the beat of a series of isochronous tones (“rhythmic”). Task activation and task-based functional connectivity analyses were carried out to compare neural responses between speaking conditions and groups after controlling for speaking rate.ResultsAdults who stutter produced fewer disfluent trials in the rhythmic condition than in the normal condition. Adults who stutter did not have any significant changes in activation between the rhythmic condition and the normal condition, but when groups were collapsed, participants had greater activation in the rhythmic condition in regions associated with speech sequencing, sensory feedback control, and timing perception. Adults who stutter also demonstrated increased functional connectivity among cerebellar regions during rhythmic speech as compared to normal speech and decreased connectivity between the left inferior cerebellum and the left prefrontal cortex.ConclusionsModulation of connectivity in the cerebellum and prefrontal cortex during rhythmic speech suggests that this fluency-inducing technique activates a compensatory timing system in the cerebellum and potentially modulates top-down motor control and attentional systems. These findings corroborate previous work associating the cerebellum with fluency in adults who stutter and indicate that the cerebellum may be targeted to enhance future therapeutic interventions.Supplemental Materialhttps://doi.org/10.23641/asha.14417681

Purpose To better define the contributions of somatosensory and auditory feedback in vocal motor control, a laryngeal perturbation experiment was conducted with and without masking of auditory feedback. Method Eighteen native speakers of English produced a sustained vowel while their larynx was physically and externally displaced on a subset of trials. For the condition with auditory masking, speech-shaped noise was played via earphones at 90 dB SPL. Responses to the laryngeal perturbation were compared to responses by the same participants to an auditory perturbation experiment that involved a 100-cent downward shift in fundamental frequency ( f o ). Responses were also examined in relation to a measure of auditory acuity. Results Compensatory responses to the laryngeal perturbation were observed with and without auditory masking. The level of compensation was greatest in the laryngeal perturbation condition without auditory masking, followed by the condition with auditory masking; the level of compensation was smallest in the auditory perturbation experiment. No relationship was found between the degree of compensation to auditory versus laryngeal perturbations, and the variation in responses in both perturbation experiments was not related to auditory acuity. Conclusions The findings indicate that somatosensory and auditory feedback control mechanisms work together to compensate for laryngeal perturbations, resulting in the greatest degree of compensation when both sources of feedback are available. In contrast, these two control mechanisms work in competition in response to auditory perturbations, resulting in an overall smaller degree of compensation. Supplemental Material https://doi.org/10.23641/asha.12559628

Traditional group fMRI (functional magnetic resonance imaging) analyses are not designed to detect individual differences that may be crucial to better understanding speech disorders. Single-subject research could therefore provide a richer characterization of the neural substrates of speech production in development and disease. Before this line of research can be tackled, however, it is necessary to evaluate whether healthy individuals exhibit reproducible brain activation across multiple sessions during speech production tasks. In the present study, we evaluated the reliability and discriminability of cortical functional magnetic resonance imaging data from twenty neuro-typical subjects who participated in two experiments involving reading aloud mono- or bisyllabic speech stimuli. Using traditional methods like the Dice and intraclass correlation coefficients, we found that most individuals displayed moderate to high reliability, with exceptions likely due to increased head motion in the scanner. Further, this level of reliability for speech production was not directly correlated with reliable patterns in the underlying average blood oxygenation level dependent signal across the brain. Finally, we found that a novel machine-learning subject classifier could identify these individuals by their speech activation patterns with 97% accuracy from among a dataset of seventy-five subjects. These results suggest that single-subject speech research would yield valid results and that investigations into the reliability of speech activation in people with speech disorders are warranted.

Purpose: This study investigated the nature of phonological working memory (PWM) structures and speech motor programming units by examining how performance gains from practicing non-native phoneme sequences generalize to novel sequences that overlap to varying degrees with the practiced sequences.Method: CCVCC words were constructed using consonant clusters that violated English phonological constraints, thus making them difficult to produce initially. After practicing a subset of the words over two consecutive days, participants were tested on the production of several types of word pairs, including novel words containing unpracticed clusters, practiced cluster words containing consonant clusters that were practiced but in different words from the test words, and fully learned words that were practiced in their entirety.Results: Utterance duration improvements from practicing clusters in one syllabic context fully transferred to novel words that included these clusters, while error rate improvements from practicing clusters in one syllabic context only partially generalized to new syllables utilizing these practiced clusters. Additionally, error rates for the first word in the pair (which depend primarily on motor program structure) showed partial improvements for learned clusters regardless of whether the cluster was practiced in the same part of the syllabic frame (onset or coda), whereas error rates for the second word (which reflect both PWM and motor programming mechanisms) were higher than even novel words if the cluster was practiced in the wrong syllable frame location, presumably due to interference effects in PWM. Conclusions: These results provide support for an onset-nucleus-coda syllabic frame structure in PWM and a syllable-frame-independent representation of common phoneme sub-sequences for motor programs.

Speech production is a highly complex sensorimotor task involving tightly coordinated processing across large expanses of the cerebral cortex. Historically, the study of the neural underpinnings of speech suffered from the lack of an animal model. The development of non-invasive structural and functional neuroimaging techniques in the late 20(th) century has dramatically improved our understanding of the speech network. Techniques for measuring regional cerebral blood flow have illuminated the neural regions involved in various aspects of speech, including feedforward and feedback control mechanisms. In parallel, we have designed, experimentally tested, and refined a neural network model detailing the neural computations performed by specific neuroanatomical regions during speech. Computer simulations of the model account for a wide range of experimental findings, including data on articulatory kinematics and brain activity during normal and perturbed speech. Furthermore, the model is being used to investigate a wide range of communication disorders.

Two auditory perturbation experiments were used to investigate the integrity of neural circuits responsible for speech sensorimotor adaptation in acquired apraxia of speech (AOS). This has implications for understanding the nature of AOS as well as normal speech motor control. Two experiments were conducted. In Experiment 1, compensatory responses to unpredictable fundamental frequency (F0) perturbations during vocalization were investigated in healthy older adults and adults with acquired AOS plus aphasia. F0 perturbation involved upward and downward 100-cent shifts versus no shift, in equal proportion, during 2 s vocalizations of the vowel /a/. In Experiment 2, adaptive responses to sustained first formant (F1) perturbations during speech were investigated in healthy older adults, adults with AOS and adults with aphasia only (APH). The F1 protocol involved production of the vowel /ε/ in four consonant-vowel words of Australian English (pear, bear, care, dare), and one control word with a different vowel (paw). An unperturbed Baseline phase was followed by a gradual Ramp to a 30% upward F1 shift stimulating a compensatory response, a Hold phase where the perturbation was repeatedly presented with alternating blocks of masking trials to probe adaptation, and an End phase with masking trials only to measure persistence of any adaptation. AOS participants showed normal compensation to unexpected F0 perturbations, indicating that auditory feedback control of low-level, non-segmental parameters is intact. Furthermore, individuals with AOS displayed an adaptive response to sustained F1 perturbations, but age-matched controls and APH participants did not. These findings suggest that older healthy adults may have less plastic motor programs that resist modification based on sensory feedback, whereas individuals with AOS have less well-established and more malleable motor programs due to damage from stroke.

Stuttering is a neurodevelopmental disorder that affects the smooth flow of speech production. Stuttering onset occurs during a dynamic period of development when children first start learning to formulate sentences. Although most children grow out of stuttering naturally, ∼1% of all children develop persistent stuttering that can lead to significant psychosocial consequences throughout one’s life. To date, few studies have examined neural bases of stuttering in children who stutter, and even fewer have examined the basis for natural recovery versus persistence of stuttering. Here we report the first study to conduct surface-based analysis of the brain morphometric measures in children who stutter. We used FreeSurfer to extract cortical size and shape measures from structural MRI scans collected from the initial year of a longitudinal study involving 70 children (36 stuttering, 34 controls) in the 3–10-year range. The stuttering group was further divided into two groups: persistent and recovered, based on their later longitudinal visits that allowed determination of their eventual clinical outcome. A region of interest analysis that focused on the left hemisphere speech network and a whole-brain exploratory analysis were conducted to examine group differences and group × age interaction effects. We found that the persistent group could be differentiated from the control and recovered groups by reduced cortical thickness in left motor and lateral premotor cortical regions. The recovered group showed an age-related decrease in local gyrification in the left medial premotor cortex (supplementary motor area and and pre-supplementary motor area). These results provide strong evidence of a primary deficit in the left hemisphere speech network, specifically involving lateral premotor cortex and primary motor cortex, in persistent developmental stuttering. Results further point to a possible compensatory mechanism involving left medial premotor cortex in those who recover from childhood stuttering.

Control of robots in safety-critical tasks and situations where costly errors may occur is paramount for realizing the vision of pervasive human-robot collaborations. For these cases, the ability to use human cognition in the loop can be key for recuperating safe robot operation. This paper combines two streams of human biosignals, electrical muscle and brain activity via EMG and EEG, respectively, to achieve fast and accurate human intervention in a supervisory control task. In particular, this paper presents an end-to-end system for continuous rolling-window classification of gestures that allows the human to actively correct the robot on demand, discrete classification of Error-Related Potential signals (unconsciously produced by the human supervisor’s brain when observing a robot error), and a framework that integrates these two classification streams for fast and effective human intervention. The system also allows “plug-and-play” operation, demonstrating accurate performance even with new users whose biosignals have not been used for training the classifiers. The resulting hybrid control system for safety-critical situations is evaluated with 7 untrained human subjects in a supervisory control scenario where an autonomous robot performs a multi-target selection task., Boeing Company

OBJECTIVE:This study examined adaptive responses to auditory perturbation of fundamental frequency (fo) in speakers with Parkinson's disease (PD) and control speakers. METHOD:Sixteen speakers with PD and nineteen control speakers produced sustained vowels while they received perturbed auditory feedback (i.e., fo shifted upward or downward). Speakers' pitch acuity was quantified using a just-noticeable-difference (JND) paradigm. Twelve listeners provided estimates of the speech intelligibility for speakers with PD. RESULTS:Fifteen responses from each speaker group for each shift direction were included in analyses. While control speakers generally showed consistent adaptive responses opposing the perturbation, speakers with PD showed no compensation on average, with individual PD speakers showing highly variable responses. In the PD group, the degree of compensation was not significantly correlated with age, disease progression, pitch acuity, or intelligibility. CONCLUSIONS:These findings indicate reduced adaptation to sustained fo perturbation and higher variability in PD compared to control participants. No significant differences were seen in pitch acuity between groups, suggesting that the fo adaptation deficit in PD is not the result of purely perceptual mechanisms. SIGNIFICANCE:These results suggest there is an impairment in vocal motor control in PD. Building on these results, contributions can be made to developing targeted voice treatments for PD.

© 2017 IEEE. Communication with a robot using brain activity from a human collaborator could provide a direct and fast feedback loop that is easy and natural for the human, thereby enabling a wide variety of intuitive interaction tasks. This paper explores the application of EEG-measured error-related potentials (ErrPs) to closed-loop robotic control. ErrP signals are particularly useful for robotics tasks because they are naturally occurring within the brain in response to an unexpected error. We decode ErrP signals from a human operator in real time to control a Rethink Robotics Baxter robot during a binary object selection task. We also show that utilizing a secondary interactive error-related potential signal generated during this closed-loop robot task can greatly improve classification performance, suggesting new ways in which robots can acquire human feedback. The design and implementation of the complete system is described, and results are presented for realtime closed-loop and open-loop experiments as well as offline analysis of both primary and secondary ErrP signals. These experiments are performed using general population subjects that have not been trained or screened. This work thereby demonstrates the potential for EEG-based feedback methods to facilitate seamless robotic control, and moves closer towards the goal of real-time intuitive interaction.

Auditory feedback (AF), the speech signal received by a speaker’s own auditory system, contributes to the online control of speech movements. Recent studies based on AF perturbation provided evidence for abnormalities in the integration of auditory error with ongoing articulation and phonation in persons who stutter (PWS), but stopped short of examining connected speech. This is a crucial limitation considering the importance of sequencing and timing in stuttering. In the current study, we imposed time-varying perturbations on AF while PWS and fluent participants uttered a multisyllabic sentence. Two distinct types of perturbations were used to separately probe the control of the spatial and temporal parameters of articulation. While PWS exhibited only subtle anomalies in the AF-based spatial control, their AF-based fine-tuning of articulatory timing was substantially weaker than normal, especially in early parts of the responses, indicating slowness in the auditory–motor integration for temporal control.

BACKGROUND/PURPOSE: Differentiating the symptom complex due to phonological-level disorders, speech delay and pediatric motor speech disorders is a controversial issue in the field of pediatric speech and language pathology. The present study investigated the developmental interaction between neurological deficits in auditory and motor processes using computational modeling with the DIVA model. METHOD: In a series of computer simulations, we investigated the effect of a motor processing deficit alone (MPD), and the effect of a motor processing deficit in combination with an auditory processing deficit (MPD+APD) on the trajectory and endpoint of speech motor development in the DIVA model. RESULTS: Simulation results showed that a motor programming deficit predominantly leads to deterioration on the phonological level (phonemic mappings) when auditory self-monitoring is intact, and on the systemic level (systemic mapping) if auditory self-monitoring is impaired. CONCLUSIONS: These findings suggest a close relation between quality of auditory self-monitoring and the involvement of phonological vs. motor processes in children with pediatric motor speech disorders. It is suggested that MPD+APD might be involved in typically apraxic speech output disorders and MPD in pediatric motor speech disorders that also have a phonological component. Possibilities to verify these hypotheses using empirical data collected from human subjects are discussed. Learning outcomes: The reader will be able to: (1) identify the difficulties in studying disordered speech motor development; (2) describe the differences in speech motor characteristics between SSD and subtype CAS; (3) describe the different types of learning that occur in the sensory–motor system during babbling and early speech acquisition; (4) identify the neural control subsystems involved in speech production; (5) describe the potential role of auditory self-monitoring in developmental speech disorders. (PsycINFO Database Record (c) 2014 APA, all rights reserved) (journal abstract)

A typical white-matter integrity and elevated dopamine levels have been reported for individuals who stutter. We investigated how such abnormalities may lead to speech dysfluencies due to their effects on a syllable-sequencing circuit that consists of basal ganglia (BG), thalamus, and left ventral premotor cortex (vPMC). “Neurally impaired” versions of the neurocomputational speech production model GODIVA were utilized to test two hypotheses: (1) that white-matter abnormalities disturb the circuit via corticostriatal projections carrying copies of executed motor commands, and (2) that dopaminergic abnormalities disturb the circuit via the striatum. Simulation results support both hypotheses: in both scenarios, the neural abnormalities delay readout of the next syllable’s motor program, leading to dysfluency. The results also account for brain imaging findings during dysfluent speech. It is concluded that each of the two abnormality types can cause stuttering moments, probably by affecting the same BG-thalamus-vPMC circuit.

Numerous studies have examined the brain bases of autism; few, however, have specifically examined the neurobiology of speech and language impairments in children and adults on the spectrum, especially those characterized as low functioning or minimally verbal, due to compliance issues. With exciting new advances in the development of paradigms and tools, and the ability to image children at risk for autism as young as 6 months of age, functional neuroimaging (EEG, magnetoencephalography and functional MRI) holds tremendous promise. Findings of reduced activation and structural and functional connectivity in the language network, together with deficits in social reciprocity and motivation, and a preference for visual over verbal information, appear to be carving out a neurobiological profile for the impaired social–communication brain in autism.

Technology that harnesses brain signals releases “locked-in” patients—too paralyzed to speak— from their prison of silence.

Impairments in language and communication are core features of Autism Spectrum Disorder (ASD), and a substantial percentage of children with ASD do not develop speech. ASD is often characterized as a disorder of brain connectivity, and a number of studies have identified white matter impairments in affected individuals. The current study investigated white matter integrity in the speech network of high-functioning adults with ASD. Diffusion tensor imaging (DTI) scans were collected from 18 participants with ASD and 18 neurotypical participants. Probabilistic tractography was used to estimate the connection strength between ventral premotor cortex (vPMC), a cortical region responsible for speech motor planning, and five other cortical regions in the network of areas involved in speech production. We found a weaker connection between the left vPMC and the supplementary motor area in the ASD group. This pathway has been hypothesized to underlie the initiation of speech motor programs. Our results indicate that a key pathway in the speech production network is impaired in ASD, and that this impairment can occur even in the presence of normal language abilities. Therapies that result in normalization of this pathway may hold particular promise for improving speech output in ASD., Highlights • We used diffusion tensor imaging to measure white matter (WM) tracts in autism. • Autistic participants were high-functioning individuals with normal language skills. • WM between left supplementary motor and premotor areas is impaired in autism. • This tract is believed to be involved in the initiation of speech articulation. • Speech production may be impaired in the absence of language deficits in autism.

Many individuals with minimal movement capabilities use AAC to communicate. These individuals require both an interface with which to construct a message (e.g., a grid of letters) and an input modality with which to select targets. This study evaluated the interaction of two such systems: (a) an input modality using surface electromyography (sEMG) of spared facial musculature, and (b) an onscreen interface from which users select phonemic targets. These systems were evaluated in two experiments: (a) participants without motor impairments used the systems during a series of eight training sessions, and (b) one individual who uses AAC used the systems for two sessions. Both the phonemic interface and the electromyographic cursor show promise for future AAC applications.

The DIVA model of speech production provides a computationally and neuroanatomically explicit account of the network of brain regions involved in speech acquisition and production. An overview of the model is provided along with descriptions of the computations performed in the different brain regions represented in the model. The latest version of the model, which contains a new right-lateralized feedback control map in ventral premotor cortex, will be described, and experimental results that motivated this new model component will be discussed. Application of the model to the study and treatment of communication disorders will also be briefly described.

The relation between auditory acuity, somatosensory acuity and the magnitude of produced sibilant contrast was investigated with data from 18 participants. To measure auditory acuity, stimuli from a synthetic sibilant continuum ([s]-[ʃ]) were used in a four-interval, two-alternative forced choice adaptive-staircase discrimination task. To measure somatosensory acuity, small plastic domes with grooves of different spacing were pressed against each participant's tongue tip and the participant was asked to identify one of four possible orientations of the grooves. Sibilant contrast magnitudes were estimated from productions of the words 'said,' 'shed,' 'sid,' and 'shid'. Multiple linear regression revealed a significant relation indicating that a combination of somatosensory and auditory acuity measures predicts produced acoustic contrast. When the participants were divided into high- and low-acuity groups based on their median somatosensory and auditory acuity measures, separate ANOVA analyses with sibilant contrast as the dependent variable yielded a significant main effect for each acuity group. These results provide evidence that sibilant productions have auditory as well as somatosensory goals and are consistent with prior results and the theoretical framework underlying the DIVA model of speech production.

In order to test whether auditory feedback is involved in the planning of complex articulatory gestures in time-varying phonemes, the current study examined native Mandarin speakers' responses to auditory perturbations of their auditory feedback of the trajectory of the first formant frequency during their production of the triphthong ∕iau∕. On average, subjects adaptively adjusted their productions to partially compensate for the perturbations in auditory feedback. This result indicates that auditory feedback control of speech movements is not restricted to quasi-static gestures in monophthongs as found in previous studies, but also extends to time-varying gestures. To probe the internal structure of the mechanisms of auditory-motor transformations, the pattern of generalization of the adaptation learned on the triphthong ∕iau∕ to other vowels with different temporal and spatial characteristics (produced only under masking noise) was tested. A broad but weak pattern of generalization was observed; the strength of the generalization diminished with increasing dissimilarity from ∕iau∕. The details and implications of the pattern of generalization are examined and discussed in light of previous sensorimotor adaptation studies of both speech and limb motor control and a neurocomputational model of speech motor control.

Speech production demands a number of integrated processing stages. The system must encode the speech motor programs that command movement trajectories of the articulators and monitor transient spatiotemporal variations in auditory and somatosensory feedback. Early models of this system proposed that independent neural regions perform specialized speech processes. As technology advanced, neuroimaging data revealed that the dynamic sensorimotor processes of speech require a distributed set of interacting neural regions. The DIVA (Directions into Velocities of Articulators) neurocomputational model elaborates on early theories, integrating existing data and contemporary ideologies, to provide a mechanistic account of acoustic, kinematic, and functional magnetic resonance imaging (fMRI) data on speech acquisition and production. This large-scale neural network model is composed of several interconnected components whose cell activities and synaptic weight strengths are governed by differential equations. Cells in the model are associated with neuroanatomical substrates and have been mapped to locations in Montreal Neurological Institute stereotactic space, providing a means to compare simulated and empirical fMRI data. The DIVA model also provides a computational and neurophysiological framework within which to interpret and organize research on speech acquisition and production in fluent and dysfluent child and adult speakers. The purpose of this review article is to demonstrate how the DIVA model is used to motivate and guide functional imaging studies. We describe how model predictions are evaluated using voxel-based, region-of-interest-based parametric analyses and inter-regional effective connectivity modeling of fMRI data.

This paper briefly reviews current silent speech methodologies for normal and disabled individuals. Current techniques utilizing electromyographic (EMG) recordings of vocal tract movements are useful for physically healthy individuals but fail for tetraplegic individuals who do not have accurate voluntary control over the speech articulators. Alternative methods utilizing EMG from other body parts (e.g., hand, arm, or facial muscles) or electroencephalography (EEG) can provide capable silent communication to severely paralyzed users, though current interfaces are extremely slow relative to normal conversation rates and require constant attention to a computer screen that provides visual feedback and/or cueing. We present a novel approach to the problem of silent speech via an intracortical microelectrode brain computer interface (BCI) to predict intended speech information directly from the activity of neurons involved in speech production. The predicted speech is synthesized and acoustically fed back to the user with a delay under 50 ms. We demonstrate that the Neurotrophic Electrode used in the BCI is capable of providing useful neural recordings for over 4 years, a necessary property for BCIs that need to remain viable over the lifespan of the user. Other design considerations include neural decoding techniques based on previous research involving BCIs for computer cursor or robotic arm control via prediction of intended movement kinematics from motor cortical signals in monkeys and humans. Initial results from a study of continuous speech production with instantaneous acoustic feedback show the BCI user was able to improve his control over an artificial speech synthesizer both within and across recording sessions. The success of this initial trial validates the potential of the intracortical microelectrode-based approach for providing a speech prosthesis that can allow much more rapid communication rates.

Purpose The authors examined the involvement of 2 speech motor programming processes identified by S. T. Klapp (1995, 2003) during the articulation of utterances differing in syllable and sequence complexity. According to S. T. Klapp, 1 process, INT, resolves the demands of the programmed unit, whereas a second process, SEQ, oversees the serial order demands of longer sequences. Method A modified reaction time paradigm was used to assess INT and SEQ demands. Specifically, syllable complexity was dependent on syllable structure, whereas sequence complexity involved either repeated or unique syllabi within an utterance. Results INT execution was slowed when articulating single syllables in the form CCCV compared to simpler CV syllables. Planning unique syllables within a multisyllabic utterance rather than repetitions of the same syllable slowed INT but not SEQ. Conclusions The INT speech motor programming process, important for mental syllabary access, is sensitive to changes in both syllable structure and the number of unique syllables in an utterance.

Tongue shape can vary greatly for allophones of /r/ produced in different phonetic contexts but the primary acoustic cue used by listeners, lowered F3, remains stable. For the current study, it was hypothesized that auditory feedback maintains the speech motor control mechanisms that are constraining acoustic variability of F3 in /r/; thus the listener’s percept remains /r/ despite the range of articulatory configurations employed by the speaker. Given the potential importance of auditory feedback, postlingually deafened speakers should show larger acoustic variation in /r/ allophones than hearing controls, and auditory feedback from a cochlear implant could reduce that variation over time. To test these hypotheses, measures were made of phoneme perception and of production of tokens containing /r/, stop consonants, and /r/+stop clusters in hearing controls and in eight postlingually deafened adults pre- and postimplant. Postimplant, seven of the eight implant speakers did not differ from the control mean. It was also found that implant users’ production of stop and stop+/r/ blend improved with time but the measured acoustic contrast between these was still better in the control speakers than for the implant group even after the implant users had experienced a year of improved auditory feedback.

Purpose This study investigated the network of brain regions involved in overt production of vowels, monosyllables, and bisyllables to test hypotheses derived from the Directions Into Velocities of Articulators (DIVA) model of speech production (Guenther, Ghosh, & Tourville, 2006). The DIVA model predicts left lateralized activity in inferior frontal cortex when producing a single syllable or phoneme and increased cerebellar activity for consonant–vowel syllables compared with steady-state vowels. Method Sparse sampling functional magnetic resonance imaging (fMRI) was used to collect data from 10 right-handed speakers of American English while producing isolated monosyllables (e.g., “ba,” “oo”). Data were analyzed using both voxel-based and participant-specific anatomical region-of-interest–based techniques. Results Overt production of single monosyllables activated a network of brain regions, including left ventral premotor cortex, left posterior inferior frontal gyrus, bilateral supplementary motor area, sensorimotor cortex, auditory cortex, thalamus, and cerebellum. Paravermal cerebellum showed greater activity for consonant-vowel syllables compared to vowels. Conclusions The finding of left-lateralized premotor cortex activity supports the DIVA model prediction that this area contains cell populations representing syllable motor programs without regard for semantic content. Furthermore, the superior paravermal cerebellum is more active for consonant–vowel syllables compared with vowels, perhaps due to increased timing constraints for consonant production.

Collaborative robots, or co-robots, are semi-autonomous robotic agents designed to work alongside humans in shared workspaces. To be effective, co-robots require the ability to respond and adapt to dynamic scenarios encountered in natural environments. One way to achieve this is through exploratory learning, or “learning by doing,” an unsupervised method in which co-robots are able to build an internal model for motor planning and coordination based on real-time sensory inputs. In this paper, we present an adaptive neural network-based system for co-robot control that employs exploratory learning to achieve the coordinated motor planning needed to navigate toward, reach for, and grasp distant objects. To validate this system we used the 11-degrees-of-freedom RoPro Calliope mobile robot. Through motor babbling of its wheels and arm, the Calliope learned how to relate visual and proprioceptive information to achieve hand-eye-body coordination. By continually evaluating sensory inputs and externally provided goal directives, the Calliope was then able to autonomously select the appropriate wheel and joint velocities needed to perform its assigned task, such as following a moving target or retrieving an indicated object.

PurposeResponses to intensity perturbation during running speech were measured to understand whether prosodic features are controlled in an independent or integrated manner.MethodNineteen English-speaking healthy adults (age range = 21–41 years) produced 480 sentences in which emphatic stress was placed on either the 1st or 2nd word. One participant group received an upward intensity perturbation during stressed word production, and the other group received a downward intensity perturbation. Compensations for perturbation were evaluated by comparing differences in participants' stressed and unstressed peak fundamental frequency (F0), peak intensity, and word duration during perturbed versus baseline trials.ResultsSignificant increases in stressed–unstressed peak intensities were observed during the ramp and perturbation phases of the experiment in the downward group only. Compensations for F0 and duration did not reach significance for either group.ConclusionsConsistent with previous work, speakers appear sensitive to auditory perturbations that affect a desired linguistic goal. In contrast to previous work on F0 perturbation that supported an integrated-channel model of prosodic control, the current work only found evidence for intensity-specific compensation. This discrepancy may suggest different F0 and intensity control mechanisms, threshold-dependent prosodic modulation, or a combined control scheme.

Purpose This study was designed to test two hypotheses about apraxia of speech (AOS) derived from the Directions Into Velocities of Articulators (DIVA) model (Guenther et al., 2006): the feedforward system deficit hypothesis and the feedback system deficit hypothesis. Method The authors used noise masking to minimize auditory feedback during speech. Six speakers with AOS and aphasia, 4 with aphasia without AOS, and 2 groups of speakers without impairment (younger and older adults) participated. Acoustic measures of vowel contrast, variability, and duration were analyzed. Results Younger, but not older, speakers without impairment showed significantly reduced vowel contrast with noise masking. Relative to older controls, the AOS group showed longer vowel durations overall (regardless of masking condition) and a greater reduction in vowel contrast under masking conditions. There were no significant differences in variability. Three of the 6 speakers with AOS demonstrated the group pattern. Speakers with aphasia without AOS did not differ from controls in contrast, duration, or variability. Conclusion The greater reduction in vowel contrast with masking noise for the AOS group is consistent with the feedforward system deficit hypothesis but not with the feedback system deficit hypothesis; however, effects were small and not present in all individual speakers with AOS. Theoretical implications and alternative interpretations of these findings are discussed.