Search

Individuals with very high spinal cord injuries (e.g. C1-C3) may be ventilator-dependent and therefore unable to support speech breathing. However, their facial musculature is intact, given that these muscles are innervated by cranial nerves. We developed a system using surface electromyography (sEMG) recorded from facial muscles to control a phonemic interface and voice synthesizer and tested the system in healthy individuals. Users were able to use five facial gestures to control an onscreen cursor and the phonemic interface. Users had mean information transfer rates (ITRs) of 59.5 bits/min when calculating ITRs using the number of phonemes selected. To compare with orthographic systems, ITRs were also calculated using the equivalent number of letters required to spell the selected word. With this calculation, users had a mean ITR of 70.1. Results are promising for further development and testing in individuals with high spinal cord injuries.

In the target article, Hickok asserts that auditory and somatosensory feedback control operate at different levels in the speech production hierarchy, with somatosensory feedback control involved in lower-level phonemic processing and auditory feedback control involved in higher-level syllabic processing. This assertion is based in part on a characterization of phonemes as timeless feature bundles. In this commentary I argue that the linguistic conception of phonemes as timeless feature bundles is insufficient for characterizing the production of phonemes by the motor system, and I review evidence that auditory feedback control is used even at the sub-phonemic level of speech production, contrary to its role in the hierarchical state feedback control model.

The authors of this article suggest that the slight but consistent posture-dependent curvature of the spatial paths in the kinematic transformation between intrinsic and extrinsic coordinates may result in a systematic curvature of movements initially planned as straight-line trajectories toward the target. A kinematic planning model is presented that takes into account the anisotropy of the intrinsic and extrinsic transformation and tends to avoid movements that require excessive joint rotations by introducing slight deviations from a straight-line trajectory. Preliminary simulations showed reasonably good agreement with experimental data, especially considering that the current model is strictly based on kinematics. A quantitative analysis showed that the strategy used in the model achieves a favorable compromise between straight-line movements and angular joint changes: By slightly increasing the spatial length of the movement (i.e., by introducing curvature), an individual can greatly reduce the total amount of joint rotation required to produce the movement.

The purpose of this article is to demonstrate that self-produced auditory feedback is sufficient to train a mapping between auditory target space and articulator space under conditions in which the structures of speech production are undergoing considerable developmental restructuring. One challenge for competing theories that propose invariant constriction targets is that it is unclear what teaching signal could specify constriction location and degree so that a mapping between constriction target space and articulator space can be learned. It is predicted that a model trained by auditory feedback will accomplish speech goals, in auditory target space, by continuously learning to use different articulator configurations to adapt to the changing acoustic properties of the vocal tract during development. The Maeda articulatory synthesis part of the DIVA neural network model (Guenther et al., 1998) was modified to reflect the development of the vocal tract by using measurements taken from MR images of children. After training, the model was able to maintain the 11 English vowel targets in auditory planning space, utilizing varying articulator configurations, despite morphological changes that occur during development. The vocal-tract constriction pattern (derived from the vocal-tract area function) as well as the formant values varied during the course of development in correspondence with morphological changes in the structures involved with speech production. Despite changes in the acoustical properties of the vocal tract that occur during the course of development, the model was able to demonstrate motor-equivalent speech production under lip-restriction conditions. The model accomplished this in a self-organizing manner even though there was no prior experience with lip restriction during training.

A theoretical overview and supporting data are presented about the control of the segmental component of speech production. Findings of “motor-equivalent” trading relations between the contributions of two constrictions to the same acoustic transfer function provide preliminary support for the idea that segmental control is based on acoustic or auditory-perceptual goals. The goals are determined partly by non-linear, quantal relations (called “saturation effects”) between motor commands and articulatory movements and between articulation and sound. Since processing times would be too long to allow the use of auditory feedback for closed-loop error correction in achieving acoustic goals, the control mechanism must use a robust “internal model” of the relation between articulation and the sound output that is learned during speech acquisition. Studies of the speech of cochlear implant and bilateral acoustic neuroma patients provide evidence supporting two roles for auditory feedback in adults: maintenance of the internal model, and monitoring the acoustic environment to help assure intelligibility by guiding relatively rapid adjustments in “postural” parameters underlying average sound level, speaking rate and the amount of prosodically-based inflection of F0 and SPL.

Co-robotic assistants, or Cobots can improve the quality of life for individuals with locked-in syndrome (LIS) by allowing augmented control over their surroundings. Implemented in collaboration between the Boston University Neuromorphics Lab and Neural Prosthesis Lab, this work provides a proof of concept of an autonomous robot coupled with a non-invasive brain machine interface (BMI). The system uses Steady State Visually Evoked Potential (SSVEP) for target selection and a massively parallel neural network that models functionality of the primate “where” and “what” visual pathways. The simulated visual processes perform object recognition on images streamed from the Cobot equipped with a pan-and-tilt camera. In this paper, we describe a subcomponent of the system designed to allow the neural network to learn the identity of objects, the user to select a target object and the Cobot to perform autonomous visual investigation.

A neural model is described of how the brain may autonomously learn a body-centered representation of a three-dimensional (3-D) target position by combining information about retinal target position, eye position, and head position in real time. Such a body-centered spatial representation enables accurate movement commands to the limbs to be generated despite changes in the spatial relationships between the eyes, head, body, and limbs through time. The model learns a vector representation—otherwise known as a parcellated distributed representation—of target vergence with respect to the two eyes, and of the horizontal and vertical spherical angles of the target with respect to a cyclopean egocenter. Such a vergence-spherical representation has been reported in the caudal midbrain and medulla of the frog, as well as in psychophysical movement studies in humans. A head-centered vergence-spherical representation of foveated target position can be generated by two stages of opponent processing that combine corollary discharges of outflow movement signals to the two eyes. Sums and differences of opponent signals define angular and vergence coordinates, respectively. The head-centered representation interacts with a binocular visual representation of nonfoveated target position to learn a visuomotor representation of both foveated and nonfoveated target position that is capable of commanding yoked eye movements. This head-centered vector representation also interacts with representations of neck movement commands to learn a body-centered estimate of target position that is capable of Commanding coordinated arm movements. Learning occurs during head movements made while gaze remains fixed on a foveated target. An initial estimate is stored and a VOR-mediated gating signal prevents the stored estimate from being reset during a gaze-maintaining head movement. As the head moves, new estimates are compared with the stored estimate to compute difference vectors which act as error signals that drive the learning process, as well as control the on-line merging of multimodal information.

Auditory feedback is instrumental in the online control of speech, allowing speakers to compare their self-produced speech signal with a desired auditory target and correct for errors. However, there is little account of the representation of “target” and “error”: does error depend purely on acoustic distance from a target, or is error enhanced by phoneme category changes? Here, we show an effect of vowel boundaries on compensatory responses to a real-time auditory perturbation. While human subjects spoke monosyllabic words, event-triggered functional magnetic resonance imaging was used to characterize neural responses to unexpected changes in auditory feedback. Capitalizing on speakers' natural variability, we contrasted the responses to feedback perturbations applied to two classes of utterances: (1) those that fell nearer to the category boundary, for which perturbations were designed to change the phonemic identity of the heard speech; and (2) those that fell farther from the boundary, for which perturbations resulted in only sub-phonemic auditory differences. Subjects' behavioral compensation was more than three times greater when feedback shifts were applied nearer to a category boundary. Furthermore, a near-boundary shift resulted in stronger cortical responses, most notably in right posterior superior temporal gyrus, than an identical shift that occurred far from the boundary. Across participants, a correlation was found between the amount of compensation to the perturbation and the amount of activity in a network of superior temporal and inferior frontal brain regions. Together, these results demonstrate that auditory feedback control of speech is sensitive to linguistic categories learned through auditory experience.

The perceptual magnet effect is one of the earliest known language-specific phenomena arising in infant speech development. The effect is characterized by a warping of perceptual space near phonemic category centers. Previous explanations have been formulated within the theoretical framework of cognitive psychology. The model proposed in this paper builds on research from both psychology and neuroscience in working toward a more complete account of the effect. The model embodies two principal hypotheses supported by considerable experimental and theoretical research from the neuroscience literature: (1) sensory experience guides language-specific development of an auditory neural map, and (2) a population vector can predict psychological phenomena based on map cell activities. These hypotheses are realized in a self-organizing neural network model. The magnet effect arises in the model from language-specific nonuniformities in the distribution of map cell firing preferences. Numerical simulations verify that the model captures the known general characteristics of the magnet effect and provides accurate fits to specific psychophysical data.

Auditory feedback plays an important role in speech motor learning and in the online correction of speech movements. Speakers can detect and correct auditory feedback errors at the segmental and suprasegmental levels during ongoing speech. The frontal brain regions that contribute to these corrective movements have also been shown to be more active during speech in persons who stutter (PWS) compared to fluent speakers. Further, various types of altered auditory feedback can temporarily improve the fluency of PWS, suggesting that atypical auditory-motor interactions during speech may contribute to stuttering disfluencies. To investigate this possibility, we have developed and improved Audapter, a software that enables configurable dynamic perturbation of the spatial and temporal content of the speech auditory signal in real time. Using Audapter, we have measured the compensatory responses of PWS to static and dynamic perturbations of the formant content of auditory feedback and compared these responses with those from matched fluent controls. Our findings indicate deficient utilization of auditory feedback by PWS for short-latency online control of the spatial and temporal parameters of articulation during vowel production and during running speech. These findings provide further evidence that stuttering is associated with aberrant auditory-motor integration during speech.

Understanding the neural mechanisms underlying speech production can aid the design and implementation of brain–computer interfaces for speech communication. Specifically, the act of speech production is unequivocally a motor behavior; speech arises from the precise activation of all of the muscles of the respiratory and vocal mechanisms. Speech also preferentially relies on auditory output to communicate information between conversation partners. However, self-perception of one’s own speech is also important for maintaining error-free speech and proper production of intended utterances. This chapter discusses our efforts to use motor cortical neural output during attempted speech production for control of a communication BCI device by an individual with locked-in syndrome while taking advantage of neural circuits used for learning and maintaining speech. The end result is a BCI capable of producing instantaneously vocalized output within a framework of motor-based brain-computer interfacing that provides appropriate auditory feedback to the user.

Speech production involves a combination of feedforward and sensory feedback-based control mechanisms. The latter have been characterized in experiments involving real-time perturbations during speech. Unexpected perturbations of auditory feedback result in corrective motor responses with a minimum latency of approximately 100 ms after perturbation onset. These responses have been shown for both pitch and formant frequency perturbations, and the responsible neural circuitry includes portions of the superior temporal gyrus and ventral premotor cortex (vPMC). Corrective responses to somatosensory perturbations (such as a downward force applied to the jaw) occur approximately 50 ms after perturbation onset and are mediated by ventral somatosensory cortex and vPMC. The degree to which an individual weights auditory versus somatosensory feedback varies substantially. Such differences rely in part on differences in sensory acuity, e.g., a speaker with relatively poor hearing is likely to rely more heavily on somatosensory feedback control mechanisms than auditory feedback control mechanisms. Additionally, reliance on feedback control may be modulated to compensate for impairments in the feedforward system for speech, e.g., adults who stutter show a reduced reliance on auditory feedback control compared to fluent speakers, perhaps because auditory feedback can have a deleterious effect on speech initiation in stuttering.

This article describes a neural network model of speech motor skill acquisition and speech production that explains a wide range of data on variability, motor equivalence, coarticulation, and rate effects. Model parameters are learned during a babbling phase. To explain how infants learn language-specific variability limits, speech sound targets take the form of convex regions, rather than points, in orosensory coordinates. Reducing target size for better accuracy during slower speech leads to differential effects for vowels and consonants, as seen in experiments previously used as evidence for separate control processes for the 2 sound types. Anticipatory coarticulation arises when targets are reduced in size on the basis of context; this generalizes the well-known look-ahead model of coarticulation. Computer simulations verify the model's properties.

In this paper we present a framework for reducing the development time needed for creating applications for use in non-invasive brain-computer interfaces (BCI). Our framework is primarily focused on facilitating rapid software “app” development akin to current efforts in consumer portable computing (e.g. smart phones and tablets). This is accomplished by handling intermodule communication without direct user or developer implementation, instead relying on a core subsystem for communication of standard, internal data formats. We also provide a library of hardware interfaces for common mobile EEG platforms for immediate use in BCI applications. A use-case example is described in which a user with amyotrophic lateral sclerosis participated in an electroencephalography-based BCI protocol developed using the proposed framework. We show that our software environment is capable of running in real-time with updates occurring 50–60 times per second with limited computational overhead (5 ms system lag) while providing accurate data acquisition and signal analysis.

This article describes a computational model, called DIVA, that provides a quantitative framework for understanding the roles of various brain regions involved in speech acquisition and production. An overview of the DIVA model is first provided, along with descriptions of the computations performed in the different brain regions represented in the model. Particular focus is given to the model's speech sound map, which provides a link between the sensory representation of a speech sound and the motor program for that sound. Neurons in this map share with ''mirror neurons'' described in monkey ventral premotor cortex the key property of being active during both production and perception of specific motor actions. As the DIVA model is defined both computationally and anatomically, it is ideal for generating precise predictions concerning speech-related brain activation patterns observed during functional imaging experiments. The DIVA model thus provides a well-defined framework for guiding the interpretation of experimental results related to the putative human speech mirror system. Refereed/Peer-reviewed

Within the human motor repertoire, speech production has a uniquely high level of spatiotemporal complexity. The production of running speech comprises the traversing of spatial positions with precisely coordinated articulator movements to produce 10–15 sounds/s. How does the brain use auditory feedback, namely the self-perception of produced speech sounds, in the online control of spatial and temporal parameters of multisyllabic articulation? This question has important bearings on the organizational principles of sequential actions, yet its answer remains controversial due to the long latency of the auditory feedback pathway and technical challenges involved in manipulating auditory feedback in precisely controlled ways during running speech. In this study, we developed a novel technique for introducing time-varying, focal perturbations in the auditory feedback during multisyllabic, connected speech. Manipulations of spatial and temporal parameters of the formant trajectory were tested separately on two groups of subjects as they uttered “I owe you a yo-yo.” Under these perturbations, significant and specific changes were observed in both the spatial and temporal parameters of the produced formant trajectories. Compensations to spatial perturbations were bidirectional and opposed the perturbations. Furthermore, under perturbations that manipulated the timing of auditory feedback trajectory (slow-down or speed-up), significant adjustments in syllable timing were observed in the subjects' productions. These results highlight the systematic roles of auditory feedback in the online control of a highly over-learned action as connected speech articulation and provide a first look at the properties of this type of sensorimotor interaction in sequential movements.

This paper reports on studies involving brain-machine interfaces (BMIs) that provide near-instantaneous audio feedback from a speech synthesizer to the BMI user. In one study, neural signals recorded by an intracranial electrode implanted in a speech-related region of the left precentral gyrus of a human volunteer suffering from locked-in syndrome were transmitted wirelessly across the scalp and used to drive a formant synthesizer, allowing the user to produce vowels. In a second, pilot study, a neurologically normal user was able to drive the formant synthesizer with imagined movements detected using electroencephalography. Our results support the feasibility of neural prostheses that have the potential to provide near-conversational synthetic speech for individuals with severely impaired speech output.

We conducted a neurophysiological study of attempted speech production in a paralyzed human volunteer using chronic microelectrode recordings. The volunteer suffers from locked-in syndrome leaving him in a state of near-total paralysis, though he maintains good cognition and sensation. In this study, we investigated the feasibility of supervised classification techniques for prediction of intended phoneme production in the absence of any overt movements including speech. Such classification or decoding ability has the potential to greatly improve the quality-of-life of many people who are otherwise unable to speak by providing a direct communicative link to the general community. We examined the performance of three classifiers on a multi-class discrimination problem in which the items were 38 American English phonemes including monophthong and diphthong vowels and consonants. The three classifiers differed in performance, but averaged between 16 and 21% overall accuracy (chance-level is 1/38 or 2.6%). Further, the distribution of phonemes classified statistically above chance was non-uniform though 20 of 38 phonemes were classified with statistical significance for all three classifiers. These preliminary results suggest supervised classification techniques are capable of performing large scale multi-class discrimination for attempted speech production and may provide the basis for future communication prostheses.

This article describes how corollary discharges from outflow eye movement commands can be transformed by two stages of opponent neural processing into a head-centered representation of 3-D target position. This representation implicitly defines a cyclopean coordinate system whose variables approximate the binocular vergence and spherical horizontal and vertical angles with respect to the observer's head. Various psychophysical data concerning binocular distance perception and reaching behavior are clarified by this representation. The representation provides a foundation for learning head-centered and body-centered invariant representations of both foveated and non-foveated 3-D target positions. It also enables a solution to be developed of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space.

A neural network model is described for how an invariant head-centered representation of 3-D target position can be autonomously learned by the brain in real time. Once learned, such a target representation may be used to control both eye and limb movements. The target representation is derived from the positions of both eyes in the head, and the locations which the target activates on the retinas of both eyes. A Vector Associative Map (VAM) learns the many-to-one transformation from multiple combinations of eye-and-retinal position to invariant 3-D target position. Eye position is derived from outflow movement signals to the eye muscles. Two successive stages of opponent processing convert these corollary discharges into a head-centered representation that closely approximates the azimuth, elevation, and vergence of the eyes' gaze position with respect to a cyclopean origin located between the eyes. VAM learning combines this cyclopean representation of present gaze position with binocular retinal information about target position into an invariant representation of 3-D target position with respect to the head. VAM learning can use a teaching vector that is externally derived from the positions of the eyes when they foveate the target. A VAM can also autonomously discover and learn the invariant representation, without an explicit teacher, by generating internal error signals from environmental fluctuations in which these invariant properties are implicit. VAM error signals are computed by Difference Vectors (DVs) that are zeroed by the VAM learning process. VAMs may be organized into VAM Cascades for learning and performing both sensory-to-spatial maps and spatial-to-motor maps. These multiple uses clarify why DV-type properties are computed by cells in the parietal, frontal, and motor cortices of many mammals. VAMs are modulated by gating signals that express different aspects of the will-to-act. These signals transform a single invariant representation into movements of different speed (GO signal) and size (GRO signal), and thereby enable VAM controllers to match a planned action sequence to variable environmental conditions.

This paper describes a self-organizing neural model for eye-hand coordination. Called the DIRECT model, it embodies a solution of the classical motor equivalence problem. Motor equivalence computations allow humans and other animals to flexibly employ an arm with more degrees of freedom than the space in which it moves to carry out spatially defined tasks under conditions that may require novel joint configurations. During a motor babbling phase, the model endogenously generates movement commands that activate the correlated visual, spatial, and motor information that are used to learn its internal coordinate transformations. After learning occurs, the model is capable of controlling reaching movements of the arm to prescribed spatial targets using many different combinations of joints. When allowed visual feedback, the model can automatically perform, without additional learning, reaches with tools of variable lengths, with clamped joints, with distortions of visual input by a prism, and with unexpected perturbations. These compensatory computations occur within a single accurate reaching movement. No corrective movements are needed. Blind reaches using internal feedback have also been simulated. The model achieves its competence by transforming visual information about target position and end effector position in 3-D space into a body-centered spatial representation of the direction in 3-D space that the end effector must move to contact the target. The spatial direction vector is adaptively transformed into a motor direction vector, which represents the joint rotations that move the end effector in the desired spatial direction from the present arm configuration. Properties of the model are compared with psychophysical data on human reaching movements, neurophysiological data on the tuning curves of neurons in the monkey motor cortex, and alternative models of movement control.

Purpose A feedback perturbation paradigm was used to investigate whether prosodic cues are controlled independently or in an integrated fashion during sentence production. Method Twenty-one healthy speakers of American English were asked to produce sentences with emphatic stress while receiving real-time auditory feedback of their productions. The fundamental frequency (F0) of the stressed word in each 4-word sentence was selectively shifted in a sensorimotor adaptation protocol. Speakers experienced either an upward or a downward shift of the stressed word, which gradually altered the perceived stress of the sentence. Results Participants in the Up and Down groups adapted to F0 shifts by altering the contrast between stressed and unstressed words differentially, such that the two groups deviated from each other in the perturbation phase. Furthermore, selective F0 perturbation in sentences with emphatic stress resulted in compensatory changes in both F0 and intensity. Conclusions Present findings suggest that F0 and intensity are controlled in an integrated fashion to maintain the contrast between stressed and unstressed words. When a cue is impaired through perturbation, speakers not only oppose the perturbation but enhance other prosodic cues to achieve emphatic stress.

Brain-computer interfaces (BCIs) have been developed over the past decade to restore communication to persons with severe paralysis. In the most severe cases of paralysis, known as locked-in syndrome, patients retain cognition and sensation, but are capable of only slight voluntary eye movements. For these patients, no standard communication method is available, although some can use BCIs to communicate by selecting letters or words on a computer. Recent research has sought to improve on existing techniques by using BCIs to create a direct prediction of speech utterances rather than to simply control a spelling device. Such methods are the first steps towards speech prostheses as they are intended to entirely replace the vocal apparatus of paralyzed users. This article outlines many well known methods for restoration of communication by BCI and illustrates the difference between spelling devices and direct speech prediction or speech prosthesis.

Somatosensory feedback plays a critical role in the coordination of articulator movements for speech production. In response to unexpected resistance to lip or jaw movements during speech, fluent speakers can use the difference between the somatosensory expectations of a speech sound and the actual somatosensory feedback to adjust the trajectories of functionally relevant but unimpeded articulators. In an effort to investigate the neural substrates underlying the somatosensory feedback control of speech, we used an event-related sparse sampling functional magnetic resonance imaging paradigm and a novel pneumatic device that unpredictably blocked subjects' jaw movements. In comparison to speech, perturbed speech, in which jaw perturbation prompted the generation of compensatory speech motor commands, demonstrated increased effects in bilateral ventral motor cortex, right-lateralized anterior supramarginal gyrus, inferior frontal gyrus pars triangularis and ventral premotor cortex, and bilateral inferior posterior cerebellum (lobule VIII). Structural equation modeling revealed a significant increased influence from left anterior supramarginal gyrus to right anterior supramarginal gyrus and from left anterior supramarginal gyrus to right ventral premotor cortex as well as a significant increased reciprocal influence between right ventral premotor cortex and right ventral motor cortex and right anterior supramarginal gyrus and right inferior frontal gyrus pars triangularis for perturbed speech relative to speech. These results suggest that bilateral anterior supramarginal gyrus, right inferior frontal gyrus pars triangularis, right ventral premotor and motor cortices are functionally coupled and influence speech motor output when somatosensory feedback is unexpectedly perturbed during speech production.

This study explores the idea that a speaker's choice of a strategy to compensate for a vocal-tract perturbation depends on the shape of the perturbed vocal tract. Speakers' palatal shapes were perturbed with palatal prostheses. Three speakers used an alveolar prosthesis that effectively moved the alveolar ridge toward the back; three used a central prosthesis that effectively flattened the palate. We hypothesized that during production of the front-rounded vowel /y/ the speakers with the alveolar prosthesis would compensate for the shortened anterior cavity with increased lip protrusion. Lip and tongue movement data from EMA recordings of the speakers' adaptive behavior supported the hypothesis: those whose front cavity was shortened by the palatal prosthesis increased lip protrusion; those with a flattened palate did not. This difference in adaptation strategies was investigated further using simulations with the DIVA model of speech production. The model's vocal tract was adapted to fit two of the speakers' vocal tracts (one with each type of prosthesis), using vocal-tract shape data from structural MRI recordings. Simulations of the model agree with the experimental results: compensation for the alveolar prosthesis was accomplished mainly with lip protrusion, whereas with the central prosthesis, it was accomplished with tongue movement.

International audience; Functional neuroimaging studies have converged on a core network of brain regions that supports speech production, but the sublexical processing stages performed by the different parts of this network remain unclear. Using an fMRI adaptation paradigm and quantitative analysis of patterns of activation rather than contrast subtractions alone. we were able to identify a set of neural substrates predominantly engaged in phonemic, syllabic, and supra-syllabic levels of processing during speech. Phoneme-level processes were found in the left SMA, pallidum, posterior superior temporal gyrus, and superior lateral cerebellum. Syllable-level processes were found in the left ventral premotor cortex, and supra-syllabic processes related to phonological chunking were found in the right superior lateral cerebellum, Active regions that were not sensitive to sublexical manipulations included primary motor and auditory cortical areas. and medial cerebellum. These results offer a quantitative technique for localizing sublexical neural processes that are difficult to dissociate using non-invasive imaging techniques and provide the beginnings of a ``brain map'' for language output. (C) 2009 Elsevier Inc. All rights reserved.

Purpose Childhood apraxia of speech (CAS) has been associated with a wide variety of diagnostic descriptions and has been shown to involve different symptoms during successive stages of development. In the present study, the authors attempted to associate the symptoms of CAS in a particular developmental stage with particular information-processing deficits by using computational modeling with the Directions Into Velocities of Articulators (DIVA) model. The hypothesis was that the speech production system in CAS suffers from poor feed-forward control and, consequently, an increased reliance on the feedback control subsystem. Method In a series of computer simulations, the authors systematically varied the ratio between feed-forward and feedback control during production attempts in the acquisition of feed-forward motor commands. The simulations were evaluated acoustically on 4 selected key symptoms of CAS. Results Results showed that increasing the reliance on feedback control causes increased severity of these 4 symptoms of CAS: deviant coarticulation, speech sound distortion, searching articulation, and increased variability. Conclusions The findings support the idea that the key symptoms found in CAS could result from an increased reliance on feedback control due to poor feed-forward commands. Two possible root causes of degraded feed-forward control in CAS are discussed: reduced somatosensory information and increased levels of neural noise.

Summary The source of conscious experience has fueled scientific and philosophical debates for centuries. In the auditory and motor domains, it is not yet known how consciously and unconsciously obtained information combine to enable the production and perception of speaking and singing. Both forms of vocalization rely upon the interaction of brain networks responsible for perception and action. While perceptual experience and executed actions are usually well coupled, dissociations between perception and action can be informative. Here we report such a dissociation: tone-deaf individuals, who cannot consciously perceive pitch differences, can paradoxically reproduce pitch intervals in correct directions. Our results suggest that multiple neural pathways have evolved for sound perception and production, so that pitch information sufficient for intact speech can be obtained separately from pathways necessary for conscious perception.

The role of auditory feedback in speech motor control was explored in three related experiments. Experiment 1 investigated auditory sensorimotor adaptation: the process by which speakers alter their speech production to compensate for perturbations of auditory feedback. When the first formant frequency (F1) was shifted in the feedback heard by subjects as they produced vowels in consonant-vowel-consonant (CVC) words, the subjects' vowels demonstrated compensatory formant shifts that were maintained when auditory feedback was subsequently masked by noise-evidence of adaptation. Experiment 2 investigated auditory discrimination of synthetic vowel stimuli differing in F1 frequency, using the same subjects. Those with more acute F1 discrimination had compensated more to F1 perturbation. Experiment 3 consisted of simulations with the directions into velocities of articulators model of speech motor planning, which showed that the model can account for key aspects of compensation. In the model, movement goals for vowels are regions in auditory space; perturbation of auditory feedback invokes auditory feedback control mechanisms that correct for the perturbation, which in turn causes updating of feedforward commands to incorporate these corrections. The relation between speaker acuity and amount of compensation to auditory perturbation is mediated by the size of speakers' auditory goal regions, with more acute speakers having smaller goal regions.

PurposeTo assess the effects of short- and long-term changes in auditory feedback on vowel and sibilant contrasts and to evaluate hypotheses arising from a model of speech motor planning.MethodThe perception and production of vowel and sibilant contrasts were measured in 8 postlingually deafened adults prior to activation of their cochlear implant speech processors, 1 month postactivation, and 1 year postactivation. Measures were taken postactivation both with and without auditory feedback. Contrast measures were also made for a group of speakers with reportedly normal hearing speaking with masked and unmasked auditory feedback.ResultsVowel and sibilant contrasts, measured in the absence of auditory feedback after 1 month of prosthesis use, were diminished compared with their values measured before prosthesis. Contrasts measured in the absence of auditory feedback after 1 year’s experience with the prosthesis were increased compared with their values after 1 month’s experience. In both time samples, contrasts were enhanced when auditory feedback was restored.ConclusionThe provision of prosthetic hearing to postlingually deafened adults impaired their phonemic contrasts at first, as their auditory feedback had novel characteristics. Once auditory feedback became recalibrated with prosthesis use, it could, in turn, revise feedforward commands that control the contrasts in its absence.

The neural substrates underlying auditory feedback control of speech were investigated using a combination of functional magnetic resonance imaging (fMRI) and computational modeling. Neural responses were measured while subjects spoke monosyllabic words under two conditions: (i) normal auditory feedback of their speech, and (ii) auditory feedback in which the first formant frequency of their speech was unexpectedly shifted in real time. Acoustic measurements showed compensation to the shift within approximately 135 ms of onset. Neuroimaging revealed increased activity in bilateral superior temporal cortex during shifted feedback, indicative of neurons coding mismatches between expected and actual auditory signals, as well as right prefrontal and Rolandic cortical activity. Structural equation modeling revealed increased influence of bilateral auditory cortical areas on right frontal areas during shifted speech, indicating that projections from auditory error cells in posterior superior temporal cortex to motor correction cells in right frontal cortex mediate auditory feedback control of speech.

The role of auditory feedback in speech production was investigated by examining speakers’ phonemic contrasts produced under increases in the noise to signal ratio (N/S). Seven cochlear implant users and seven normal-hearing controls pronounced utterances containing the vowels /i/, /u/, /e/ and /ae/ and the sibilants /s/ and /ʃ/ while hearing their speech mixed with noise at seven equally spaced levels between their thresholds of detection and discomfort. Speakers’ average vowel duration and SPL generally rose with increasing N/S. Average vowel contrast was initially flat or rising; at higher N/S levels, it fell. A contrast increase is interpreted as reflecting speakers’ attempts to maintain clarity under degraded acoustic transmission conditions. As N/S increased, speakers could detect the extent of their phonemic contrasts less effectively, and the competing influence of economy of effort led to contrast decrements. The sibilant contrast was more vulnerable to noise; it decreased over the entire range of increasing N/S for controls and was variable for implant users. The results are interpreted as reflecting the combined influences of a clarity constraint, economy of effort and the effect of masking on achieving auditory phonemic goals—with implant users less able to increase contrasts in noise than controls.

Speech production involves the integration of auditory, somatosensory, and motor information in the brain. This article describes a model of speech motor control in which a feedforward control system, involving premotor and primary motor cortex and the cerebellum, works in concert with auditory and somatosensory feedback control systems that involve both sensory and motor cortical areas. New speech sounds are learned by first storing an auditory target for the sound, then using the auditory feedback control system to control production of the sound in early repetitions. Repeated production of the sound leads to tuning of feedforward commands which eventually supplant the feedback-based control signals. Although parts of the model remain speculative, it accounts for a wide range of kinematic, acoustic, and neuroimaging data collected during speech production and provides a framework for investigating communication disorders that involve malfunction of the cerebral cortex and interconnected subcortical structures.Readers will be able to: (1) describe several types of learning that occur in the sensory-motor system during babbling and early speech, (2) identify three neural control subsystems involved in speech production, (3) identify regions of the brain involved in monitoring auditory and somatosensory feedback during speech production, and (4) identify regions of the brain involved in feedforward control of speech.

Fluent speech comprises sequences that are composed from a finite alphabet of learned words, syllables, and phonemes. The sequencing of discrete motor behaviors has received much attention in the motor control literature, but relatively little has been focused directly on speech production. In this paper, we investigate the cortical and subcortical regions involved in organizing and enacting sequences of simple speech sounds. Sparse event-triggered functional magnetic resonance imaging (fMRI) was used to measure responses to preparation and overt production of non-lexical three-syllable utterances, parameterized by two factors: syllable complexity and sequence complexity. The comparison of overt production trials to preparation only trials revealed a network related to the initiation of a speech plan, control of the articulators, and to hearing one's own voice. This network included the primary motor and somatosensory cortices, auditory cortical areas, supplementary motor area (SMA), the precentral gyrus of the insula, and portions of the thalamus, basal ganglia, and cerebellum. Additional stimulus complexity led to increased engagement of the basic speech network and recruitment of additional areas known to be involved in sequencing non-speech motor acts. In particular, the left hemisphere inferior frontal sulcus and posterior parietal cortex, and bilateral regions at the junction of the anterior insula and frontal operculum, the SMA and pre-SMA, the basal ganglia, anterior thalamus, and the cerebellum showed increased activity for more complex stimuli. We hypothesize mechanistic roles for the extended speech production network in the organization and execution of sequences of speech sounds.

This study explores the effects of hearing status and bite blocks on vowel production. Normal-hearing controls and postlingually deaf adults read elicitation lists of ∕hVd∕ syllables with and without bite blocks and auditory feedback. Deaf participants’ auditory feedback was provided by a cochlear prosthesis and interrupted by switching off their implant microphones. Recording sessions were held before prosthesis was provided and one month and one year after. Long-term absence of auditory feedback was associated with heightened dispersion of vowel tokens, which was inflated further by inserting bite blocks. The restoration of some hearing with prosthesis reduced dispersion. Deaf speakers’ vowel spaces were reduced in size compared to controls. Insertion of bite blocks reduced them further because of the speakers’ incomplete compensation. A year of prosthesis use increased vowel contrast with feedback during elicitation. These findings support the inference that models of speech production must assign a role to auditory feedback in error-based correction of feedforward commands for subsequent articulatory gestures.

This paper describes a neural model of speech acquisition and production that accounts for a wide range of acoustic, kinematic, and neuroimaging data concerning the control of speech movements. The model is a neural network whose components correspond to regions of the cerebral cortex and cerebellum, including premotor, motor, auditory, and somatosensory cortical areas. Computer simulations of the model verify its ability to account for compensation to lip and jaw perturbations during speech. Specific anatomical locations of the model's components are estimated, and these estimates are used to simulate fMRI experiments of simple syllable production.

fMRI analysis techniques are presented that test functional hypotheses at the region of interest (ROI) level. An SPM-compatible Matlab toolbox has been developed that allows the creation of subject-specific ROI masks based on anatomical markers and the testing of functional hypotheses on the regional response using multivariate time-series analysis techniques. The combined application of subject-specific ROI definition and region-level functional analysis is shown to appropriately compensate for intersubject anatomical variability, offering finer localization and increased sensitivity to task-related effects than standard techniques based on whole-brain normalization and voxel or cluster-level functional analysis, while providing a more direct link between discrete brain region hypotheses and the statistical analyses used to test them.

A neural network is introduced which provides a solution of the classical motor equivalence problem, whereby many different joint configurations of a redundant manipulator can all be used to realize a desired trajectory in 3-D space. To do this, the network self-organizes a mapping from motion directions in 3-D space to velocity commands in joint space. Computer simulations demonstrated that without any additional learning the network can generate accurate movement commands that compensate for variable tool lengths, clamping of joints, distortions of visual input by a prism, and unexpected limb perturbations. Blind reaches have also been simulated. >

A neural network model of how the brain forms spatial representations with which to control sensory-guided and memory-guided eye and limb movements is introduced. These spatial representations are expressed in both head-centered coordinates and body-centered coordinates because the eyes move within the head, whereas the head, arms and legs move with respect to the body. A key process in the formation of spatial representations whereby humans and other mammals can skilfully act upon objects in 3D space despite the variable relative location of sensing and acting segments is analyzed. The resulting spatial representations are built up from the same types of computations that are used to control motor commands. The natural form of neural computations that are appropriate for representation and control of a bilaterally symmetric body is studied. Bilateral symmetry leads to the use of competitive and cooperative interactions among bilaterally symmetric body segments. These include opponent interactions between pairs of antagonistic neurons that measure spatial or motor offset with respect to an axis of symmetry. >

The DIRECT model is a self-organizing neural network designed to explain neurophysiological and psychophysical data from targeted reaching experiments. The model's learning process is indirectly influenced by the arm's kinematics, resulting in movements biased toward joint rotations that produce the most spatial movement of the end-effector. This bias causes the end-effector trajectories performed after learning to deviate slightly from the straight path which would be produced by an explicit pseudoinverse computation, but the total joint rotation is significantly reduced by this slight curvature. A simplified model of this biasing is introduced, and implications regarding human arm movements are discussed.

Beginning with the seminal work of McCulloch and Pitts in the early 1940s, the pursuit of increasingly accurate mathematical characterizations of the electrophysiological properties of individual neurons and networks of interconnected neurons has been a driving force in neural network research. This line of research has branched into descriptions of the nervous system at many different grains of analysis, including complex mathematical models of the properties of a single neuron, models of simple biological networks involving small numbers of neurons, and more abstract models which act as simplified treatments of large-scale networks involving millions of neurons in the nervous systems of humans and other vertebrates. At various points in the history of neural network research, successful neural network models have spread beyond the biological modeling domain into the domains of engineering and medical applications. Because they are capable of learning complicated nonlinear relationships from sets of training examples, neural networks are particularly well suited to pattern recognition problems involving the detection of complicated trends in high-dimensional datasets, such as the detection of medical abnormalities from physiological measures.

Psychophysical phenomena such as categorical perception and the perceptual magnet effect indicate that our auditory perceptual spaces are warped for some stimuli. This paper investigates the effects of two different kinds of training on auditory perceptual space. It is first shown that categorization training using non-speech stimuli, in which subjects learn to identify stimuli within a particular frequency range as members of the same category, can lead to a decrease in sensitivity to stimuli in that category. This phenomenon is an example of acquired similarity and apparently has not been previously demonstrated for a category-relevant dimension. Discrimination training with the same set of stimuli was shown to have the opposite effect: subjects became more sensitive to differences in the stimuli presented during training. Further experiments investigated some of the conditions that are necessary to generate the acquired similarity found in the first experiment. The results of these experiments are used to evaluate two neural network models of the perceptual magnet effect. These models, in combination with our experimental results, are used to generate an experimentally testable prediction concerning changes in the brain’s auditory maps under different training conditions.