Search

Audiovisual speech perception includes the simultaneous processing of auditory and visual speech. Deficits in audiovisual speech perception are reported in autistic individuals; however, less is known regarding audiovisual speech perception within the broader autism phenotype (BAP), which includes individuals with elevated, yet subclinical, levels of autistic traits. We investigate the neural indices of audiovisual speech perception in adults exhibiting a range of autism-like traits using event-related potentials (ERPs) in a phonemic restoration paradigm. In this paradigm, we consider conditions where speech articulators (mouth and jaw) are present (AV condition) and obscured by a pixelated mask (PX condition). These two face conditions were included in both passive (simply viewing a speaking face) and active (participants were required to press a button for a specific consonant–vowel stimulus) experiments. The results revealed an N100 ERP component which was present for all listening contexts and conditions; however, it was attenuated in the active AV condition where participants were able to view the speaker's face, including the mouth and jaw. The P300 ERP component was present within the active experiment only, and significantly greater within the AV condition compared to the PX condition. This suggests increased neural effort for detecting deviant stimuli when visible articulation was present and visual influence on perception. Finally, the P300 response was negatively correlated with autism-like traits, suggesting that higher autistic traits were associated with generally smaller P300 responses in the active AV and PX conditions. The conclusions support the finding that atypical audiovisual processing may be characteristic of the BAP in adults. [ABSTRACT FROM AUTHOR]

Lip-reading improves intelligibility in noisy acoustical environments. We hypothesized that watching mouth movements benefits speech comprehension in a cocktail party by strengthening the encoding of the neural representations of the visually paired speech stream. In an audiovisual (AV) task, EEG was recorded as participants watched and listened to videos of a speaker uttering a sentence while also hearing a concurrent sentence by a speaker of the opposite gender. A key manipulation was that each audio sentence had a 200-ms segment replaced by white noise. To assess comprehension, subjects were tasked with transcribing the AV-attended sentence on randomly selected trials. In the auditory-only trials, subjects listened to the same sentences and completed the same task while watching a static picture of a speaker of either gender. Subjects directed their listening to the voice of the gender of the speaker in the video. We found that the N1 auditory-evoked potential (AEP) time-locked to white noise onsets was significantly more inhibited for the AV-attended sentences than for those of the auditorily-attended (A-attended) and AV-unattended sentences. N1 inhibition to noise onsets has been shown to index restoration of phonemic representations of degraded speech. These results underscore that attention and congruency in the AV setting help streamline the complex auditory scene, partly by reinforcing the neural representations of the visually attended stream, heightening the perception of continuity and comprehension.

Face to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was visible) and a pixelated condition (articulatory information was not visible). Further, task demands were manipulated by having listeners respond in a passive (no response) or an active (button press response) context. The active experiment required participants to discriminate between speech stimuli and was designed to mimic environmental situations which require one to use visual information to disambiguate the speaker's message, simulating different listening conditions in real-world settings. Stimuli included a clear exemplar of the syllable /ba/ and a second exemplar in which the formant initial consonant was reduced creating an /a/-like consonant. Consistent with our hypothesis, results revealed that the greatest fixations to the mouth were present in the audiovisual active experiment and visual articulatory information led to a phonemic restoration effect for the /a/ speech token. In the pixelated condition, participants fixated on the eyes, and discrimination of the deviant token within the active experiment was significantly greater than the audiovisual condition. These results suggest that when required to disambiguate changes in speech, adults may look to the mouth for additional cues to support processing when it is available. [ABSTRACT FROM AUTHOR]

Face to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was visible) and a pixelated condition (articulatory information was not visible). Further, task demands were manipulated by having listeners respond in a passive (no response) or an active (button press response) context. The active experiment required participants to discriminate between speech stimuli and was designed to mimic environmental situations which require one to use visual information to disambiguate the speaker’s message, simulating different listening conditions in real-world settings. Stimuli included a clear exemplar of the syllable /ba/ and a second exemplar in which the formant initial consonant was reduced creating an /a/−like consonant. Consistent with our hypothesis, results revealed that the greatest fixations to the mouth were present in the audiovisual active experiment and visual articulatory information led to a phonemic restoration effect for the /a/ speech token. In the pixelated condition, participants fixated on the eyes, and discrimination of the deviant token within the active experiment was significantly greater than the audiovisual condition. These results suggest that when required to disambiguate changes in speech, adults may look to the mouth for additional cues to support processing when it is available.

Audiovisual speech perception includes the simultaneous processing of auditory and visual speech. Deficits in audiovisual speech perception are reported in autistic individuals; however, less is known regarding audiovisual speech perception within the broader autism phenotype (BAP), which includes individuals with elevated, yet subclinical, levels of autistic traits. We investigate the neural indices of audiovisual speech perception in adults exhibiting a range of autism-like traits using event-related potentials (ERPs) in a phonemic restoration paradigm. In this paradigm, we consider conditions where speech articulators (mouth and jaw) are present (AV condition) and obscured by a pixelated mask (PX condition). These two face conditions were included in both passive (simply viewing a speaking face) and active (participants were required to press a button for a specific consonant–vowel stimulus) experiments. The results revealed an N100 ERP component which was present for all listening contexts and conditions; however, it was attenuated in the active AV condition where participants were able to view the speaker’s face, including the mouth and jaw. The P300 ERP component was present within the active experiment only, and significantly greater within the AV condition compared to the PX condition. This suggests increased neural effort for detecting deviant stimuli when visible articulation was present and visual influence on perception. Finally, the P300 response was negatively correlated with autism-like traits, suggesting that higher autistic traits were associated with generally smaller P300 responses in the active AV and PX conditions. The conclusions support the finding that atypical audiovisual processing may be characteristic of the BAP in adults.

Children with autism spectrum disorders have been reported to be less influenced by a speaker's face during speech perception than those with typically development. To more closely examine these reported differences, a novel visual phonemic restoration paradigm was used to assess neural signatures (event-related potentials [ERPs]) of audiovisual processing in typically developing children and in children with autism spectrum disorder. Video of a speaker saying the syllable /ba/ was paired with (1) a synthesized /ba/ or (2) a synthesized syllable derived from /ba/ in which auditory cues for the consonant were substantially weakened, thereby sounding more like /a/. The auditory stimuli are easily discriminable; however, in the context of a visual /ba/, the auditory /a/ is typically perceived as /ba/, producing a visual phonemic restoration. Only children with ASD showed a large /ba/-/a/ discrimination response in the presence of a speaker producing /ba/, suggesting reduced influence of visual speech. [ABSTRACT FROM AUTHOR]

Intelligibility of temporally degraded speech was investigated with locally time-reversed speech (LTR) and its interrupted version (ILTR). Control stimuli comprising interrupted speech (I) were also included. Speech stimuli consisted of 200 Japanese meaningful sentences. In interrupted stimuli, speech segments were alternated with either silent gaps or pink noise bursts. The noise bursts had a level of − 10, 0 or + 10 dB relative to the speech level. Segment duration varied from 20 to 160 ms for ILTR sentences, but was fixed at 160 ms for I sentences. At segment durations between 40 and 80 ms, severe reductions in intelligibility were observed for ILTR sentences, compared with LTR sentences. A substantial improvement in intelligibility (30–33%) was observed when 40-ms silent gaps in ILTR were replaced with 0- and + 10-dB noise. Noise with a level of − 10 dB had no effect on the intelligibility. These findings show that the combined effects of interruptions and temporal reversal of speech segments on intelligibility are greater than the sum of each individual effect. The results also support the idea that illusory continuity induced by high-level noise bursts improves the intelligibility of ILTR and I sentences. [ABSTRACT FROM AUTHOR]

Abstract Speech is perceived as a continuous stream of words despite consisting of a discontinuous, quasi-periodic signal of interleaved sounds and silences. Speech perception is surprisingly robust to interference by interruption, however speech that is replaced by gaps of silence is difficult to understand. When those silences are filled with noise, the speech is once again perceived as continuous even when the underlying speech sounds are removed completely. This is a phenomenon known as phonemic restoration. Perception of normal speech is accompanied by robust phase-locking of EEG signals to acoustic and linguistic features of speech. In this study we test the theory that interrupting speech with silence impairs perception by interfering with neural speech tracking. Further, we test the theory that we can restore perception and phase-tracking of the original acoustics by inserting noise in the interruptions. We find that disruptions of the acoustic envelope reduce the tracking of both acoustic and phonemic features. By inserting amplitude modulated noise such that the original broadband envelope is restored, we improved perception of the degraded speech and restored the magnitude of the speech tracking response; however, topographic analysis suggests that the neural response to noise-interrupted speech may recruit systematically different brain areas. The acoustic envelope seems to be an important physical component of speech that facilitates the dynamic neural mechanisms for perception of spoken language, particularly in adverse listening conditions. Highlights • Periodic interruptions disrupt perception and speech-tracking brain activity. • An intact acoustic envelope during interruptions restores speech-tracking. • Noise interruptions shift speech-tracking activity to different cortical areas. [ABSTRACT FROM AUTHOR]

In spoken language, audiovisual (AV) perception occurs when the visual modality influences encoding of acoustic features (e.g., phonetic representations) at the auditory cortex. We examined how visual speech (mouth movements) transforms phonetic representations, indexed by changes to the N1 auditory evoked potential (AEP). EEG was acquired while human subjects watched and listened to videos of a speaker uttering consonant vowel (CV) syllables, /ba/ and /wa/, presented in auditory-only or AV congruent or incongruent contexts or in a context in which the consonants were replaced by white noise (noise replaced). Subjects reported whether they heard "ba" or "wa." We hypothesized that the auditory N1 amplitude during illusory perception (caused by incongruent AV input, as in the McGurk illusion, or white noise-replaced consonants in CV utterances) should shift to reflect the auditory N1 characteristics of the phonemes conveyed visually (by mouth movements) as opposed to acoustically. Indeed, the N1 AEP became larger and occurred earlier when listeners experienced illusory "ba" (video /ba/, audio /wa/, heard as "ba") and vice versa when they experienced illusory "wa" (video /wa/, audio /ba/, heard as "wa"), mirroring the N1 AEP characteristics for /ba/ and /wa/ observed in natural acoustic situations (e.g., auditory-only setting). This visually mediated N1 behavior was also observed for noise-replaced CVs. Taken together, the findings suggest that information relayed by the visual modality modifies phonetic representations at the auditory cortex and that similar neural mechanisms support the McGurk illusion and visually mediated phonemic restoration. NEW & NOTEWORTHY Using a variant of the McGurk illusion experimental design (using the syllables /ba/ and /wa/), we demonstrate that lipreading influences phonetic encoding at the auditory cortex. We show that the N1 auditory evoked potential morphology shifts to resemble the N1 morphology of the syllable conveyed visually. We also show similar N1 shifts when the consonants are replaced by white noise, suggesting that the McGurk illusion and the visually mediated phonemic restoration rely on common mechanisms. [ABSTRACT FROM AUTHOR]

Visual information on a talker's face can influence what a listener hears. Commonly used approaches to study this include mismatched audiovisual stimuli (e.g., McGurk type stimuli) or visual speech in auditory noise. In this paper we discuss potential limitations of these approaches and introduce a novel visual phonemic restoration method. This method always presents the same visual stimulus (e.g., /ba/) dubbed with a matched auditory stimulus (/ba/) or one that has weakened consonantal information and sounds more /a/-like). When this reduced auditory stimulus (or /a/) is dubbed with the visual /ba/, a visual influence will result in effectively 'restoring' the weakened auditory cues so that the stimulus is perceived as a /ba/. An oddball design in which participants are asked to detect the /a/ among a stream of more frequently occurring /ba/s while either a speaking face or face with no visual speech was used. In addition, the same paradigm was presented for a second contrast in which participants detected /pa/ among /ba/s, a contrast which should be unaltered by the presence of visual speech. Behavioral and some ERP findings reflect the expected phonemic restoration for the /ba/ vs. /a/ contrast; specifically, we observed reduced accuracy and P300 response in the presence of visual speech. Further, we report an unexpected finding of reduced accuracy and P300 response for both speech contrasts in the presence of visual speech, suggesting overall modulation of the auditory signal in the presence of visual speech. Consistent with this, we observed a mismatch negativity (MMN) effect for the /ba/ vs. /pa/ contrast only that was larger in absence of visual speech. We discuss the potential utility for this paradigm for listeners who cannot respond actively, such as infants and individuals with developmental disabilities. [ABSTRACT FROM AUTHOR]

When noise obstructs portions of target sounds the auditory system fills in missing information, a phenomenon known as auditory restoration or induction. Previous work in animal models demonstrates that neurons in primary auditory cortex (A1) are capable of restoring occluded target signals suggesting that early auditory cortex is capable of inducing continuity in discontinuous signals (i.e., endogenous restoration). Current consensus is that the neural correlates of auditory induction and perceptual restoration emerge no earlier than A1. Moreover, the neural mechanisms supporting induction in humans are poorly understood. Here, we show that in human listeners, auditory brainstem nuclei support illusory auditory continuity well before engagement of cerebral cortex. We recorded brainstem responses to modulated target tones that did or did not promote illusory auditory percepts. Auditory continuity was manipulated by introducing masking noise or brief temporal interruptions in otherwise continuous tones. We found that auditory brainstem responses paralleled illusory continuity by tagging target sounds even when they were occluded by the auditory scene. Our results reveal (i) a pre-attentive, subcortical origin to a presumed cortical function and (ii) that brainstem signal processing helps partially cancel the negative effects of masking by restoring missing portions of auditory objects that are fragmented in the soundscape. [ABSTRACT FROM AUTHOR]

This study investigates how similarly present and absent English phonemes behind noise are perceived by native and non-native speakers. Participants were English native speakers and Japanese native speakers who spoke English as a second language. They listened to English words and non-words in which a phoneme was covered by noise (added; phoneme + noise) or replaced by noise (replaced; noise only). The target phoneme was either a nasal (/m/ and /n/) or a liquid (/l/ and /r/). In experiment, participants listened to a pair of a word (or non-word) with noise (added or replaced) and a word (or non-word) without noise (original) in a row, and evaluated the similarity of the two on an eight-point scale (8: very similar, 1: not similar). The results suggested that both native and non-native speakers perceived the 'added' phoneme more similar to the original sound than the 'replaced' phoneme to the original sound. In addition, both native and non-native speakers restored missing nasals more than missing liquids. In general, a replaced phoneme was better restored in words than non-words by native speakers, but equally restored by non-native speakers. It seems that bottom-up acoustic cues and top-down lexical cues are adopted differently in the phonemic restoration of native and non-native speakers. [ABSTRACT FROM AUTHOR]

In complex listening situations, cognitive restoration mechanisms are commonly used to enhance perception of degraded speech with inaudible segments. Profoundly hearing-impaired people with a cochlear implant (CI) show less benefit from such mechanisms. However, both normal hearing (NH) listeners and CI users do benefit from visual speech cues in these listening situations. In this study we investigated if an accompanying video of the speaker can enhance the intelligibility of interrupted sentences and the phonemic restoration benefit, measured by an increase in intelligibility when the silent intervals are filled with noise. Similar to previous studies, restoration benefit was observed with interrupted speech without spectral degradations (Experiment 1), but was absent in acoustic simulations of CIs (Experiment 2) and was present again in simulations of electric-acoustic stimulation (Experiment 3). In all experiments, the additional speech information provided by the complementary visual cues lead to overall higher intelligibility, however, these cues did not influence the occurrence or extent of the phonemic restoration benefit of filler noise. Results imply that visual cues do not show a synergistic effect with the filler noise, as adding them equally increased the intelligibility of interrupted sentences with or without the filler noise. [ABSTRACT FROM AUTHOR]

Speech reading enhances auditory perception in noise. One means by which this perceptual facilitation comes about is through information from visual networks reinforcing the encoding of the congruent speech signal by ignoring interfering acoustic signals. We tested this hypothesis neurophysiologically by acquiring EEG while individuals listened to words with a fixed portion of each word replaced by white noise. Congruent (meaningful) or incongruent (reversed frames) mouth movements accompanied the words. Individuals judged whether they heard the words as continuous (illusion) or interrupted (illusion failure) through the noise. We hypothesized that congruent, as opposed to incongruent, mouth movements should further enhance illusory perception by suppressing the auditory cortex’s response to interruption onsets and offsets. Indeed, we found that the N1 auditory evoked potential (AEP) to noise onsets and offsets was reduced when individuals experienced the illusion during congruent, but not incongruent, audiovisual streams. This N1 inhibitory effect was most prominent at noise offsets, suggesting that visual influences on auditory perception are instigated to a greater extent during noisy periods. These findings suggest that visual context due to speech-reading disengages (inhibits) neural processes associated with interfering sounds (e.g., noisy interruptions) during speech perception. [ABSTRACT FROM AUTHOR]

The comprehension of fluent speech in one's native language requires that listeners integrate the detailed acoustic-phonetic information available in the sound signal with linguistic knowledge. This interplay is especially apparent in the phoneme restoration effect, a phenomenon in which a missing phoneme is "restored" via the influence of top-down information from the lexicon and through bottom-up acoustic processing. Developmental dyslexia is a disorder characterized by an inability to read at the level of one's peers without any clear failure due to environmental influences. In the current study we utilized the phonemic restoration illusion paradigm to examine individual differences in phonemic restoration across a range of reading ability, from very good to dyslexic readers. Results demonstrate that restoration occurs less in those who have high scores on measures of phonological processing. Based on these results, we suggest that the processing or representation of acoustic detail may not be as reliable in poor and dyslexic readers, with the result that lexical information is more likely to override acoustic properties of the stimuli. This pattern of increased restoration could result from a failure of perceptual tuning, in which unstable representations of speech sounds result in the acceptance of non-speech sounds as speech. An additional or alternative theory is that degraded or impaired phonological processing at the speech sound level may reflect architecture that is overly plastic and consequently fails to stabilize appropriately for speech sound representations. Therefore, the inability to separate speech and noise may result as a deficit in separating noise from the acoustic signal. [ABSTRACT FROM AUTHOR]

Cognitive skills, such as processing speed, memory functioning, and the ability to divide attention, are known to diminish with aging. The present study shows that, despite these changes, older adults can successfully compensate for degradations in speech perception. Critically, the older participants of this study were not pre-selected for high performance on cognitive tasks, but only screened for normal hearing. We measured the compensation for speech degradation using phonemic restoration, where intelligibility of degraded speech is enhanced using top-down repair mechanisms. Linguistic knowledge, Gestalt principles of perception, and expectations based on situational and linguistic context are used to effectively fill in the inaudible masked speech portions. A positive compensation effect was previously observed only with young normal hearing people, but not with older hearing-impaired populations, leaving the question whether the lack of compensation was due to aging or due to age-related hearing problems. Older participants in the present study showed poorer intelligibility of degraded speech than the younger group, as expected from previous reports of aging effects. However, in conditions that induce top-down restoration, a robust compensation was observed. Speech perception by the older group was enhanced, and the enhancement effect was similar to that observed with the younger group. This effect was even stronger with slowed-down speech, which gives more time for cognitive processing. Based on previous research, the likely explanations for these observations are that older adults can overcome age-related cognitive deterioration by relying on linguistic skills and vocabulary that they have accumulated over their lifetime. Alternatively, or simultaneously, they may use different cerebral activation patterns or exert more mental effort. This positive finding on top-down restoration skills by the older individuals suggests that new cognitive training methods can teach older adults to effectively use compensatory mechanisms to cope with the complex listening environments of everyday life. [ABSTRACT FROM AUTHOR]

Abstract: In daily communication, we can usually still hear the spoken words as if they had not been masked and can comprehend the speech when spoken words are masked by background noise. This phenomenon is known as phonemic restoration. Since little is known about the neural mechanisms underlying phonemic restoration for speech comprehension, we aimed to identify the neural mechanisms using magnetoencephalography (MEG). Twelve healthy male volunteers with normal hearing participated in the study. Participants were requested to carefully listen to and understand recorded spoken Japanese stories, which were either played forward (forward condition) or in reverse (reverse condition), with their eyes closed. Several syllables of spoken words were replaced by 300-ms white-noise stimuli with an inter-stimulus interval of 1.6–20.3s. We compared MEG responses to white-noise stimuli during the forward condition with those during the reverse condition using time–frequency analyses. Increased 3–5Hz band power in the forward condition compared with the reverse condition was continuously observed in the left inferior frontal gyrus [Brodmann's areas (BAs) 45, 46, and 47] and decreased 18–22Hz band powers caused by white-noise stimuli were seen in the left transverse temporal gyrus (BA 42) and superior temporal gyrus (BA 22). These results suggest that the left inferior frontal gyrus and left transverse and superior temporal gyri are involved in phonemic restoration for speech comprehension. Our findings may help clarify the neural mechanisms of phonemic restoration as well as develop innovative treatment methods for individuals suffering from impaired speech comprehension, particularly in noisy environments. [Copyright &y& Elsevier]

The brain, using expectations, linguistic knowledge, and context, can perceptually restore inaudible portions of speech. Such top-down repair is thought to enhance speech intelligibility in noisy environments. Hearing-impaired listeners with cochlear implants commonly complain about not understanding speech in noise. We hypothesized that the degradations in the bottom-up speech signals due to the implant signal processing may have a negative effect on the top-down repair mechanisms, which could partially be responsible for this complaint. To test the hypothesis, phonemic restoration of interrupted sentences was measured with young normal-hearing listeners using a noise-band vocoder simulation of implant processing. Decreasing the spectral resolution (by reducing the number of vocoder processing channels from 32 to 4) systematically degraded the speech stimuli. Supporting the hypothesis, the size of the restoration benefit varied as a function of spectral resolution. A significant benefit was observed only at the highest spectral resolution of 32 channels. With eight channels, which resembles the resolution available to most implant users, there was no significant restoration effect. Combined electric-acoustic hearing has been previously shown to provide better intelligibility of speech in adverse listening environments. In a second configuration, combined electric-acoustic hearing was simulated by adding low-pass-filtered acoustic speech to the vocoder processing. There was a slight improvement in phonemic restoration compared to the first configuration; the restoration benefit was observed at spectral resolutions of both 16 and 32 channels. However, the restoration was not observed at lower spectral resolutions (four or eight channels). Overall, the findings imply that the degradations in the bottom-up signals alone (such as occurs in cochlear implants) may reduce the top-down restoration of speech. [ABSTRACT FROM AUTHOR]

Perception is a product of the interaction between bottom-up sensory processing and top-down higher order cognitive activity. For example, when the initial phoneme of a word is obliterated and replaced with noise, listeners hear it as intact provided there is semantic context. We modified this phonemic restoration paradigm by masking (not obliterating) the initial phoneme of a target word and presenting it within a carrier phrase which was informative (I), uninformative (U), or misinformative (M). Bias in favor of top-down context was measured as the extent to which M trials mislead listeners into reporting a target word other than that which was presented (relative to U trials that have irrelevant top-down semantic context). Forty-one participants (20 men) completed 600 test trials (300 delayed report of the phrase, 300 forced choice). Relative to the U condition, women were more affected by both the I and M cues than men, at certain levels of audibility during the forced choice condition. Moreover, the semantic strength of the I carrier phrases was correlated with the rate of correct reports of the target words in women but not in men. This suggests that women can be more affected by top-down semantic context than men. [ABSTRACT FROM PUBLISHER]

Phonemic restoration, a form of temporal induction, occurs when the human brain compensates for masked or missing portions of speech by filling in obscured or nonexistent sounds. We tested for temporal induction and related abilities in females of the Gray Treefrog Hyla versicolor. The number of pulses in calls is used by females for assessment of males. Accordingly, an ability to "restore" or interpolate between masked or otherwise sonically degraded portions of calls could help females during mate choice in noisy choruses. In phonotaxis experiments, we employed unmodified calls and those that had a centrally placed gap, a region overlapped by a portion of another call or filtered noise, or replaced with filtered noise. When offered call alternatives with equivalent numbers of clear pulses, we found that females discriminated against calls with gaps two or more times greater than the natural 25 ms interpulse interval. When a gap was replaced with a zone of call overlap or noise (so, again the call durations of the alternatives were unequal), females discriminated either in favour (overlap) of the modified stimuli or failed to discriminate (noise). However, when the unmodified and modified stimuli were the same duration, females discriminated against the latter. Normal calls were also chosen when paired against calls with multiple noise sections. Pulses formed from noise bursts were attractive, but less so than normal pulses. In single speaker tests, standardized rates of movement did not differ between calls containing noise segments of different duration. Our results therefore do not indicate that females of the Gray Treefrog employ a form of temporal induction that is fully restorative. However, the data indicate that acoustically anomalous sections of calls can retain attractive potential provided acoustic energy and pulses are present. [ABSTRACT FROM AUTHOR]

The human auditory system perceptually restores short deleted segments of speech and other sounds (e.g. tones) when the resulting silent gaps are filled by a potential masking noise. When this phenomenon, known as ‘auditory induction’, occurs, listeners experience the illusion of hearing an ongoing sound continuing through the interrupting noise even though the perceived sound is not physically present. Such illusions suggest that a key function of the auditory system is to allow listeners to perceive complete auditory objects with incomplete acoustic information, as may often be the case in multisource acoustic environments. At present, however, we know little about the possible functions of auditory induction in the sound-mediated behaviours of animals. The present study used two-choice phonotaxis experiments to test the hypothesis that female grey treefrogs, Hyla chrysoscelis, experience the illusory perceptual restoration of discrete pulses in the male advertisement call when pulses are deleted and replaced by a potential masking noise. While added noise restored some attractiveness to calls with missing pulses, there was little evidence to suggest that the frogs actually experienced the illusion of perceiving the missing pulses. Instead, the added noise appeared to function as an acoustic appendage that made some calls more attractive than others as a result of sensory biases, the expression of which depended on the temporal order and acoustic structure of the added appendages. [Copyright &y& Elsevier]

Abstract: The auditory system is capable of perceptually restoring inaudible portions of speech. This restoration may be compromised as a result of hearing impairment, particularly if it is combined with advanced age, because of degradations in the bottom-up and top-down processes. To test this hypothesis, phonemic restoration was quantitatively measured with hearing-impaired listeners of varying ages and degrees of hearing impairment, as well as with a normal hearing control group. The results showed that the restoration benefit was negatively correlated with both hearing impairment and age, supporting the original hypothesis. Group data showed that listeners with mild hearing loss were able to perceptually restore the missing speech segments as well as listeners with normal hearing. By contrast, the moderately-impaired listeners showed no evidence of perceptual restoration. Further analysis using the articulation index showed that listeners with mild hearing loss were able to increase phonemic restoration with audibility. Moderately-impaired listeners, on the other hand, were unable to do so, even when the articulation index was high. The overall findings suggest that, in addition to insufficient audibility, degradations in the bottom-up and/or top-down mechanisms as a result of hearing loss may limit or entirely prevent phonemic restoration. [Copyright &y& Elsevier]

Abstract: The brain uses context and prior knowledge to repair degraded sensory inputs and improve perception. For example, listeners hear speech continuing uninterrupted through brief noises, even if the speech signal is artificially removed from the noisy epochs. In a functional MRI study, we show that this temporal filling-in process is based on two dissociable neural mechanisms: the subjective experience of illusory continuity, and the sensory repair mechanisms that support it. Areas mediating illusory continuity include the left posterior angular gyrus (AG) and superior temporal sulcus (STS) and the right STS. Unconscious sensory repair occurs in Broca''s area, bilateral anterior insula, and pre-supplementary motor area. The left AG/STS and all the repair regions show evidence for word-level template matching and communicate more when fewer acoustic cues are available. These results support a two-path process where the brain creates coherent perceptual objects by applying prior knowledge and filling-in corrupted sensory information. [Copyright &y& Elsevier]

Abstract: When a particular speech sound is obliterated and replaced by a non-speech sound in continuous speech the listener may not notice any disturbance in speech or have difficulties in understanding the word. The present study examined for the first time neurophysiological correlates of the perception of words with an obliterated initial phoneme. Behavioral responses and event-related potentials (ERPs) were measured while participants listened to naturally spoken sentences which had a highly or less expected final word. Half of the sentences were manipulated to have a cough replacing the beginning of the final word, thus, reducing the initial phonetic information available for the word recognition. An N1-P2 complex indicated an automatic registration of the cough''s onset. An early negativity to less expected relative to highly expected words was observed for phonetically intact words but not for manipulated words. Although the N400 effect to manipulated words was elicited later than to intact words, after the fragment onset, its amplitude was not enhanced. Further, no significant enhancement of the N400 was found for the manipulated highly expected words. This finding, together with behavioral results, indicated an easier integration of the manipulated highly expected words with the sentence context than of intact but less expected words. Taken together, the study demonstrates an efficient usage of both a context-driven expectancy of the suitable word as well as a stimulus-driven processing of the phonetic information during online perception of speech. The present ERP results support the earlier behavioral research in showing that phonemic restoration is not a bottom-up phenomenon but rather reflects a top-down repair process. [Copyright &y& Elsevier]

Abstract: Perception of a phoneme may occur even when the speech sound is missing (e.g., when an extraneous noise replaces the sound). This phenomenon, called phonemic restoration, has been observed to depend on the type of distortion. It requires a replacing sound that provides acoustic input to the auditory system, since the restoration has not been found when a speech sound was replaced by silence. We examined the brain activation underlying speech processing when the word''s initial phoneme was completely replaced by a silent gap. Event-related potentials (ERPs) and reaction times (RTs) were measured as indicators of semantic processing of sentence final words. Slower voice onset times during repetition of the manipulated words as compared to normal words indicated increased difficulty in retrieving their meaning. The N400, which is related to the increased demands of the semantic integration of words, was elicited by less expected words as compared to highly expected ones. For manipulated words, the N400 was elicited at the same latency than for normal words, with respect to the onset of the remaining word fragment. The amplitude of the N400 was not increased, nor did it last longer, thereby indicating successful retrieval of the word''s meaning based on the semantic context and remaining phonetic information. Thus, semantic retrieval does not seem to require the word''s initial phoneme to be present in a sentence context. The results suggest that both context-driven expectancy (top–down) and stimulus-driven processes (bottom–up) are utilized in word processing and contribute to the overall N400 response. [Copyright &y& Elsevier]

Phonemic restoration is the perceptual synthesis of phonemes when masked by appropriate replacement sounds by utilizing linguistic context. Current models attempting to accomplish acoustic restoration of phonemes, however, use only temporal continuity and produce poor restoration of unvoiced phonemes, and are also limited in their ability to restore voiced phonemes. We present a schema-based model for phonemic restoration. The model employs a missing data speech recognition system to decode speech based on intact portions and activates word templates corresponding to the words containing the masked phonemes. An activated template is dynamically time warped to the noisy word and is then used to restore the speech frames corresponding to the masked phoneme, thereby synthesizing it. The model is able to restore both voiced and unvoiced phonemes with a high degree of naturalness. Systematic testing shows that this model outperforms a Kalman-filter based model. [ABSTRACT FROM AUTHOR]

Listeners cannot recognize highly reduced word forms in isolation, but they can do so when these forms are presented in context (Ernestus, Baayen, & Schreuder, 2002). This suggests that not all possible surface forms of words have equal status in the mental lexicon. The present study shows that the reduced forms are linked to the canonical representations in the mental lexicon, and that these latter representations induce reconstruction processes. Listeners restore suffixes that are partly or completely missing in reduced word forms. A series of phoneme-monitoring experiments reveals the nature of this restoration: the basis for suffix restoration is mainly phonological in nature, but orthography has an influence as well. [ABSTRACT FROM AUTHOR]

Abstract: To investigate mechanisms for perceiving the duration of an auditory event, an effect of perceptual grouping upon perceived duration was studied psychophysically. In the first experiment, the perceived duration of a spoken word was measured under three conditions of acoustic continuity (i.e., (a) intact, (b) noise-replaced, and (c) gap-replaced) as a function of the duration of the target stimulus. Under the noise-replaced condition, a portion of the target stimulus was physically replaced with a noise burst. Under the gap-replaced condition, the replacement was made with a gap. The gap-replacement resulted in a prominent shrinkage of the perceived duration. In the case of noise-replacement, the amount of shrinkage was moderate but highly significant, although the word employed was perceived to be phonetically intact. Independent of this effect of replacement, the amount of shrinkage was also affected by the physical duration of the target stimulus. The second experiment tested an effect of noise replacement on the perceived duration of a tone burst. In this case, the noise replacement also shrunk the perceived duration of the non-speech stimulus. This noise-induced shrinkage could be regarded as being general for the auditory duration. The phenomenon is discussed in relation to a revised model for perceived duration. [ABSTRACT FROM AUTHOR]

Phonemic restoration-perceiving speech sounds that are actually missing-is a fundamental perceptual process that 'repairs' interrupted spoken messages during noisy everyday listening. As a dynamic, integrative process, phonemic restoration is potentially affected by neurodegenerative pathologies, but this has not been clarified. Here, we studied this phenomenon in 5 patients with typical Alzheimer's disease and 4 patients with semantic dementia, relative to 22 age-matched healthy controls. Participants heard isolated sounds, spoken real words and pseudowords in which noise bursts either overlaid a consonant or replaced it; a tendency to hear replaced (missing) speech sounds as present signified phonemic restoration. All groups perceived isolated noises normally and showed phonemic restoration of real words, most marked in Alzheimer's patients. For pseudowords, healthy controls showed no phonemic restoration, while Alzheimer's patients showed marked suppression of phonemic restoration and patients with semantic dementia contrastingly showed phonemic restoration comparable to real words. Our findings provide the first evidence that phonemic restoration is preserved or even enhanced in neurodegenerative diseases, with distinct syndromic profiles that may reflect the relative integrity of bottom-up phonological representation and top-down lexical disambiguation mechanisms in different diseases. This work has theoretical implications for predictive coding models of language and neurodegenerative disease and for understanding cognitive 'repair' processes in dementia. Future research should expand on these preliminary observations with larger cohorts., (© The Author(s) 2022. Published by Oxford University Press on behalf of the Guarantors of Brain.)

Focuses on auditory and visual influences on phonemic restoration using a psychophysical paradigm. Examination of the influence of a higher-level, phonotactic constraint; Prediction of an interactive-activation model; Restoration of voiceless stop consonants, voiced stops and alveolar stops.