Search

The present study evaluated auditory-visual speech perception in cochlear-implant users as well as normal-hearing and simulated-implant controls to delineate relative contributions of sensory experience and cues. Auditory-only, visual-only, or auditory-visual speech perception was examined in the context of categorical perception, in which an animated face mouthing ba, da, or ga was paired with synthesized phonemes from an 11-token auditory continuum. A three-alternative, forced-choice method was used to yield percent identification scores. Normal-hearing listeners showed sharp phoneme boundaries and strong reliance on the auditory cue, whereas actual and simulated implant listeners showed much weaker categorical perception but stronger dependence on the visual cue. The implant users were able to integrate both congruent and incongruent acoustic and optical cues to derive relatively weak but significant auditory-visual integration. This auditory-visual integration was correlated with the duration of the implant experience but not the duration of deafness. Compared with the actual implant performance, acoustic simulations of the cochlear implant could predict the auditory-only performance but not the auditory-visual integration. These results suggest that both altered sensory experience and improvised acoustic cues contribute to the auditory-visual speech perception in cochlear-implant users.

Speech perception in the presence of another competing voice is one of the most challenging tasks for cochlear implant users. Several studies have shown that (1) the fundamental frequency (F0) is a useful cue for segregating competing speech sounds and (2) the F0 is better represented by the temporal fine structure than by the temporal envelope. However, current cochlear implant speech processing algorithms emphasize temporal envelope information and discard the temporal fine structure. In this study, speech recognition was measured as a function of the F0 separation of the target and competing sentence in normal-hearing and cochlear implant listeners. For the normal-hearing listeners, the combined sentences were processed through either a standard implant simulation or a new algorithm which additionally extracts a slowed-down version of the temporal fine structure (called Frequency-Amplitude-Modulation-Encoding). The results showed no benefit of increasing F0 separation for the cochlear implant or simulation groups. In contrast, the new algorithm resulted in gradual improvements with increasing F0 separation, similar to that found with unprocessed sentences. These results emphasize the importance of temporal fine structure for speech perception and demonstrate a potential remedy for difficulty in the perceptual segregation of competing speech sounds.

A potential shortcoming of existing multichannel cochlear implants is electrical-field summation during simultaneous electrode stimulation. Electrical-field interactions can disrupt the stimulus waveform prior to neural activation. To test whether speech intelligibility can be degraded by electrical-field interaction, speech recognition performance and interaction were examined for three Clarion electrode arrays: the pre-curved, enhanced bipolar electrode array, the enhanced bipolar electrode with an electrode positioner, and the Hi-Focus electrode with a positioner. Channel interaction was measured by comparing stimulus detection thresholds for a probe signal in the presence of a sub-threshold perturbation signal as a function of the separation between the two simultaneously stimulated electrodes. Correct identification of vowels, consonants, and words in sentences was measured with two speech strategies: one which used simultaneous stimulation and another which used sequential stimulation. Speech recognition scores were correlated with measured electrical-field interaction for the strategy which used simultaneous stimulation but not the strategy which used sequential stimulation. Higher speech recognition scores with the simultaneous strategy were generally associated with lower levels of electrical-field interaction. Electrical-field interaction accounted for as much as 70% of the variance in speech recognition scores, suggesting that electrical-field interaction is a significant contributor to the variability found across patients who use simultaneous strategies.

Cochlear implants allow most patients with profound deafness to successfully communicate under optimal listening conditions. However, the amplitude modulation (AM) information provided by most implants is not sufficient for speech recognition in realistic settings where noise is typically present. This study added slowly varying frequency modulation (FM) to the existing algorithm of an implant simulation and used competing sentences to evaluate FM contributions to speech recognition in noise. Potential FM advantage was evaluated as a function of the number of spectral bands, FM depth, FM rate, and FM band distribution. Barring floor and ceiling effects, significant improvement was observed for all bands from 1 to 32 with the additional FM cue both in quiet and noise. Performance also improved with greater FM depth and rate, which might reflect resolved sidebands under the FM condition. Having FM present in low-frequency bands was more beneficial than in high-frequency bands, and only half of the bands required the presence of FM, regardless of position, to achieve performance similar to when all bands had the FM cue. These results provide insight into the relative contributions of AM and FM to speech communication and the potential advantage of incorporating FM for cochlear implant signal processing.

Speech recognition in noise and music perception is especially challenging for current cochlear implant users. The present study utilizes the residual acoustic hearing in the nonimplanted ear in five cochlear implant users to elucidate the role of temporal fine structure at low frequencies in auditory perception and to test the hypothesis that combined acoustic and electric hearing produces better performance than either mode alone. The first experiment measured speech recognition in the presence of competing noise. It was found that, although the residual low-frequency (<1000 Hz) acoustic hearing produced essentially no recognition for speech recognition in noise, it significantly enhanced performance when combined with the electric hearing. The second experiment measured melody recognition in the same group of subjects and found that, contrary to the speech recognition result, the low-frequency acoustic hearing produced significantly better performance than the electric hearing. It is hypothesized that listeners with combined acoustic and electric hearing might use the correlation between the salient pitch in low-frequency acoustic hearing and the weak pitch in the envelope to enhance segregation between signal and noise. The present study suggests the importance and urgency of accurately encoding the fine-structure cue in cochlear implants.

Amplitude modulation (AM) and frequency modulation (FM) are commonly used in communication, but their relative contributions to speech recognition have not been fully explored. To bridge this gap, we derived slowly varying AM and FM from speech sounds and conducted listening tests using stimuli with different modulations in normal-hearing and cochlear-implant subjects. We found that although AM from a limited number of spectral bands may be sufficient for speech recognition in quiet, FM significantly enhances speech recognition in noise, as well as speaker and tone recognition. Additional speech reception threshold measures revealed that FM is particularly critical for speech recognition with a competing voice and is independent of spectral resolution and similarity. These results suggest that AM and FM provide independent yet complementary contributions to support robust speech recognition under realistic listening situations. Encoding FM may improve auditory scene analysis, cochlear-implant, and audiocoding performance.

Different from traditional Fourier analysis, a signal can be decomposed into amplitude and frequency modulation components. The speech processing strategy in most modern cochlear implants only extracts and encodes amplitude modulation in a limited number of frequency bands. While amplitude modulation encoding has allowed cochlear implant users to achieve good speech recognition in quiet, their performance in noise is severely compromised. Here, we propose a novel speech processing strategy that encodes both amplitude and frequency modulations in order to improve cochlear implant performance in noise. By removing the center frequency from the subband signals and additionally limiting the frequency modulation's range and rate, the present strategy transforms the fast-varying temporal fine structure into a slowly varying frequency modulation signal. As a first step, we evaluated the potential contribution of additional frequency modulation to speech recognition in noise via acoustic simulations of the cochlear implant. We found that while amplitude modulation from a limited number of spectral bands is sufficient to support speech recognition in quiet, frequency modulation is needed to support speech recognition in noise. In particular, improvement by as much as 71 percentage points was observed for sentence recognition in the presence of a competing voice. The present result strongly suggests that frequency modulation be extracted and encoded to improve cochlear implant performance in realistic listening situations. We have proposed several implementation methods to stimulate further investigation. Index Terms-Amplitude modulation, cochlear implant, fine structure, frequency modulation, signal processing, speech recognition, temporal envelope.

Speech recognition performance was measured in normal-hearing and cochlear-implant listeners with maskers consisting of either steady-state speech-spectrum-shaped noise or a competing sentence. Target sentences from a male talker were presented in the presence of one of three competing talkers (same male, different male, or female) or speech-spectrum-shaped noise generated from this talker at several target-to-masker ratios. For the normal-hearing listeners, target-masker combinations were processed through a noise-excited vocoder designed to simulate a cochlear implant. With unprocessed stimuli, a normal-hearing control group maintained high levels of intelligibility down to target-to-masker ratios as low as 0 dB and showed a release from masking, producing better performance with single-talker maskers than with steady-state noise. In contrast, no masking release was observed in either implant or normal-hearing subjects listening through an implant simulation. The performance of the simulation and implant groups did not improve when the single-talker masker was a different talker compared to the same talker as the target speech, as was found in the normal-hearing control. These results are interpreted as evidence for a significant role of informational masking and modulation interference in cochlear implant speech recognition with fluctuating maskers. This informational masking may originate from increased target-masker similarity when spectral resolution is reduced.

Objective: To evaluate the performance of the various speech processing strategies supported by the Clarion S-Series implant processor., Design: Five different speech-processing strategies [the Continuous Interleaved Sampler (CIS), the Simultaneous Analog Stimulation (SAS), the Paired Pulsatile Sampler (PPS), the Quadruple Pulsatile Sampler (QPS) and the hybrid (HYB) strategies] were implemented on the Clarion Research Interface platform. These speech-processing strategies varied in the degree of electrode simultaneity, with the SAS strategy being fully simultaneous (all electrodes are stimulated at the same time), the PPS and QPS strategies being partially simultaneous and the CIS strategy being completely sequential. In the hybrid strategy, some electrodes were stimulated using SAS, and some were stimulated using CIS. Nine Clarion CIS users were fitted with the above speech processing strategies and tested on vowel, consonant and word recognition in quiet., Results: There were no statistically significant differences in the mean group performance between the CIS and SAS strategies on vowel and sentence recognition. A statistically significant difference was found only on consonant recognition. Individual results, however, indicated that most subjects performed worse with the SAS strategy compared with the CIS strategy on all tests. About 33% of the cochlear implant users benefited from the PPS and QPS strategies on consonant and word recognition., Conclusions: If temporal information were the primary factor in speech recognition with cochlear implants then SAS should consistently produce higher speech recognition scores than CIS. That was not the case, however, because most CIS users performed significantly worse with the SAS strategy on all speech tests. Hence, there seems to be a trade-off between improving the temporal resolution with an increasing number of simultaneous channels and introducing distortions from electrical-field interactions. Performance for some CI users improved when the number of simultaneous channels increased to two (PPS strategy) and four (QPS strategy). The improvement with the PPS and QPS strategies must be due to the higher rates of stimulation. The above results suggest that CIS users are less likely to benefit with the SAS strategy, and they are more likely to benefit from the PPS and QPS strategies, which provide higher rates of stimulation with small probability of channel interaction.