Decoding the Puzzle of Perceptual Restoration: The Role of Higher Order Cognitive and Linguistic Processing and Cognitive Effort during Speech Repair

Across three separate investigations, this dissertation documents work examining the role of higher-order processes and cognitive effort during the top-down compensatory repair of degraded speech, specifically perceptual restoration. Perceptual restoration is a phenomenon where a listener can perceptually restore or repair speech "missing" from an auditory signal. Because perceptual restoration improves speech intelligibility, this compensatory ability may allow some listeners to function better in challenging listening environments, such as a noisy restaurant. Prior investigations into perceptual restoration have found that there is considerable variability across individuals with regard to their ability to restore missing speech information. This work proposes that higher-order domain-general cognitive and domain-specific linguistic abilities may characterize unexplained variability during perceptual restoration tasks. This dissertation also characterizes cognitive resources devoted to task-related demands during perceptual restoration tasks, both during and after sentence listening. Understanding the role that cognitive effort plays in speech intelligibility and perceptual repair is important because two listeners can exert very different levels of effort at different times, even after listening has ended, while obtaining the same intelligibility score, highlighting aspects of speech recognition that are sometimes overlooked. This work employed speech interrupted by a gap of silence or a burst of noise to force listeners into perceptual restoration and repair. Because listeners with moderate or poorer hearing loss struggle with perceptual restoration tasks, these investigations recruited participants with normal hearing to mild sensorineural hearing loss. By asking listeners with a (near) normal auditory system to process a signal with only 50% of the original speech remaining reduces the impact of the peripheral auditory system and stresses higher-order reprocessing of speech information. The first investigation assessed a range of cognitive abilities as well as a participant's lexical knowledge and lexical access speed. Next, these abilities were related to a participant's ability to restore missing speech information across conditions that varied with regard to sentence predictability, interruption type, and linguistic complexity. This investigation found that a listener's complex working memory and lexical knowledge acted as the best predictor for perceptual restoration ability, followed next by inhibitory control and processing speed. These data suggest that domain-general cognitive processes such as working memory provide an avenue through which domain-specific linguistic mechanisms can repair and restore missing speech. Consistent with existing literature, listeners performed better when sentences were periodically interrupted by bursts of noise rather than gaps of silence, and performance was better for predictable/high context sentences than those where the last word was semantically unpredictable or not related to sentence context. The second investigation used well-validated neurocognitive measures that captured a participant's linguistic knowledge in greater detail. This study also investigated the role of other domain-general cognitive abilities, such as inhibitory control, processing speed, executive functioning, and problem-solving ability in addition to lexical knowledge and lexical access speed. Consistent with the results of the first investigation, complex working memory supported perceptual restoration regardless of condition, suggesting an important role of complex working memory regardless of interruption type, promoting the opportunity for speech recovery using linguistic processes. Notably, lexical knowledge and lexical access speed were important when speech was periodically interrupted by bursts of noise. This is consistent with data indicating that participants perceive the missing speech information as being continuous behind the noise burst, which may permit additional lexical activation and processing. While other domain general-cognitive abilities were not directly related to perceptual restoration, they were significantly correlated with complex working memory, suggesting a hierarchical relationship between these abilities and the role of more advanced cognitive processes. Finally, a third investigation characterized the level of cognitive effort required during perceptual restoration tasks using pupillometry, both during and after sentence listening. Cognitive effort during sentence listening has often been referred to as "listening effort" and reflects the time-course of effort devotion during the listening window. After the sentence has ended, ongoing cognitive effort may be devoted during a relatively short period of time known as the retention interval, which is thought to be when ongoing reprocessing and repair may occur. This investigation found that during the listening window, both interruption conditions resulted in additional pupil dilation, reflecting devotion of task-related cognitive resources compared to uninterrupted sentences. For a brief time near the end of the listening window however, the pupillary response was greatest for the noise burst interrupted sentences, which suggests additional processing occurring while the sentence is still playing. On the other hand, pupil dilation was greatest for the silent gap sentences near the end of the retention interval, which suggests sustained cognitive effort in attempting to resolve lexical or syntactic ambiguity. Taken together, these investigations offer valuable insight into the perceptual restoration process. These data provide clinicians and scientists with information regarding the cognitive and linguistic abilities that are key during perceptual restoration. This may allow for the development of auditory rehabilitation programs that emphasize perceptual restoration and cognitive training or the development of prediction algorithms for hearing aids or cochlear implants aimed at reducing cognitive load under certain listening conditions. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]