Search

In response to the shortage of ventilators caused by the COVID-19 pandemic, many organizations have designed low-cost emergency ventilators. Many of these devices are pressure-cycled pneumatic ventilators, which are easy to produce but often do not include the sensing or alarm features found on commercial ventilators. This work reports a low-cost, easy-to-produce electronic sensor and alarm system for pressure-cycled ventilators that estimates clinically useful metrics such as pressure and respiratory rate and sounds an alarm when the ventilator malfunctions. A low-complexity signal processing algorithm uses a pair of nonlinear recursive envelope trackers to monitor the signal from an electronic pressure sensor connected to the patient airway. The algorithm, inspired by those used in hearing aids, requires little memory and performs only a few calculations on each sample so that it can run on nearly any microcontroller., Comment: Open-source hardware and software: https://rapidalarm.github.io/

Dynamic range compression (DRC), a form of nonlinear gain control, is widely used in both hearing devices and music production, but in very different ways. In hearing devices, DRC boosts quiet sounds more than loud sounds to match the reduced dynamic range of listeners with hearing loss. Compression is applied to the mixture of sounds captured by the hearing device microphone, so it is known to cause distortion in noisy environments where the level of one sound can affect the gain applied to another sound. In music mixing, DRC can be applied to individual vocal or instrumental tracks or to the overall mixture. It can also be applied to combinations of signals to introduce deliberate distortion effects, for example, by adjusting the level of one track based on the level of another. In a hearing device that could process different sound sources independently, real-life sounds could be mixed like musical tracks, reducing unwanted distortion and enabling new nonlinear techniques to improve intelligibility. This talk will explore multiple-source DRC architectures that are commonly used for music but could also be advantageous for hearing devices.

Modern listening devices are equipped with air-conducted microphones and contact microphones. External microphones, like those on a listening device, have been used to estimate relative transfer functions (RTFs) at microphone arrays. With numerous active sound sources, the air-conducted microphones perform poorly while the contact microphones are robust to external noise. A drawback of contact microphones is that they are bandlimited. Past work has shown that the contact microphone and microphone array can be combined to estimate RTFs in the low frequencies. To overcome the limitations of the contact microphone, we propose a method that leverages the full-band signal at the microphone array to provide beamforming gains at higher frequencies. We demonstrate this method by separating three human talkers in a noisy environment.

Quantization noise shaping, also known as sigma-delta modulation, is widely used in analog-to-digital converters. Such converters oversample a signal in time, quantize it to one or a few bits of precision, and then use analog feedback to shape quantization noise outside the band of interest, producing a high-precision output. Similarly, if a sensor array is spatially oversampled so that its elements are much less than one-half wavelength apart, then quantization noise can be propagated between channels to shape its spatial distribution. A beamformer acts as a spatial filter, removing shaped quantization noise to produce a high-precision output even with coarsely quantized sensors. Spatial sigma-delta modulation has been explored primarily for radio-frequency applications using linear arrays. In this presentation, we apply spatial noise shaping to dense acoustic sensor arrays, including circular and spherical arrays. For immersive audio applications, we demonstrate how noise-shaping feedback can be used to push quantization noise into higher-order Ambisonic modes.

Contact microphones are accelerometers placed on the skin to detect body-conducted speech, which is inherently robust to interfering sound sources. However, contact microphones are severely bandlimited. In contrast, standard air-conduction microphones capture all relevant frequencies from all sources. To partially recover lost frequency components of speech, we can use bandwidth extension, which uses formant information at low frequencies to reconstruct the spectral envelope and harmonic structure. We use a suite of bandwidth extension methods to combine high SNR, low bandwidth audio with low SNR, full bandwidth audio for speech enhancement.

Recording on moving devices leads to constant deformations in microphone array structure, which interferes with spatial audio processing. Previously, we have established that there is a locally linear mapping between the manifolds of relative transfer functions (RTF) and array geometries. This mapping also applies to trajectories of array deformations and their corresponding RTFs. We run segmented least squares (SLS) on the RTF trajectories to discern potential states of motion in an unsupervised manner. We then examine an implementation of SLS for adaptive beamforming in a deformable setting.

Acoustic head simulators are used to collect realistic binaural audio with accurate interaural cues. Head simulators are difficult to acquire and often provide only listening and recording capabilities. For cocktail parties and conversation scenarios, it is necessary to have many head simulators that can simultaneously produce speech and capture sound. An added dimension of realism can be provided with motion. We rely on 3D printing to fabricate a large number of head simulators, each of which is equipped with a loudspeaker for mimicking speech production. We use silent multi-axis turrets to provide neck-like motion. Our actuated head simulator collection is integrated into the Mechatronic Acoustic Research System.

To support spatial audio research, we aim to take recordings from complex acoustic environments with moving sources and microphones, however we observe a lack of research tools that can accomplish this. Past approaches recorded people engaging in various tasks, which produces rich data that unfortunately lacks repeatability. We propose using robots to recreate dynamic scenes without the inherent variability of human motion. To be useful, this Mechatronic Acoustic Research System must be remotely accessible, offer concise representations of dynamic scenes, support a variety of robot and audio devices, and synchronize robot motion. In this talk, we show how we solved these challenges. Remote experimentation is facilitated by our virtual interface, which uses a simple GUI to describe robot motion and audio playback/recording. A digital twin physical simulation is used for visualization and validation of motion paths. We propose using the Robot Operating System for multi-robot coordination so that networked robots can be incorporated with little overhead. We use MARS to run experiments where a cable-driven parallel robot moves a loudspeaker along a 3D path while being recorded from distributed Matrix Voice microphone arrays. We evaluate the measured audio to show repeatability of the system, justifying its use in research.

Hearing aids and other listening devices perform poorly in noisy, reverberant venues like restaurants and conference centers with numerous active sound sources. Microphone arrays can use array processing techniques like beamforming to isolate talkers from a specific region in the room while attenuating undesired sound sources. However, beamforming often removes spatial cues and is typically restricted to isolating a single talker at a time. Previous work has shown the effectiveness of remote microphones worn by talkers and adapting the signal at the earpiece to improve the intelligibility of group conversations. Due to the increase in hybrid meetings and classrooms, many spaces are equipped with high throughput, low latency devices including large microphone arrays. In this work, we present a system that aggregates information collected by microphone arrays distributed in a room to enhance the intelligibility of talkers in a group conversation. The beamformed signal from the microphone arrays is adapted to match the magnitude and phase of the earpiece microphones. The filters are continuously updated in order to track motion of both the listeners and talkers.

Large-scale distributed arrays can obtain high spatial resolution, but they typically rely on a rigid array structure. If we want to form distributed arrays from mobile and wearable devices, our models need to account for motion. The motion of multiple microphones worn by humans can be difficult to track, but through manifold techniques we can learn the movement through its acoustic response. We show that the mapping between the array geometry and its acoustic response is locally linear and can be exploited in a semi-supervised manner for a given acoustic environment. We will also investigate generative modelling of microphone positions based on their acoustic response to both synthetic and recorded data. Prior work has shown a similar locally linear mapping between source locations and their spatial cues, and we will attempt to combine these findings with our own to develop a localization model suitable for dynamic array geometries.

Hearing aids use dynamic range compression (DRC), a form of automatic gain control, to make quiet sounds louder and loud sounds quieter. Compression can improve listening comfort, but it can also cause unwanted distortion in noisy environments. It has been widely reported that DRC performs poorly in noise, but there has been little mathematical analysis of these noise-induced distortion effects. This work introduces a mathematical model to study the behavior of DRC in noise. By making simplifying assumptions about the signal envelopes, we define an effective compression function that models the compression applied to one signal in the presence of another. Using the properties of concave functions, we prove results about DRC that have been previously observed experimentally: that the effective compression applied to each sound in a mixture is weaker than it would have been for the signal alone; that uncorrelated signal envelopes become negatively correlated when compressed as a mixture; and that compression can reduce the long-term signal-to-noise ratio in certain conditions. These theoretical results are supported by software experiments using recorded speech signals.

Face masks muffle speech and make communication more difficult, especially for people with hearing loss. This study examines the acoustic attenuation caused by different face masks, including medical, cloth, and transparent masks, using a head-shaped loudspeaker and a live human talker. The results suggest that all masks attenuate frequencies above 1 kHz, that attenuation is greatest in front of the talker, and that there is substantial variation between mask types, especially cloth masks with different materials and weaves. Transparent masks have poor acoustic performance compared to both medical and cloth masks. Most masks have little effect on lapel microphones, suggesting that existing sound reinforcement and assistive listening systems may be effective for verbal communication with masks.

Augmented listening devices, such as hearing aids and augmented reality headsets, enhance human perception by changing the sounds that we hear. Microphone arrays can improve the performance of listening systems in noisy environments, but most array-based listening systems are designed to isolate a single sound source from a mixture. This work considers a source-remixing filter that alters the relative level of each source independently. Remixing rather than separating sounds can help to improve perceptual transparency: it causes less distortion to the signal spectrum and especially to the interaural cues that humans use to localize sounds in space., To appear at ICASSP 2020

Many public venues, such as theaters and houses of worship, use assistive listening systems that broadcast a copy of the sound being played over a public address system to the listening devices of patrons with hearing loss. While these wireless systems are generally intended to amplify sound for listening device users, they can also be used to cancel nuisance sounds, such as loud background music that makes conversation difficult. In this work, we demonstrate a binaural listening device that uses an assistive listening system transmitter and receiver to attenuate music played through loudspeakers while preserving speech sounds from nearby talkers. A set of adaptive filters tracks the acoustic transfer functions from the loudspeakers to the earpiece microphones, predicts the music signal received by each earpiece, and subtracts that predicted signal from the mixture signal of the corresponding earpiece. The binaural adaptive system tracks changes in the unwanted music signal as the user moves while preserving the acoustic and spatial cues of the desired speech sounds. [Work supported by the Intelligence Community Postdoctoral Research Fellowship Program, National Science Foundation, and Discovery Partners Institute.]