Search

In this paper, we build a speech privacy attack that exploits speech reverberations from a smartphone's inbuilt loudspeaker captured via a zero-permission motion sensor (accelerometer). We design our attack Spearphone, and demonstrate that speech reverberations from inbuilt loudspeakers, at an appropriate loudness, can impact the accelerometer, leaking sensitive information about the speech. In particular, we show that by exploiting the affected accelerometer readings and carefully selecting feature sets along with off-the-shelf machine learning techniques, Spearphone can perform gender classification (accuracy over 90%) and speaker identification (accuracy over 80%) for the audio/video playback on the smartphone for our recorded dataset. We use lightweight classifiers and an off-the-shelf machine learning tool so that the attacking effort is minimized, making our attack practical. Our results with testing the attack on a voice call and voice assistant response were also encouraging, showcasing the impact of the proposed attack. In addition, we perform speech recognition and speech reconstruction to extract more information about the eavesdropped speech to an extent. Our work brings to light a fundamental design vulnerability in many currently-deployed smartphones, which may put people's speech privacy at risk while using the smartphone in the loudspeaker mode during phone calls, media playback or voice assistant interactions.

Recent advances in speaker verification and speech processing technology have seen voice authentication being adopted on a wide scale in commercial applications like online banking and customer care support and on devices such as smartphones and IoT voice assistant systems. However, it has been shown that the current voice authentication systems can be ineffective against voice synthesis attacks that mimic a user's voice to high precision. In this work, we suggest a paradigm shift from the traditional voice authentication systems operating in the audio domain but susceptible to speech synthesis attacks (in the same audio domain). We leverage a motion sensor's capability to pick up phonatory vibrations, that can help to uniquely identify a user via voice signatures in the vibration domain. The user's speech is played/echoed back by a device's speaker for a short duration (hence our method is termed EchoVib) and the resulting non-linear phonatory vibrations are picked up by the motion sensor for speaker recognition. The uniqueness of the device's speaker and its accelerometer results in a device-specific fingerprint in response to the echoed speech. The use of the vibration domain and its non-linear relationship with audio allows EchoVib to resist the state-of-the-art voice synthesis attacks, shown to be successful in the audio domain. We develop an instance of EchoVib using the onboard loudspeaker and the accelerometer embedded in smartphones, as the authenticator, based on machine learning techniques. Our evaluation shows that even with the low-quality loudspeaker and the low-sampling rate of accelerometer recordings, EchoVib can identify users with an accuracy of over 90%. We also analyze our system against state-of-art-voice synthesis attacks and show that it can distinguish between the morphed and the original speaker's voice samples, correctly rejecting the morphed samples with a success rate of 85% for voice conversion and voice modeling attacks. We believe that using the vibration domain to detect synthesized speech attacks is effective due to the hardness of preserving the unique phonatory vibration signatures and is difficult to mimic due to the non-linear mapping of the unique speaker and accelerometer response in the vibration domain to the voice in the audio domain.

In this paper, we propose Voicefox1, a defense against the threat of automated voice synthesis attacks in machine-based and human-based speaker verification applications. Voicefox is based on a hitherto undiscovered potential of speech-to-text transcription, already built into these applications. Voicefox relies on the premise that while the synthesized samples might be falsely accepted by the speaker verification systems and human listeners, they cannot be transcribed as accurately as a natural human voice by transcribers. Voicefox is not a speaker verification system, but rather an independent module that can be integrated with any speaker verification system to enhance its security against voice synthesis attacks. To test our premise and as an essential pre-requisite for building Voicefox, we ran an extensive study that measures the accuracy of off-the-shelf speech-to-text techniques when confronted with the synthesized samples generated by the state-of-the-art speech synthesis techniques. Our results show that the transcription error rate for the synthesized voices is significantly higher, on average 2-3x, than the error rate for natural voices. This study quantitatively proves our hypothesis that human voices are transcribed more accurately than synthesized voices. We further propose several post-transcription rules in designing Voicefox, including acceptance of transcribed text even if up to a certain number of words are not transcribed correctly, and ignoring the words not available in the reference dictionary. Using these rules, Voicefox can effectively reduce the false rejection rates to as low as 1.20-4.69% depending on the application and the transcriber used, and reduce the false accept rates to 0% for dictionaries with phonetically-distinct words.

Voice access technologies are widely adopted in mobile devices and voice assistant systems as a convenient way of user interaction. Recent studies have demonstrated a potentially serious vulnerability of the existing voice interfaces on these systems to "hidden voice commands". This attack uses synthetically rendered adversarial sounds embedded within a voice command to trick the speech recognition process into executing malicious commands, without being noticed by legitimate users. In this paper, we employ low-cost motion sensors, in a novel way, to detect these hidden voice commands. In particular, our proposed system extracts and examines the unique audio signatures of the issued voice commands in the vibration domain. We show that such signatures of normal commands vs. synthetic hidden voice commands are distinctive, leading to the detection of the attacks. The proposed system, which benefits from a speaker-motion sensor setup, can be easily deployed on smartphones by reusing existing on-board motion sensors or utilizing a cloud service that provides the relevant setup environment. The system is based on the premise that while the crafted audio features of the hidden voice commands may fool an authentication system in the audio domain, their unique audio-induced surface vibrations captured by the motion sensor are hard to forge. Our proposed system creates a harder challenge for the attacker as now it has to forge the acoustic features in both the audio and vibration domains, simultaneously. We extract the time and frequency domain statistical features, and the acoustic features (e.g., chroma vectors and MFCCs) from the motion sensor data and use learning-based methods for uniquely determining both normal commands and hidden voice commands. The results show that our system can detect hidden voice commands vs. normal commands with 99.9% accuracy by simply using the low-cost motion sensors that have very low sampling frequencies.

Keyboard acoustic side channel attacks to date have been mostly studied in the context of an adversary eavesdropping on keystrokes by placing a listening device near the intended victim creating a local eavesdropping scenario. However, being in close physical proximity of the victim significantly limits the applicability of the attack. In this paper, we study the keyboard acoustic side channel attacks in remote attack settings and propose countermeasures in such attack settings. Specifically, we introduce an offense-defense system that: (1) highlights the threat of a remote adversary eavesdropping on keystrokes while the victim is on a VoIP call, and (2) builds a way to mask the leakage through the use of system-generated sounds. On the offensive side, we show the feasibility of existing acoustic side channel attacks adapted to a remote eavesdropper setting against sensitive input such as random passwords, PINs etc. On the defensive side, we demonstrate a software-based approach towards masking the keystroke emanations as a defense mechanism against such attacks and evaluate its effectiveness. In particular, we study the use of white noise and fake keystrokes as masking sounds and show the latter to be an effective means to cloak such side channel attacks. Finally, we discuss a novel way of masking by virtually inserting the masking signal in remote voice calls without distracting the user.

An interesting approach to pairing devices involves the use of a vibrational channel, over which the keying material (e.g., a short PIN) is sent. This approach is efficient (only a unidirectional transfer of PIN is needed) and simple (the sending device requires a vibration motor and receiving device requires an accelerometer). However, it has been shown to be susceptible to acoustic emanations usually produced by the vibration motor. Recent research introduced a mechanism to defeat these attacks by attempting to mask the acoustic leakage with deliberate acoustic noises. In this paper, we pursue a systematic investigation of the security of such a "noisy vibrational pairing" mechanism in a strong yet realistic adversarial model where the eavesdropper is co-located with the victim device(s).Our contributions are two-fold. First, we show that existing noisy vibrational pairing mechanisms - based on white noise as the masking signal - are vulnerable against a co-located eavesdropper (although they may defeat a distant eavesdropper). We build our attack based on standard signal processing and noise filtering techniques, and show that it can result in a complete compromise of pairing security. Second, we propose a defense that bolsters the masking signal with low-frequency audio tones. We present and address the challenges associated with producing such low-frequency sounds with current commodity hardware. We show that our defensive approach can not only resist our above attack but is also robust to more sophisticated, noise filtering and source separation methods when applicable. We also establish that the insertion of low-frequency sounds does not affect the receiving device's capability to sense the vibrations generated by the sending device. The suggested defense may therefore be used to enhance the security of noisy vibrational pairing without affecting its performance on a wide variety of devices.

Deauthentication is an important component of any computing system that promises to offer legitimate access to restricted services residing on the system. As computing devices are ubiquitous, it has underscored the need to design zero-effort deauthentication systems from a usability perspective. While the design of such deauthentication systems is geared towards making them more usable, often the security implication of these deigns overlook the physical security of the system resulting in various side channel vulnerabilities in the system. This issue highlights the need to design a defense mechanism that is capable of minimizing the threat posed by such side channel attacks while having minimal impact on the design of the system.In this paper, we aim to address the sound-based vulnerability recently introduced in the literature, against one of the prominent zero-effort deauthentication schemes, called ZEBRA, that transparently and continuously authenticates the user using a wearable device wirelessly connected with the authentication terminal. To this end, we propose YELP, a novel and practical defense mechanism based on the principle of sound masking. YELP uses two different types of masking sounds, namely "white noise", and "music" for cloaking the acoustic side channel leakage underlying the ZEBRA system. We believe that the use of such masking sounds at a reasonable volume level can hide the acoustic leakage emanating from the physical component of the system, and thereby reduce, if not eliminate, the imposed sound-based vulnerability. Indeed, our results show that white noise, as a masking sound, can effectively hide the acoustic leakage from ZEBRA system, thereby significantly reducing the attack success rate of an audio-based side channel attacker while music can moderately hide the acoustic leakage from the system. Our work therefore shows that sound masking can be used as an effective tool in improving the security of (de)authentication systems against an audio-based side channel attack without affecting its original design and without requiring additional effort from the user.

Pairing between wireless devices may be secured by the use of an auxiliary channel such as audio, visuals or vibrations. A simple approach to pairing involves one of the devices initiating the transmission of a key, or keying material like a short password, over the auxiliary channel to the other device. A successful pairing is achieved when the receiving device is able to decode the key without any errors while the attacker is unable to eavesdrop the key.In this paper, we focus on the security of the vibration channel when used for the key transmission. As shown in some recent work, sending the keying material over a clear vibrational channel poses a significant risk of an acoustic side channel attack. Specifically, an adversary can listen onto the acoustic sounds generated by the vibration motor of the sending device and infer the keying material with a high accuracy. To counteract this threat, we propose a novel pairing scheme, called Vibreaker (a ``Vibrating speaker''), that involves active injection of acoustic noise in order to mask the key signal. In this scheme, the sending device artificially injects noise in the otherwise clear audio channel while transmitting the keying material via vibrations. We experiment with several choices for the noise signal and demonstrate that the security of the audio channel is significantly enhanced with Vibreaker when appropriate noise is used. The scheme requires no additional effort by the user, and imposes minimum hardware requirement and hence can be applied to many different contexts, such as pairing of IoT and implanted devices, wearables and other commodity gadgets.

It is well known that reasoning about substitution is a huge "distraction" that inevitably gets in the way of formalizing interesting properties of languages with variable bindings. Most formalizations have their own separate definitions of terms and substitution, and properties about it. However there is a great deal of uniformity in the way substitution works and the reasons why its properties hold. We expose this uniformity by defining terms, substitution and α-equality generically in Coq by parametrizing them over a Context Free Grammar annotated with Variable binding information (CFGV). We also provide proofs of many properties about the above definitions (enough to formalize the PER semantics of Nuprl in Coq). Unlike many other tools which generate a custom definition of substitution for each input, all instantiations of our term model share the same substitution function. The proofs about this function have been accepted by Coq's typechecker once and for all.