Search

Here we present AusKidTalk [1], an audio-visual (AV) corpus of Australian children’s speech collected to facilitate the development of speech based technological solutions for children. It builds upon the technology and expertise developed through the collection of an earlier corpus of Australian adult speech, AusTalk [2,3]. This multi-site initiative was established to remedy the dire shortage of children’s speech corpora in Australia and around the world that are sufficiently sized to train accurate automated speech processing tools for children. We are collecting ~600 hours of speech from children aged 3–12 years that includes single word and sentence productions as well as narrative and emotional speech. In this paper, we discuss the key requirements for AusKidTalk and how we designed the recording setup and protocol to meet them. We also discuss key findings from our feasibility study of the recording protocol, recording tools, and user interface.

Effective sleep analysis is hampered by the lack of automated tools catering to disordered sleep patterns and cumbersome monitoring hardware. In this paper, we apply deep learning on a set of 57 EEG features extracted from a maximum of two EEG channels to accurately differentiate between patients with insomnia or controls with no sleep complaints. We investigated two different approaches to achieve this. The first approach used EEG data from the whole sleep recording irrespective of the sleep stage (stage-independent classification), while the second used only EEG data from insomnia-impacted specific sleep stages (stage-dependent classification). We trained and tested our system using both healthy and disordered sleep collected from 41 controls and 42 primary insomnia patients. When compared with manual assessments, an NREM + REM based classifier had an overall discrimination accuracy of 92% and 86% between two groups using both two and one EEG channels, respectively. These results demonstrate that deep learning can be used to assist in the diagnosis of sleep disorders such as insomnia.

Advanced metering infrastructure (AMI) is the primary step to establish a modern smart grid. AMI enables a flexible two-way communication between smart meters and utility company for monitoring and billing purposes. However, AMI suffers from the deceptive behavior of malicious consumers who report false electricity usage in order to reduce their bills, which is known as electricity theft cyber-attacks. In this chapter, we present deep learning-based detectors that can efficiently thwart electricity theft cyber-attacks in smart grid AMI networks. First, we present a customer-specific detector based on a deep feed-forward and recurrent neural networks (RNN). Then, we develop generalized electricity theft detectors that are more robust against contamination attacks compared with customer-specific detectors. In all detectors, optimization of hyperparameters is investigated to improve the performance of the developed detectors. In particular, the hyperparameters of the detectors are optimized via sequential, random, and genetic optimization-based grid search approaches. Extensive test studies are carried out against real energy consumption data to investigate all detectors performance. Also, the performance of the developed deep learning-based detectors is compared with a shallow machine learning approach and a superior performance is observed for the deep learning-based detectors.

Chronic insomnia can significantly impair an individual's quality of life leading to a high societal cost. Unfortunately, limited automated tools exist that can assist clinicians in the timely detection of insomnia. In this paper, we propose a two stage approach to automatically detect insomnia from an overnight EEG recording. In the first stage we trained a sleep stage scoring model and an epoch-level insomnia detection model. Both models are deep neural network (DNN)- based which are fed by a set of temporal and spectral features derived from 2 EEG channels. In the second stage we computed two subject-level feature sets. One is computed using the output of the sleep stage scoring model and consists of the sleep stage ratios, the stage pair ratios and the stage transition ratios. The second feature set is derived from the output of the epoch-level insomnia detection model and represents the ratio of detected insomniac epochs in each stage and their average posterior probability. These features are then used to train a final binary classifier to classify each subject as control, i.e., with no sleep complaints, or insomniac. We compared 5 different binary classifiers, namely the linear discriminant analysis (LDA), the classification and regression trees (CART) and the support vector machine (SVM) with linear, Gaussian and sigmoid kernels. The system was evaluated against data collected from 115 participants, 61 control and 54 with insomnia, and achieved $F1$ score, sensitivity and specificity of 0.88, 84% and 91% respectively.

Modern smart grids rely on advanced metering infrastructure (AMI) networks for monitoring and billing purposes. However, such an approach suffers from electricity theft cyberattacks. Different from the existing research that utilizes shallow, static, and customer-specific-based electricity theft detectors, this paper proposes a generalized deep recurrent neural network (RNN)-based electricity theft detector that can effectively thwart these cyberattacks. The proposed model exploits the time series nature of the customers' electricity consumption to implement a gated recurrent unit (GRU)-RNN, hence, improving the detection performance. In addition, the proposed RNN-based detector adopts a random search analysis in its learning stage to appropriately fine-tune its hyper-parameters. Extensive test studies are carried out to investigate the detector's performance using publicly available real data of 107,200 energy consumption days from 200 customers. Simulation results demonstrate the superior performance of the proposed detector compared with state-of-the-art electricity theft detectors.

The automatic assessment of speech plays an important role in Computer Aided Pronunciation Learning systems. However, modeling both the correct and incorrect pronunciation of each phoneme to achieve accurate pronunciation verification is unfeasible due to the lack of sufficient mispronounced samples in training datasets. In this paper, we propose a novel approach that handles this unbalanced data distribution by building multiple one-class SVMs to evaluate each phoneme as correct or incorrect. We model the correct pronunciation of each individual phoneme with a one-class SVM trained using a set of speech attributes features, namely the manner and place of articulation. These features are extracted from a bank of pre-trained DNN speech attributes classifiers. The one-class SVM model measures the similarity between the new data and the training set and then classifies it as normal (correct) or an anomaly (incorrect). We evaluated the system using native speech corpus and disordered speech corpus and compared it with the conventional Goodness of Pronunciation (GOP) algorithm. The results show that our approach reduces the false-acceptance and false-rejection rates by around 26% and 39% respectively.

Automated tool to assess productions from children with apraxia of speech.Consists of clinician interface, mobile application and speech processing engine.Automatically detects groping errors, articulation errors and prosodic errors.Lattice-based Pronunciation Verification module detects articulation errors.Lexical Stress Pattern Verification module detects prosodic errors. Children with developmental disabilities such as childhood apraxia of speech (CAS) require repeated intervention sessions with a speech therapist, sometimes extending over several years. Technology-based therapy tools offer the potential to reduce the demanding workload of speech therapists as well as time and cost for families. In response to this need, we have developed "Tabby Talks," a multi-tier system for remote administration of speech therapy. This paper describes the speech processing pipeline to automatically detect common errors associated with CAS. The pipeline contains modules for voice activity detection, pronunciation verification, and lexical stress verification. The voice activity detector evaluates the intensity contour of an utterance and compares it against an adaptive threshold to detect silence segments and measure voicing delays and total production time. The pronunciation verification module uses a generic search lattice structure with multiple internal paths that covers all possible pronunciation errors (substitutions, insertions and deletions) in the child's production. Finally, the lexical stress verification module classifies the lexical stress across consecutive syllables into strong-weak or weak-strong patterns using a combination of prosodic and spectral measures. These error measures can be provided to the therapist through a web interface, to enable them to adapt the child's therapy program remotely. When evaluated on a dataset of typically developing and disordered speech from children ages 4-16years, the system achieves a pronunciation verification accuracy of 88.2% at the phoneme level and 80.7% at the utterance level, and lexical stress classification rate of 83.3%.

Future smart grids rely on advanced metering infrastructure (AMI) networks for monitoring and billing purposes. However, several research works have revealed that such AMI networks are vulnerable to different kinds of cyber attacks. In this research work, we consider one type of such cyber attacks that targets electricity theft and we propose a novel detection mechanism based on a deep machine learning approach. While existing research papers focus on shallow machine learning architectures to detect these cyber attacks, we propose a deep feedforward neural network (D-FF-NN) detector that can thwart such cyber attacks efficiently. To optimize the D-FF-NN hyper-parameters, we apply a sequential grid search technique that significantly improves the detector»s performance while reducing the associated complexity in the learning process. We carry out extensive studies to test the proposed detector based on a publicly available real load profile data of 5000 customers. The detector»s performance is investigated against a mixture of different attacks including partial reduction attacks, selective by-pass attacks, and price-based load control attacks. Our study reveals that the proposed D-FF-NN detector presents a superior performance compared with state-of-the-art detector»s that are based on shallow machine learning architectures

Sleep disorders are becoming increasingly prevalent in society. However most of the burgeoning research on automated sleep analysis has been in the realm of sleep stage classification with limited focus on accurately diagnosing these disorders. In this paper, we explore two different models to discriminate between control and insomnia patients using support vector machine (SVM) classifiers. We validated the models using data collected from 124 participants, 70 control and 54 with insomnia. The first model uses 57 features derived from two channels of EEG data and achieved an accuracy of 81%. The second model uses 15 features from each participant's hypnogram and achieved an accuracy of 74%. The impetus behind using these two models is to follow the clinician's diagnostic decision-making process where both the EEG signals and the hypnograms are used. These results demonstrate that there is potential for further experimentation and improvement of the predictive capability of the models to help in diagnosing sleep disorders like insomnia.

This paper introduces a pronunciation verification method to be used in an automatic assessment therapy tool of child disordered speech. The proposed method creates a phonebased search lattice that is flexible enough to cover all probable mispronunciations. This allows us to verify the correctness of the pronunciation and detect the incorrect phonemes produced by the child. We compare between two different acoustic models, the conventional GMM-HMM and the hybrid DNN-HMM. Results show that the hybrid DNNHMM outperforms the conventional GMM-HMM for all experiments on both normal and disordered speech. The total correctness accuracy of the system at the phoneme level is above 85% when used with disordered speech.

Background and Objectives Childhood Apraxia of Speech (CAS) is a speech disorder characterized by articulation errors, i.e. the replacement of certain phonemes with alternatives. In previous work we proposed a simple method to evaluate the child's speech as correct or incorrect with an overall accuracy of 88.2%. In this work we present an enhanced method that increases the accuracy of the correct/incorrect evaluation to 92.7%, in addition to identifying the incorrect phonemes with an accuracy of 60%. Method The goal of the mispronunciation detection system is to compare each phoneme in the child's production to their given prompt and identify mispronunciations. Figure 1 shows the block diagram of the system, which uses a search lattice for each prompt in the child's speech therapy treatment protocol to identify errors made. Each prompt is transcribed as per the corresponding phoneme sequence using the CMU pronunciation dictionary and then passed to the lattice generator along with the expected mispronunciation rules to generate the search lattice. Mel Frequency Cepstral Coefficients (MFCC) are extracted from the speech signal with delta and acceleration to produce a 39- dimensional feature vector per frame. The extracted features are then fed to the speech recognizer along with the created lattice and the Hidden Markov Model (HMM) acoustic models to generate a sequence of phones from the child's utterance. An evaluation report is then generated by matching the recognized phoneme sequence with the correct phoneme sequence and specifying the errors made by the child. We use a search lattice with a specific number of alternative pronunciations for each phoneme; this limits the decoder search, making it faster and more accurate. Each phoneme in the correct phoneme sequence is compared with expected mispronunciation rules developed by a therapist after an assessment of 20 children with CAS; if a rule is matched, the pronunciation variants are added as alternative arcs to the current phoneme sequence. The mispronunciation rules depend on the type of the phoneme (consonant/vowel), the phoneme position in the word (Initial/Medial/Final) and the context of the phoneme. The lattice is then created using the matched rules as shown in Figure 2, where the garbage model absorbs any mispronounced phoneme not in the lattice. PA and PG are insertion penalties added to the alternative and the garbage arcs respectively so the decoder does not align the speech to the alternative error phonemes or the garbage node unless it is confident enough. Results The system overall system accuracy is 92.7% where the Correct Acceptance (CA) is 97.6% and the Correct Rejection (CR) is 83.1%. The system also detects phoneme errors made by the child with 60% accuracy. Conclusion In this paper we proposed a mispronunciation detection tool that can detect phonemes mispronounced by children with CAS and specify the errors made.

Technology based speech therapy systems are severely handicapped due to the absence of accurate prosodic event identification algorithms. This paper introduces an automatic method for the classification of strong-weak (SW) and weak-strong (WS) stress patterns in children speech with American English accent, for use in the assessment of the speech dysprosody. We investigate the ability of two sets of features used to train classifiers to identify the variation in lexical stress between two consecutive syllables. The first set consists of traditional features derived from measurements of pitch, intensity and duration, whereas the second set consists of energies of different filter banks. Three different classifiers were used in the experiments: an Artificial Neural Network (ANN) classifier with a single hidden layer, Support Vector Machine (SVM) classifier with both linear and Gaussian kernels and the Maximum Entropy modeling (MaxEnt). these features. Best results were obtained using an ANN classifier and a combination of the two sets of features. The system correctly classified 94% of the SW stress patterns and 76% of the WS stress patterns.