Search

This project proposes a voice activated home automation system with location functionalities. The system consists of microcontrollers distributed around the home with a microphone that perform a voice activity detection in real-time. Then a communication protocol has been designed to manage the audio transmission from the microcontrollers to the controller device using WiFi. The controller can then locate the source of the audio performing trilateration with the time difference of arrival of the signals into the microphones. Time precision is needed in the voice detection and the network needs to be synchronized. Several methods are proposed and tested. Using the WiFi beacons as reference. Generating multicast UDP datagrams periodically. At the end, the synchronization is implemented using the output of a zero crossing detector connected to the mains electricity because the other methods had some limitations. Finally, a provisioning method is proposed for easier deployment. Este proyecto propone un sistema domótico activado por voz con funcionalidades de localización. El sistema consiste en microcontroladores distribuidos por el hogar con un micrófono que realizan una detección de voz en tiempo real. Después se ha diseñado un protocolo de comunicación para gestionar la transmisión de audio de los microcontroladores al dispositivo controlador utilizando WiFi. El controlador puede localizar el origen de audio haciendo una trilateración de la diferencia de los tiempos de llegada de la señal a los micrófonos. Es necesaria una gran precisión temporal en la detección de voz y la red debe estar sincronizada. Varios métodos han sido propuestos y probados. Utilizando los beacons WiFi como referencia. Generando datagramas UDP en multidifusión de forma periódica. Al final, la sincronización ha sido implementada utilizando la salida de un circuito detector de paso por cero conectado a la toma de corriente del hogar ya que los otros métodos presentan limitaciones. Finalmente, se ha propuesto un método de aprovisionamiento para un despliegue más fácil. Aquest projecte proposa un sistema domòtic activat per veu amb funcionalitats de localització. El sistema consisteix en microcontroladors distribuïts per la llar amb un micròfon que realitzen una detecció de veu en temps real. Després s?ha dissenyat un protocol de comunicació per a gestionar la transmissió d?àudio dels microcontroladors al dispositiu controlador utilitzant WiFi. El controlador pot localitzar l?origen d?àudio fent una trilateració de la diferència dels temps d?arribada de les senyals als micròfons. És necessària una gran precisió temporal a la detecció de veu i la xarxa ha d?estar sincronitzada. Diversos mètodes han estat proposats i provats. Utilitzant els beacons WiFi com a referència. Generant datagramas UDP en difusió selectiva de forma periòdica. Al final, la sincronitació ha estat implementada utilitzant la sortida d?un circuit de pas per zero connectat a la presa de corrent de la llar ja que els altres mètodes presenten limitacions. Finalment, s?ha proposat un mètode d?aprovisionament per a un desplegament més fàcil.

Speech enhancement improves recorded voice utterances to eliminate noise that might be impeding their intelligibility or compromising their quality. Typical speech enhancement systems are based on regression approaches that subtract noise or predict clean signals. Most of them do not operate directly on waveforms. In this work, we propose a generative approach to regenerate corrupted signals into a clean version by using generative adversarial networks on the raw signal. We also explore several variations of the proposed system, obtaining insights into proper architectural choices for an adversarially trained, convolutional autoencoder applied to speech. We conduct both objective and subjective evaluations to assess the performance of the proposed method. The former helps us choose among variations and better tune hyperparameters, while the latter is used in a listening experiment with 42 subjects, confirming the effectiveness of the approach in the real world. We also demonstrate the applicability of the approach for more generalized speech enhancement, where we have to regenerate voices from whispered signals.

The use of private layers is explored for a multi-task learning approach to speech emotion recognition. The methods used include LSTMs, multi-task learning, high level representation, and "in the wild" testing.

In the last years, i-vectors followed by cosine or PLDA scoringtechniques were the state-of-the-art approach in speaker veri-fication. PLDA requires labeled background data, and thereexists a significant performance gap between the two scoringtechniques. In this work, we propose to reduce this gap by us-ing an autoencoder to transform i-vector into a new speaker vec-tor representation, which will be referred to as ae-vector. Theautoencoder will be trained to reconstruct neighbor i-vectors in-stead of the same training i-vectors, as usual. These neighbori-vectors will be selected in an unsupervised manner accordingto the highest cosine scores to the training i-vectors. The evalua-tion is performed on the speaker verification trials of VoxCeleb-1 database. The experiments show that our proposed ae-vectorsgain a relative improvement of 42% in terms of EER comparedto the conventional i-vectors using cosine scoring, which fillsthe performance gap between cosine and PLDA scoring tech-niques by 92%, but without using speaker labels

The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation. This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO).

Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level. Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation. In this work we propose the use of an attention mechanism to obtain a discriminative speaker embedding given non fixed length speech utterances. Our system is based on a Convolutional Neural Network (CNN) that encodes short-term speaker features from the spectrogram and a self multi-head attention model that maps these representations into a long-term speaker embedding. The attention model that we propose produces multiple alignments from different subsegments of the CNN encoded states over the sequence. Hence this mechanism works as a pooling layer which decides the most discriminative features over the sequence to obtain an utterance level representation. We have tested this approach for the verification task for the VoxCeleb1 dataset. The results show that self multi-head attention outperforms both temporal and statistical pooling methods with a 18\% of relative EER. Obtained results show a 58\% relative improvement in EER compared to i-vector+PLDA., 4+1 pages. 4 Figures. Accepted for Interspeech 2009

The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation. This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO).

The influence of emotional content on language processing remains unclear. Previous research conducted in English has obtained contradictory results regarding the effects of valence on word recognition. Whereas some studies indicate that valence predicts recognition latencies such that negative words are processed more slowly than positive words, other studies indicate facilitation of responses to emotional (both positive and negative) compared to neutral words. The authors examined the influence of valence and arousal on word recognition reaction time (RT) using large-scale word naming and lexical decision data-sets in Spanish. They found that linear mixed-effects model estimates revealed a valence but not an arousal effect on reading latencies. The influence of valence was better captured by a graded (RTs to positive words< neutral< negative) than by a categorical (positive< negative) valence effect. A categorical emotional versus neutral effect was not reliably observed. In an advance on previous research, the authors' analyses showed that the valence effect is substantially more prominent in lexical decision than in pronunciation. These results mirror some of those reported previously in English, adding evidence to support their validity, and demonstrating important parallels in word recognition processes in orthographically shallow as well as deep languages.(PsycINFO Database Record (c) 2018 APA, all rights reserved).

Silent Speech Interface (SSI) is a technology able to synthesize speech in the absence of any acoustic signal. It can be useful in cases like laryngectomy patients, noisy environments or silent calls. This thesis explores the particular case of SSI using ultrasound images of the tongue as input signals. A 'direct synthesis' approach based on Deep Neural Networks and Mel-generalized cepstral coefficients is proposed. This document is an extension of Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface". Several deep learning models, such as the basic Feed-forward Neural Networks, Convolutional Neural Networks and Recurrent Neural Networks are presented and discussed. A denoising pre-processing based on a Deep Convolutional Autoencoder has also been studied. A considerable number of experiments using a set of different deep learning architectures and an extensive hyperperameter optimization study have been realized. The different experiments have been testing and rating several objective and subjective quality measures. According to the experiments, an architecture based on a CNN and bidirectional LSTM layers has shown the best results in both objective and subjective terms. Silent Speech Interface (SSI) is a technology able to synthesize speech in the absence of any acoustic signal. It can be useful in cases like laryngectomy patients, noisy environments or silent calls. This thesis explores the particular case of SSI using ultrasound images of the tongue as input signals. A 'direct synthesis' approach based on Deep Neural Networks and Mel-generalized cepstral coefficients is proposed. This document is an extension of Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface". Several deep learning models, such as the basic Feed-forward Neural Networks, Convolutional Neural Networks and Recurrent Neural Networks are presented and discussed. A denoising pre-processing based on a Deep Convolutional Autoencoder has also been studied. A considerable number of experiments using a set of different deep learning architectures and an extensive hyperperameter optimization study have been realized. The different experiments have been testing and rating several objective and subjective quality measures. According to the experiments, an architecture based on a CNN and bidirectional LSTM layers has shown the best results in both objective and subjective terms. Silent Speech Interface (SSI) és una tecnologia capaç de sintetitzar veu partint únicament de senyals no-acústiques. Pot tenir gran utilitat en casos com pacients de laringectomia, ambients sorollosos o trucades silencioses. Aquesta tèsis explora el cas particular de SSI utilitzant imatges de la llengua captades amb ultrasons com a senyals d'entrada. Es proposa un enfocament de 'síntesis directa' basat en Xarxes Neuronals Profundes i coeficients Mel-generalized cepstral. Aquest document és una extensió del treball de Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface" . Diversos models de xarxes neuronals són presentats i discutits, com les bàsiques xarxes neuronals directes, xarxes neuronals convolucionals o xarxes neuronals recurrents. També s'ha estudiat un pre-processat reductor de soroll basat en un Autoencoder convolucional profund. S'ha portat a terme un nombre considerable d'experiments utilitzant diverses arquitectures de Deep Learning, així com un extens estudi d'optimització d'hyperparàmetres. Els diferents experiments han estat evaluar i qualificar a partir de diferentes mesures de qualitat objectives i subjectives. Els millors resultats, tant en termes objectius com subjectius, els ha presentat una arquitectura basada en una CNN i capes bidireccionals de LSTMs.

HighlightsOverview of hybrid machine translation guided by the rule-based paradigm.Overview of hybrid machine translation guided by the corpus-based paradigm.Review of works on machine translation applications with hybrid components. This survey on hybrid machine translation (MT) is motivated by the fact that hybridization techniques have become popular as they attempt to combine the best characteristics of highly advanced pure rule or corpus-based MT approaches. Existing research typically covers either simple or more complex architectures guided by either rule or corpus-based approaches. The goal is to combine the best properties of each type.This survey provides a detailed overview of the modification of the standard rule-based architecture to include statistical knowledge, the introduction of rules in corpus-based approaches, and the hybridization of approaches within this last single category. The principal aim here is to cover the leading research and progress in this field of MT and in several related applications.

Alzheimer’s disease (AD) is the most prevalent form of degenerative dementia; it has a high socio-economic impact in Westerncountries. The purpose of our project is to contribute to earlier diagnosis of AD and allow better estimates of its severity by usingautomatic analysis performed through new biomarkers extracted through non-invasive intelligent methods. The method selectedis based on speech biomarkers derived from the analysis of spontaneous speech (SS). Thus the main goal of the present work isfeature search in SS, aiming at pre-clinical evaluation whose results can be used to select appropriate tests for AD diagnosis. Thefeature set employed in our earlier work offered some hopeful conclusions but failed to capture the nonlinear dynamics of speechthat are present in the speech waveforms. The extra information provided by the nonlinear features could be especially useful whentraining data is limited. In this work, the fractal dimension (FD) of the observed time series is combined with linear parameters inthe feature vector in order to enhance the performance of the original system while controlling the computational cost.© 2014 Elsevier Ltd. All rights reserved.

Most speech synthesis systems require a linguistic module to produce the features that drive the speech generation module. In this project, system will be designed using a deep architecture and automatically learned to produce either linguistic features or speech from the raw letter representation. Recently, neural networks have become the state of the art for speech synthesis from raw text tasks and they are actually representing a powerful force in the industry. In this project, we present an end-to-end deep learning-based TTS system, able to generate a voice signal from characters. In order to fulfil this task we developed a re-implementation of the Uncodicional-SampleRNN neural vocoder, in order to be conditioned under an adaptation of MUSA, which predicts vocoder parameters from text. Recientemente, las redes neuronales se han convertido en el estado del arte para las tareas de síntesis del habla y actualmente representan una fuerza poderosa en la industria. En este proyecto, presentamos un sistema de conversión de texto a voz (Text-to-Speech) basado en aprendizaje profundo (Deep learning), capaz de generar una señal de voz a partir de caracteres. Para realizar dicha tarea desarrollamos una adaptación de MUSA, encargado de realizar una predicción de los parámetros del vocoder a partir del texto para, posteriormente, condicionar una reimplementación del vocoder neuronal Uncodicional-SampleRNN. Recentment, les xarxes neuronals s'han convertit en l'estat de l'art per a les tasques de síntesis de la parla i actualment representen una força poderosa en la indústria. En aquest projecte, presentem un sistema de conversió de text a veu (Text-to-Speech) basat en aprenentatge profund (Deep learning), capaç de generar un senyal de veu a partir de caràcters. Per realitzar aquesta tasca desenvolupem una adaptació de MUSA, encarregat de realitzar una predicció dels paràmetres del vocoder a partir del text, per condicionar posteriorment una reimplementació del vocoder neuronal Uncodicional-SampleRNN.

Speech Recognition (reconocimiento de voz) es la tarea que pretende indentificar palabras habladas y convertirlas a texto. Este trabajo de fin de grado se centra en utilizar técnicas de deep learning para construir un sistema de Speech Recognition entrenándolo end-to-end. Como paso preliminar, hacemos un resumen de los métodos más relevantes llevados a cabo los últimos años. A continuación estudiamos uno de los trabajos más recientes en este área que propone un modelo sequence to sequence con atención entrenado end-to-end. Después, reproducimos satisfactoriamente el modelo y lo avaluamos con la base de datos TIMIT. Analizamos los parecidos y diferencias entre la implementación propuesta y el trabajo teórico original. Y finalmente, experimentamos y contrastamos el modelo utilizando diferentes parámetros (e.g. numero de neuronas por capa, la tasa de aprendizaje -learning rate y los batch sizes) y reducimos el Phoneme Error Rate cerca del 12% relativo. Speech recognition is the task aiming to identify words in spoken language and convert them into text. This bachelor's thesis focuses on using deep learning techniques to build an end-to-end Speech Recognition system. As a preliminary step, we overview the most relevant methods carried out over the last several years. Then, we study one of the latest proposals for this end-to-end approach that uses a sequence to sequence model with attention-based mechanisms. Next, we successfully reproduce the model and test it over the TIMIT database. We analyze the similarities and differences between the current implementation proposal and the original theoretical work. And finally, we experiment and contrast using different parameters (e.g. number of layer units, learning rates and batch sizes) and reduce the Phoneme Error Rate in almost 12% relative. Speech Recognition (reconeixement de veu) és la tasca que pretén indentificar paraules del llenguatge parlat i convertir-les a text. Aquest treball de fi de grau es centra en utilitzar tècniques de deep learning per construir un sistema d'Speech Recognition entrenant-lo end-to-end. Com a pas preliminar, fem un resum dels mètodes més rellevants duts a terme els últims anys. A continuació, estudiem un dels treballs més recents en aquesta àrea que proposa un model sequence to sequence amb l?atenció entrenat end-to-end. Després, reproduim satisfactòiament el model i l'avaluem amb la base de dades TIMIT. Analitzem les semblances i diferències entre l'implementació proposada i el treball teòric original. I finalment, experimentem i contrastem el model utilitzant diferents paràmetres (e.g. nombre de neurones per capa, la taxa d'aprenentatge -learning rate- i els batch sizes) i reduim el Phoneme Error Rate gairebé un 12% relatiu.

Treball de fi de grau en Sistemes Audiovisuals Tutora: Mireia Farrús i Cabeceran La interacció persona-màquina ha augmentat exponencialment durant els últims anys. Un exemple d’aquest creixement es pot veure en el camp del processament de la parla, i en concret en el de la síntesi de veu. Aquest treball va sobre el tractament de la síntesi de veu dins el marc del projecte KRISTINA, orientat al sector mèdic, on s’utilitzarà veu sintetitzada perquè els pacients amb dificultats de comprensió de la llengua autòctona del país on viuen s’hi puguin comunicar. Els sintetitzadors de veu (o Text-To-Speech (TTS) systems, en anglès) acostumen a ser monòtons i amb poca naturalitat. Un dels objectius d’aquest treball és aconseguir generar una millor veu per a l’avatar del projecte KRISTINA, dotant-la de més naturalitat i expressivitat, de manera que s’acosti al màxim a una veu humana. Hem treballat la prosòdia a nivell de paràgraf i a nivell d’estructura comunicativa, utilitzant diferents sintetitzadors de veu. Mitjançant l’anàlisi i l’aplicació de diversos llenguatges d’etiquetes de veu utilitzats per a modificar determinades característiques prosòdiques relacionades amb l’entonació, la durada i la intensitat, hem aconseguit millores en la naturalitat i l’expressivitat de la veu sintetitzada. The human-computer interaction has increased exponentially in recent years. An example of this growth can be seen in the field of speech processing, specifically in speech synthesis. This work is about the treatment of speech synthesis within the framework of KRISTINA, a project oriented in the medical sector, which develop a conversational avatar to facilitate communication with patients with difficulties in understanding the native language of the country where they live. The majority of Text-to-Speech (TTS) voice synthesizers tend to be monotonous and with few naturalness, and one of the aims of this work is to create a better voice for KRISTINA avatar, providing it with more naturalness and expressiveness, approaching as close as possible to a human voice. We have worked prosody paragraph level and in terms of communicative structure, by using several speech synthesizers. By analyzing and applying different speech synthesis markup languages used to modify certain prosodic characteristics related to intonation, duration and intensity, we achieved improvements in naturalness and expressiveness of the synthesized voice.

The following document will drive the reader through the whole development of a project consisting in the investigation of the feasibility and development of an access control security system that uses voice biometrics as its cornerstone to make rooms as safe and controlled as possible in a two-factor authentication approach using open software. For that, an initial decision is presented, in which the technology that is going to be used to make the Speaker Recognition functionality possible is chosen. A detailed explanation of it is included so that the reader will know and understand the basics to follow the project development. With this, the document will go step by step through the different stages that I have faced and the requirements that came up and I had to adapt to, including the integration of the system that is nowadays up and running at IIT. The User Interface of the final application is presented together with an explanation of every step that users will have to follow to use the system correctly. Last, an estimation of the costs associated to this project is presented, which will give the reader an idea of how much starting this project from scratch would cost if the same hardware is used. Also, for those developers that would like to get some of this functionality in their systems, a link to a GitHub repository where they can find the code is also given.

In this project we present a first attempt at a Voice Conversion system based on Deep Learning in which the alignment between the training data is intrinsic to the model. Our system is structured in three main blocks. The first performs a vocoding of the speech (we have used Ahocoder for this task) and a normalization of the data. The second and main block consists of a Sequence-to-Sequence model. It consists of an RNN-based encoder-decoder structure with an Attention Mechanism. Its main strengts are the ability to process variable-length sequences, as well as aligning them internallly. The third block of the system performs a denormalization and reconstructs the speech signal. For the development of our system we have used the Voice Conversion Challenge 2016 dataset, as well as a part of the TC-STAR dataset. Unfortunately we have not obtained the results we expected. At the end of this thesis we present them and discuss some hypothesis to explain the reasons behind them. En este proyecto presentamos un primer intento en la realización de un sistema de Conversión de Voz basado en Aprendizaje Profundo (\emph{Deep Learning}) en el cual el alineamiento de los datos de entrenamiento es intrínseco al modelo. Nuestro sistema está estructurado en tres bloques principales. El primer bloque codifica la señal de voz en parámetros (\emph{vocoding}). Hemos elegido el \emph{vocoder} Ahocoder para esta tarea. Este bloque también normaliza los parámetros codificados. El segundo bloque consiste en un modelo \emph{Sequence-to-Sequence}. Este modelo está formado por una estructura codificador-decodificador basada en Redes Neuronales Recurrentes (RNN) con un Mecanismo de Atención. Sus puntos fuertes son la capacidad de procesar secuencias de longitud variable, a la vez que las alinea internamente. El tercer bloque del sistema desnormaliza los parámetros, y reconstruye la señal de voz a partir de ellos. Para el desarrollo del modelo hemos usado el conjunto de datos (\emph{dataset}) del \emph{Voice Conversion Challenge} 2016. También hemos usado una parte del conjunto TC-STAR. Desafortunadamente no hemos obtenido los resultados que esperábamos. Al final de esta tesis los presentamos y proponemos varias hipótesis que los explican. En aquest projecte presentem un primer itent en la realització d'un sistema de Conversió de Veu basat en Aprenentatge Profund (Deep Learning) en el qual l'alineament entre les dades d'entrenament sigui intrínsec al model. El nostre sistema s'estructura en tres blocs principals. El primer bloc codifica la veu en paràmetres (\emph{vocoding}). Hem usat el codificador Ahocoder per a aquesta tasca. A més a més, aquest primer bloc normalitza les dades. El segon bloc consisteix en un model \emph{Sequence-to-Sequence}. Consisteix en una estructura codificador-decodificador basada en Xarxes Neuronals Recurrents (RNN) amb un Mecanisme d'Atenció (\emph{Attention Mechanism}). Els punts forts d'aquest model són la capacitat per a tractar seqüències de durada variable, alhora que les alinea internament. El tercer bloc del sistema desnormalitza les seqüències i reconstrueix els senyals de veu. Per a desenvolupar el sistema hem usat el conjunt de dades del \emph{Voice Conversion Challenge} 2016. Hem fet servir també una part del conjunt TC-STAR. Desafortunadament no hem obtingut els resultats que esperàvem. Al final d'aquesta tesis presentem aquests resultats i plantegem algunes hipòtesis que els expliquen.

The transcription of spontaneous speech still poses a challenge to state-of-the-art methods for automatic speech recognition. The present thesis describes the comprehensive development of a large vocabulary continuous speech recognition system for the transcription of Catalan broadcast news and conversions and evolves towards novel approaches for analysis and modelling of acoustic reduction in spontaneous speech. It emphasises initially on various conventional methods for acoustic analysis, acoustic and language modelling and hypothesis search. Improvements over the original single-pass baseline system are mainly attained by domain and speaking style emphasising interpolation of individually estimated language models, linear discriminating projection of acoustic observations that improves the phonetic class separability, speaker normalisation of the acoustic observations, speaker adaptive training and acoustic model adaptation in a multi-pass system approach. The analysis of acoustic reduction initially emphasises on context independent vowel and consonant specific spectral and temporal properties whose parameters display statistically significant differences between the phoneme prototypes in spontaneous speech and their canonical realisations in planned speech. The introduction of the feature space analysis provides the general means to reveal these differences in conventional acoustic observations for automatic speech recognition. It displays statistically significant differences context-independently but also in a syllable context between adjacent phonemes suggesting particular reduction patterns. The analysis furthermore challenges the often suggested coherence between the co-occurring reduction of spectral and temporal properties. The modelling of acoustic reduction first emphasises on segment conditioned discriminating variables and variability class dependent models and variability class specific adaptation of the original acoustic model. It introduces phoneme rate as means to analyse temporal properties and feature space reduction ratio as means to analyse the reduction of spectral properties in conventional feature space for large vocabulary continuous speech recognition as discriminating variables. These variables are clustered and determine the classes for segment conditioned variability class dependent models and their scoring during the hypothesis search in recognition. Both approaches displays no significant performance improvement. Furthermore the modelling advances towards segment constituent predictability dependent models that introduce predictability as discriminating variable for variability class dependent models relying on the fundamental coherence between predictability and acoustic reduction that is suggested through the principle of least effort and the redundancy theory. It thereby emphasises on word and phoneme predictability. This approach displays no significant performance improvement. Planned speech is apparently antagonising the principle of least effort. Thus, a prior segment conditioned analysis of acoustic reduction may indicate its average degree of reduction, while their within-segment variation may indicate whether it exhibits sufficient relaxation of the speaking style to adopt the principle of least effort. Thus, segments exhibiting small within-segment variation may be modelled separately from those of large within-segment variation, whereas modelling the latter by word, syllable or phoneme predictability dependent models may provide a research perspective., La transcripció de converses espontànies encara suposa un repte per als mètodes actuals de reconeixement automàtic de veu. Aquesta tesi descriu el desenvolupament d'un sistema de reconeixement de veu continu de vocabulari gran per a la transcripció de converses i notícies emeses en català i condueix cap a noves aproximacions per a l'anàlisi i modelat de la reducció acústica en converses espontànies. Es centra inicialment en diversos mètodes convencionals per a l'anàlisi acústica, modelat acústic i del llenguatge i en la cerca d'hipòtesis. Les millores respecte el sistema original d'única passada són principalment degudes al domini i l'estil en la parla posant èmfasi en la interpolació de models de llenguatge, discriminació lineal i projecció d'observacions acústiques, entrenament adaptat al locutor per millorar la separació de les classes fonètiques, normalització de les observacions acústiques, i adaptació del model acústic en una sistema de múltiples passades. L'anàlisi de reducció acústica posa inicialment èmfasi en les propietats espectrals i temporals independents de vocals i consonant específiques, els paràmetres de les quals mostren diferències estadísticament significatives entre els prototips de fonemes en la conversa espontània i la seva realització canònica en el discurs planejat. La introducció de l'anàlisi del espai de característiques proporciona els mitjans generals per a revelar aquestes diferències en observacions acústiques convencionals per al reconeixement automàtic de veu. Mostra diferències estadísticament significatives independents de context però també entre fonemes adjacents en el context de síl·laba suggerint patrons de reducció particulars. A més, l'anàlisi desafia la, sovint suggerida, coherència entre les reducció simultànies de les propietats espectrals i temporals. El modelat de la reducció acústica primer fa èmfasi en variables discriminants de cada segment, models dependents de la variabilitat de la classe i l'adaptació del model acústic original. Introdueix la taxa de fonemes com a mitjà d'analitzar propietats temporals i la proporció de la reducció del espai de característiques com a mitjà d'analitzar la reducció dels propietats espectrals en el espai de característiques convencional per al reconeixement de veu continu de vocabulari gran com a variables discriminants. Aquestes variables s'agrupen i determinen les classes per a models dependents de la variabilitat de cada segment i la seva puntuació durant el reconeixement i cerca d'hipòtesi. Ambdues aproximacions no mostren una millora significativa en el rendiment. A més a més, les tècniques de modelat es dirigeixen cap a models dependents de la predicibilitat del segment que introdueixen la predicibilitat com a variable discriminant per a models dependents de la classe de variabilitat basats en la coherència fonamental entre predicibilitat i reducció acústica que es suggereix pel principi del mínim esforç i la teoria de la redundància. Per tant, emfatitza la predicibilitat de les paraules i dels fonemes. Aquesta aproximació no suposa cap millora significativa de rendiment. El discurs planejat és aparentment antagònic amb el principi del mínim esforç. Per tant, un anàlisi previ condicionat al segment de la reducció acústica pot indicar el seu grau mig de reducció, mentre la variació intra-segmental pot indicar si exhibeix prou relaxació en l'estil de parlar per adoptar el principi del mínim esforç. Per tant, segments amb poca variació intra-segmental poden ser modelats apart dels que tenen gran variació intra-segmental, mentre que modelar aquestes darreres mitjançant models dependents de predicibilitat de paraula, síl·laba o fonema poden aportar una perspectiva viable de recerca.

Conversion from text to speech relies on the accurate mapping from linguistic to acoustic symbol sequences, for which current practice employs recurrent statistical models such as recurrent neural networks. Despite the good performance of such models (in terms of low distortion in the generated speech), their recursive structure with intermediate affine transformations tends to make them slow to train and to sample from. In this work, we explore two different mechanisms that enhance the operational efficiency of recurrent neural networks, and study their performance&ndash, speed trade-off. The first mechanism is based on the quasi-recurrent neural network, where expensive affine transformations are removed from temporal connections and placed only on feed-forward computational directions. The second mechanism includes a module based on the transformer decoder network, designed without recurrent connections but emulating them with attention and positioning codes. Our results show that the proposed decoder networks are competitive in terms of distortion when compared to a recurrent baseline, whilst being significantly faster in terms of CPU and GPU inference time. The best performing model is the one based on the quasi-recurrent mechanism, reaching the same level of naturalness as the recurrent neural network based model with a speedup of 11.2 on CPU and 3.3 on GPU.

Restricted Boltzmann Machines (RBMs) have shown success in both the front-end and backend of speaker verification systems. In this paper, we propose applying RBMs to the front-end for the tasks of speaker clustering and speaker tracking in TV broadcast shows. RBMs are trained to transform utterances into a vector based representation. Because of the lack of data for a test speaker, we propose RBM adaptation to a global model. First, the global model&mdash, which is referred to as universal RBM&mdash, is trained with all the available background data. Then an adapted RBM model is trained with the data of each test speaker. The visible to hidden weight matrices of the adapted models are concatenated along with the bias vectors and are whitened to generate the vector representation of speakers. These vectors, referred to as RBM vectors, were shown to preserve speaker-specific information and are used in the tasks of speaker clustering and speaker tracking. The evaluation was performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed speaker clustering system gained up to 12% relative improvement, in terms of Equal Impurity (EI), over the baseline system. On the other hand, in the task of speaker tracking, our system has a relative improvement of 11% and 7% compared to the baseline system using cosine and Probabilistic Linear Discriminant Analysis (PLDA) scoring, respectively.

Alzheimer's disease is the most prevalent form of progressive degenerative dementia; it has a high socio-economic impact in Western countries. Therefore it is one of the most active research areas today. Alzheimer's is sometimes diagnosed by excluding other dementias, and definitive confirmation is only obtained through a post-mortem study of the brain tissue of the patient. The work presented here is part of a larger study that aims to identify novel technologies and biomarkers for early Alzheimer's disease detection, and it focuses on evaluating the suitability of a new approach for early diagnosis of Alzheimer’s disease by non-invasive methods. The purpose is to examine, in a pilot study, the potential of applying Machine Learning algorithms to speech features obtained from suspected Alzheimer sufferers in order help diagnose this disease and determine its degree of severity. Two human capabilities relevant in communication have been analyzed for feature selection: Spontaneous Speech and Emotional Response. The experimental results obtained were very satisfactory and promising for the early diagnosis and classification of Alzheimer’s disease patients.

The Speech Emotion Recognition project verses about the design and implementation of an emotion recognition system by analyzing the characteristics of the voice signal, based on the use of the Ivectors technique, which is considered the state of the art in the area in treatment. The use of this technique is compared with two baseline systems (Gaussian Mixture Models and Speaker factors) and it is shown that the degree of recognition improves. The aim of this project is to recognize, with as much precision as possible, a set of 7 emotions, available in the INTERFACE database, partially developed by UPC. To achieve this, we design a series of experiments that are used to test the performance of the system under different situations. El proyecto Speech Emotion Recognition trata del diseño y la implementación de un sistema de reconocimiento de emociones mediante el análisis de las características de la señal de voz, basado en el uso de la técnica de Ivectors, que se considera el estado del arte del área en cuestión. El uso de esta técnica se compara con dos sistemas base (Gaussian Mixture Modelos y Speaker factores) y se demuestra que el grado de reconocimiento mejora. El objetivo de este proyecto es el de reconocer con la máxima precisión posible un conjunto de 7 emociones, disponibles en la base de datos INTERFACE, desarrollada parcialmente por la UPC. Para lograrlo, se diseñan una serie de experimentos que se utilizan para probar el rendimiento del sistema bajo diferentes casuísticas. El projecte Speech Emotion Recognition tracta del disseny i la implementació d'un sistema de reconeixement d'emocions mitjançant l'anàlisi de les característiques de la senyal de veu, basat en l'ús de la tècnica d'Ivectors, que es considera l'estat de l'art de l'àrea en qüestió. L'ús d'aquesta tècnica es compara amb dos sistemes base (Gaussian Mixture Models i Speaker factors) i es demostra que el grau de reconeixement millora. L'objectiu d'aquest projecte és el de reconèixer amb la màxima precisió possible un conjunt de 7 emociones, disponibles en la base de dades INTERFACE, desenvolupada parcialment per la UPC. Per a assolir-lo, es dissenyen un seguit d'experiments que s'utilitzen per probar el rendiment del sistema sota diferents casuístiques.

Speech synthesis is the task of generating speech using computers. Due to the limitations of classical techniques, these systems are normally not suitable for applications that would benefit from expressiveness in the speech, such as audiobook reading. In this project, we attempt to develop a text-to-speech speech synthesizer that is capable of reacting to the semantic content of the input text to produce expressive speech. The system is based on the Socrates text-to-speech framework developed in the VEU research lab at UPC and the Keras deep learning library. La sintesis de voz consiste en utilizar ordenadores para generar voz humana. Debido a las limitaciones de las técnicas clásicas, estos sitemas normalmente no son adecuados para aplicaciones que requieren voz expresiva como en la lectura automática de audiolibros. En este proyecto, tratamos de desarrollar un sintetizador de voz capaz de reaccionar al contenido semántico del texto para producir voz expresiva. El sistema está basado en el framework de síntesis de voz Socrates, desarrollado en el grupo VEU de la UPC, y en la librería de deep learning Keras. La síntesi de veu consisteix en fer servir ordinadors per generar veu humana. Degut a les limitacions de les tècniques clàssiques, aquests sistemes normalment no són adequats per aplicacions que requireixen veu expressiva com és el cas de la lectura de audiollibres automàtica. En aquest projecte, desenvolupem un sintetitzador de veu capaç de reaccionar al contingut semàntic del text per produir veu expressiva. El sistema està basat en el framework de síntesi de veu Socrates, desenvolupat al grup de recerca VEU de la UPC, i en la llibreria de deep learning Keras.

Speaker diarization has received several research attentions over the last decade. Among the different domains of speaker diarization, diarization in meeting domain is the most challenging one. It usually contains spontaneous speech and is, for example, susceptible to reverberation. The appropriate selection of speech features is one of the factors that affect the performance of speaker diarization systems. Mel Frequency Cepstral Coefficients (MFCC) are the most widely used short-term speech features in speaker diarization. Other factors that affect the performance of speaker diarization systems are the techniques employed to perform both speaker segmentation and speaker clustering. In this thesis, we have proposed the use of jitter and shimmer long-term voice-quality features both for Gaussian Mixture Modeling (GMM) and i-vector based speaker diarization systems. The voice-quality features are used together with the state-of-the-art short-term cepstral and long-term speech ones. The long-term features consist of prosody and Glottal-to-Noise excitation ratio (GNE) descriptors. Firstly, the voice-quality, prosodic and GNE features are stacked in the same feature vector. Then, they are fused with cepstral coefficients at the score likelihood level both for the proposed Gaussian Mixture Modeling (GMM) and i-vector based speaker diarization systems. For the proposed GMM based speaker diarization system, independent HMM models are estimated from the short-term and long-term speech feature sets. The fusion of the short-term descriptors with the long-term ones in speaker segmentation is carried out by linearly weighting the log-likelihood scores of Viterbi decoding. In the case of speaker clustering, the fusion of the short-term cepstral features with the long-term ones is carried out by linearly fusing the Bayesian Information Criterion (BIC) scores corresponding to these feature sets. For the proposed i-vector based speaker diarization system, the speaker segmentation is carried out exactly the same as in the previously mentioned GMM based speaker diarization system. However, the speaker clustering technique is based on the recently introduced factor analysis paradigm. Two set of i-vectors are extracted from the speaker segmentation hypothesis. Whilst the first i-vector is extracted from short-term cepstral features, the second one is extracted from the voice quality, prosody and GNE descriptors. Then, the cosine-distance and Probabilistic Linear Discriminant Analysis (PLDA) scores of i-vectors are linearly weighted to obtain a fused similarity score. Finally, the fused score is used as speaker clustering distance. We have also proposed the use of delta dynamic features for speaker clustering. The motivation for using deltas in clustering is that delta dynamic features capture the transitional characteristics of the speech signal which contain speaker specific information. This information is not captured by the static cepstral coefficients. The delta features are used together with the short-term static cepstral coefficients and long-term speech features (i.e., voice-quality, prosody and GNE) both for GMM and i-vector based speaker diarization systems. The experiments have been carried out on Augmented Multi-party Interaction (AMI) meeting corpus. The experimental results show that the use of voice-quality, prosody, GNE and delta dynamic features improve the performance of both GMM and i-vector based speaker diarization systems., La diarización del altavoz ha recibido varias atenciones de investigación durante la última década. Entre los diferentes dominios de la diarización del hablante, la diarización en el dominio del encuentro es la más difícil. Normalmente contiene habla espontánea y, por ejemplo, es susceptible de reverberación. La selección apropiada de las características del habla es uno de los factores que afectan el rendimiento de los sistemas de diarización de los altavoces. Los Coeficientes Cepstral de Frecuencia Mel (MFCC) son las características de habla de corto plazo más utilizadas en la diarización de los altavoces. Otros factores que afectan el rendimiento de los sistemas de diarización del altavoz son las técnicas empleadas para realizar tanto la segmentación del altavoz como el agrupamiento de altavoces. En esta tesis, hemos propuesto el uso de jitter y shimmer características de calidad de voz a largo plazo tanto para GMM y i-vector basada en sistemas de diarización de altavoces. Las características de calidad de voz se utilizan junto con el estado de la técnica a corto plazo cepstral y de larga duración de habla. Las características a largo plazo consisten en la prosodia y los descriptores de relación de excitación Glottal-a-Ruido (GNE). En primer lugar, las características de calidad de voz, prosódica y GNE se apilan en el mismo vector de características. A continuación, se fusionan con coeficientes cepstrales en el nivel de verosimilitud de puntajes tanto para los sistemas de diarización de altavoces basados ¿¿en el modelo Gaussian Mixture Modeling (GMM) como en los sistemas basados ¿¿en i-vector. . Para el sistema de diarización de altavoces basado en GMM propuesto, se calculan modelos HMM independientes a partir de cada conjunto de características. En la segmentación de los altavoces, la fusión de los descriptores a corto plazo con los de largo plazo se lleva a cabo mediante la ponderación lineal de las puntuaciones log-probabilidad de decodificación Viterbi. En la agrupación de altavoces, la fusión de las características cepstrales a corto plazo con las de largo plazo se lleva a cabo mediante la fusión lineal de las puntuaciones Bayesian Information Criterion (BIC) correspondientes a estos conjuntos de características. Para el sistema de diarización de altavoces basado en un vector i, la fusión de características se realiza exactamente igual a la del sistema basado en GMM antes mencionado. Sin embargo, la técnica de agrupación de altavoces se basa en el paradigma de análisis de factores recientemente introducido. Dos conjuntos de i-vectores se extraen de la hipótesis de segmentación de altavoz. Mientras que el primer vector i se extrae de características espectrales a corto plazo, el segundo se extrae de los descriptores de calidad de voz apilados, prosódicos y GNE. A continuación, las puntuaciones de coseno-distancia y Probabilistic Linear Discriminant Analysis (PLDA) entre i-vectores se ponderan linealmente para obtener una puntuación de similitud fundida. Finalmente, la puntuación fusionada se utiliza como distancia de agrupación de altavoces. También hemos propuesto el uso de características dinámicas delta para la agrupación de locutores. La motivación para el uso de deltas en la agrupación es que las características dinámicas delta capturan las características de transición de la señal de voz que contienen información específica del locutor. Esta información no es capturada por los coeficientes cepstrales estáticos. Las características delta se usan junto con los coeficientes cepstrales estáticos a corto plazo y las características de voz a largo plazo (es decir, calidad de voz, prosodia y GNE) tanto para sistemas de diarización de altavoces basados en GMM como en sistemas i-vector. Los resultados experimentales sobre AMI muestran que el uso de calidad vocal, prosódica, GNE y dinámicas delta mejoran el rendimiento de los sistemas de diarización de altavoces basados en GMM e i-vector.

In this paper, we describe the TALP- UPC participation in the News Task for German-English and Finish-English. Our primary submission implements a fully character to character neural machine translation architecture with an additional rescoring of a n-best list of hypothesis us- ing a forced back-translation to the source sentence. This model gives consistent im- provements on different pairs of languages for the language direction with the low- est performance while keeping the qual- ity in the direction with the highest perfor- mance. Additional experiments are reported for multilingual character to character neural machine translation, phrase-based trans- lation and the additional Turkish-English language pair.

Literature review on prosody reveals the lack of corpora for prosodic studies in Catalan and Spanish. In this paper, we present a corpus intended to fill this gap. The corpus comprises two distinct data-sets, a news subcorpus and a dialogue subcorpus, the latter containing either conversational or task-oriented speech. More than 25 h were recorded by twenty eight speakers per language. Among these speakers, eight were professional (four radio news broadcasters and four advertising actors). The entire material presented here has been transcribed, aligned with the acoustic signal and prosodically annotated. Two major objectives have guided the design of this project: (i) to offer a wide coverage of representative real-life communicative situations which allow for the characterization of prosody in these two languages; and (ii) to conduct research studies which enable us to contrast the speakers different speaking styles and discursive practices. All material contained in the corpus is provided under a Creative Commons Attribution 3.0 Unported License.

With the increasingly global nature of our everyday interactions, the need for multilingual technologies to support efficient and efective information access and communication cannot be overemphasized. Computational modeling of language has been the focus of Natural Language Processing, a subdiscipline of Artificial Intelligence. One of the current challenges for this discipline is to design methodologies and algorithms that are cross-language in order to create multilingual technologies rapidly. The goal of this JAIR special issue on Cross-Language Algorithms and Applications (CLAA) is to present leading research in this area, with emphasis on developing unifying themes that could lead to the development of the science of multi- and cross-lingualism. In this introduction, we provide the reader with the motivation for this special issue and summarize the contributions of the papers that have been included. The selected papers cover a broad range of cross-lingual technologies including machine translation, domain and language adaptation for sentiment analysis, cross-language lexical resources, dependency parsing, information retrieval and knowledge representation. We anticipate that this special issue will serve as an invaluable resource for researchers interested in topics of cross-lingual natural language processing.

This work presents a study on the suitability of prosodic andacoustic features, with a special focus on i-vectors, in expressivespeech analysis and synthesis. For each utterance of two dif-ferent databases, a laboratory recorded emotional acted speech,and an audiobook, several prosodic and acoustic features are ex-tracted. Among them, i-vectors are built not only on the MFCCbase, but also on F0, power and syllable durations. Then, un-supervised clustering is performed using different feature com-binations. The resulting clusters are evaluated calculating clus-ter entropy for labeled portions of the databases. Additionally,synthetic voices are trained, applying speaker adaptive training,from the clusters built from the audiobook. The voices are eval-uated in a perceptual test where the participants have to edit anaudiobook paragraph using the synthetic voices.The objective results suggest that i-vectors are very use-ful for the audiobook, where different speakers (book charac-ters) are imitated. On the other hand, for the laboratory record-ings, traditional prosodic features outperform i-vectors. Also,a closer analysis of the created clusters suggest that differentspeakers use different prosodic and acoustic means to conveyemotions. The perceptual results suggest that the proposed i-vector based feature combinations can be used for audiobookclustering and voice training.

Recogniton of peple by speech and face in TV shows. Participation in the international competition Meiaeval 2016. The Project described in this document falls within the topic of person recognition in TV recordings by mean of multimodal systems. It has been developed as collaboration with image and audio processing groups in the signal theory department in UPC. Thus, it is a project about the development of a person recognition system in TV Broadcast videos implemented for the participation in the Whorkshop Mediaeval 2016. The aim of the competition consist in find the names of the people that appears and talks in each shot of different videos of a given database. This discovery should be done in totally unsupervised manner using only the information in each shot as image, audio or text. For this purpose, it has been proposed to implement the fusion of three monomodal algorithms. These technologies will process the information present in the text, image and audio in independent manner. Information after monomodal technologies will be fused with objective of create e a multimodal algorithm able to tag the shots in the database. Finally, this thesis is centered in the monomodal audio algorithm development for which it has been proposed a tracking system based in i-vectors. El proyecto descrito en este documento se enmarca en el tópico del descubrimiento de personas en vídeos de televisión mediante sistemas multimodales. Forma parte de una colaboración con los grupos de procesado de audio e imagen del departamento de Teoría de señal. Así pues, se trata del desarrollo de un sistema de descubrimiento de personas en un entorno de televisión broadcast para la participación en un Workshop denominado Mediaeval 2016. El reto de la competición consiste en encontrar los nombres de las personas que aparecen hablando en cada escena de los diferentes shows de un conjunto de videos. Dicho descubrimiento debe realizarse de manera totalmente no supervisada, utilizando únicamente la información presente en cada escena, como la imagen, el audio o el texto. Para ello, se propone utilizar la fusión de tres algoritmos monomodales que procesen de manera independiente el texto, la imagen y el audio, para lograr un sistema multimodal capaz de etiquetar cada escena dada una base de datos. Finalmente, el presente proyecto se centrará en el desarrollo del algoritmo de audio a nivel monomodal, para el cual se ha propuesto la implementación de un sistema de tracking basado en i-vectors.