Search

We live in an era where intelligent systems are becoming more and more part of our lives. These systems require a large amount of data to learn different tasks and, in many cases, not enough content is available to train them. Data augmentation stems from the necessity of creating, in an artificial way, new data similar to the original to help in better training. It applies to many types of intelligent systems, although this project focuses on data augmentation for speech. In this work, a computer program, capable of processing large databases and expanding their content using different techniques, will be presented. Vivimos en una era donde los sistemas inteligentes forman cada vez más parte de nuestra vida. Estos sistemas necesitan una gran cantidad de datos para conseguir aprender diferentes tareas y, en muchos casos, no se dispone de suficiente contenido para entrenarlos. El aumento de datos nace de la necesidad de crear artificialmente nuevos datos, similares a los originales, que ayuden a hacer un mejor entrenamiento. Es aplicable a muchos tipos de sistemas inteligentes, pero este trabajo se centra en el aumento de datos de voz. En él se presentará un programa informático capaz de procesar grandes bases de datos y ampliar su contenido mediante diferentes técnicas. Vivim en una era on els sistemes intel·ligents formen cada cop més part de la nostra vida. Aquests sistemes necessiten una gran quantitat de dades per aconseguir aprendre diferents tasques i, en molts casos, no es disposa de suficient contingut per entrenar-los. L'augment de dades neix de la necessitat de crear artificialment noves dades, similars a les originals, que ajudin a fer un millor entrenament. És aplicable a molts tipus de sistemes intel·ligents, però aquest treball se centra en l'augment de dades de veu. En aquest es presentarà un programa informàtic capaç de processar grans bases de dades i ampliar-ne el contingut mitjançant diferents tècniques.

Advances in digital health and phenotyping technologies are crucial to ensureincreased access to high-quality mental health support services and treatment. Speech is uniquely placed in this regard, as no other mobile health signal contains its singular combination of cognitive, neuromuscular and physiological information. It is this complexity which makes speech a suitable marker for different mentalhealth conditions.However, much research exploring links between speech and depression is limited, and co-morbidities with conditions such as anxiety have not been exploited to help improve machine learning models.The purpose of this project is to jointly learn depression and anxietydirectly from speech signals.For this project, speech signals were split into segments that were converted into Mel-spectrograms. Automatic feature extraction was performed using a CNN-LSTM model that can classify into5 severities ofdepression. With transfer learning, this model was then usedas a pre-trained model for other tasks, such as classifying speech signals into different 4 severities of anxiety or improving modelsfor both co-morbiditiesin different languages. Finally, a Multi-Task learning model is used to jointly detect depression and anxiety. Models that use transfer learning to detectanxiety achieve an improvement from 67% to 72% of accuracy, while multi-Task learning models achieve an accuracy of 71% for both co-morbidities, anxiety and depression. The experiments show promising results, discussing the viability of jointly detecting mental health conditions such as depression and anxiety as well as exploiting the viability of using models pre-trained for just one condition, language or task to fine-tune a model for another condition, language or task, demonstrating that co-morbidities can help to improve models for joint learning severities directly from speech signals.

In order to analyze and extract information about social trends in the Spanish and Portuguese environment from an objective point of view, the implementation of this project was requested. This consists of extracting information from different and varied sources of internet information through the web scraping technique, and then using artificial intelligence techniques to process the texts obtained and extract keywords. Finally, two different ways of presenting the obtained results have been created in order to extract as much insights as possible from them. Con el fin de analizar y extraer información sobre las tendencias sociales en el entorno español y portugués desde un punto de vista objetivo, se solicitó la implementación de este proyecto. Este consiste en extraer información de diferentes y variadas fuentes de información de internet a través de la técnica del "web scraping", y posteriormente utilizar técnicas de inteligencia artificial para procesar los textos obtenidos y extraer palabras clave. Por último, se han creado dos formas diferentes de presentar los resultados obtenidos, con el fin de extraer de ellos la mayor cantidad de información posible. Per tal d'analitzar i extreure informació sobre les tendències socials de l'entorn espanyol i portuguès des d'un punt de vista objectiu, es va sol·licitar la implementació d'aquest projecte. Aquest consisteix en extreure informació de diferents i variades fonts d'informació d'internet mitjançant la tècnica del "web scraping", i després utilitzar tècniques d'intel·ligència artificial per processar els textos obtinguts i extreure'n paraules clau. Finalment, s'han creat dues maneres diferents de presentar els resultats obtinguts, per tal d'obtenir-ne el màxim d'informació possible.

Common Voice és un projecte que convida a la gent a donar la seva veu en la seva llengua materna amb el propòsit de reunir corpus de veus. Aquests corpus, disponibles en més de noranta idiomes, estan formats per una gran quantitat d’àudios curts, en els quals els usuaris es graven llegint una frase, i es poden utilitzar per millorar els sistemes de reconeixement i síntesi de parla. Precisament en català la majoria d’aquests sistemes actualment necessiten millores en els seus models i la base de dades de Common Voice pot arribar a ser de gran utilitat. Un dels seus avantatges és l'accessibilitat per a tothom, tant per la gent que vol desenvolupar sistemes de reconeixement com per la que vol proporcionar la seva veu. Per aquest motiu, aquest projecte utilitza el corpus català amb la intenció de validar-lo, intentant generar una segmentació fonètica dels àudios i posteriorment analitzant la precisió de les etiquetes, amb l’objectiu de poder contribuir en el desenvolupament de les tecnologies de la parla, a més d’adquirir coneixements sobre el funcionament de gestió de dades i entrenament de models amb xarxes neuronals. Inicialment, es pretenia processar la base de dades completa, formada per 25 GB de dades i més de 1000 hores gravades, però això requereix una gran quantitat de temps i de recursos computacionals, així que s’ha optat per seleccionar un conjunt reduït de dades i treballar a menor escala. El procediment ha sigut realitzar un alineament forçat amb els àudios i les seves transcripcions amb els programes Festival i Montreal Forced Aligner. Per aquest últim s’ha necessitat entrenar un model acústic, utilitzant el mateix corpus de Common Voice però amb diferents arxius. Finalment s’ha analitzat la precisió temporal de les etiquetes resultants, en les que s’ha vist que no eren del tot perfectes. Common Voice es un proyecto que invita a la gente a donar su voz en su lengua materna con el propósito de reunir corpus de voz. Estos corpus, disponibles en más de noventa idiomas, estan formados por una gran cantidad de audios cortos, en los que los usuarios se graban leyendo una frase, y se pueden utilizar para mejorar los sistemes de reconocimiento y síntesi del habla. Precisamente en catalán la mayoría de estos sistemas actualmente necesitan mejoras en sus models y la base de datos de Common Voice puede llegar a ser de gran utilidad. Una de sus ventajas es la accesibilidad para todo el mundo, tanto para la gente que quiere desarrollar sistemas de reconocimiento como para la que quiere proporcionar su voz. Por eso, este proyecto utiliza el corpus catalán con la intención de validarlo, intentando generar una segmentación fonética de los audios y posteriormente analizando la precisión de las etiquetas, con el objetivo de poder contribuir con el desarrollo de las tecnologías del habla, además de adquirir conocimientos sobre el funcionamiento de gestión de datos y entrenamiento de modelos con redes neuronales. Inicialmente, se pretendía procesar la base de datos completa, formada por 25 GB de datos y más de 1000 horas grabadas, pero esto requiere una gran cantidad de tiempo y de recursos computacionales, así que se ha optado por seleccionar un conjunto reducido de datos y trabajar a menor escala. El procedimiento ha sido realizar un alineamiento forzado con los audios y sus transcripciones con los programas Festival y Montreal Forced Aligner. Para este último se ha necesitado entrenar un modelo acústico, usando el mismo corpus de Common Voice pero con diferentes archivos. Finalmente se ha analizado la precisión temporal de las etiquetas resultantes, en las que se ha visto que no eran del todo perfectas. Common Voice is a project that invites people to donate their voice in their mother tongue in order to collect speech corpora. These corpora, available in more than ninety languages, consist of a large quantity of short audio files, in which users record themselves reading a sentence, and can be used to improve speech recognition and synthesis systems. In catalan, most of these systems currently need improvement in their models and the Common Voice dataset may be very helpful. One of its advantages is the accessibility for everyone, both for people who want to develop speech recognition systems and for people who want to provide their voice data. For this reason, this project uses the catalan corpus with the purpose of validating it, generating a phonetic segmentation of the audios and subsequently analyzing the labeling precision, with the aim of being able to contribute in the development of speech technologies, as well as gaining knowledge about data management and model training. The initial idea was processing the whole dataset, consisting of 25 GB of data and more than 1000 hours of recording, but this requires a large amount of time and computing resources, so a later choice was selecting a reduced set of audios and work on a smaller scale. The procedure was performing forced alignment on the audios and their orthographic transcriptions with Festival and Montreal Forced Aligner. For the latter, the training of a new acoustic model was needed, using the same Common Voice corpus but with different files. Finally the accuracy of the labels was analyzed, but the results showed that they were not entirely accurate.

La caracterización de un locutor es una de las tareas más relevantes en muchas aplicaciones de inteligencia artificial. Del mismo modo que estas tecnologías mejoran y se aumenta la cantidad de datos disponibles, es también importante adaptarlas a distintos idiomas y contextos. En este proyecto, se propone una red neuronal para clasificar el género, edad y acento de un locutor en catalán. Distintas variaciones serán exploradas, incluyendo algunas t´ecnicas punteras como el mecanismo de pooling basado en Double Multi-Head Attention. Por otro lado, se llevará a cabo análisis de datos con tal de obtener los mejores resultados posibles, incluyendo la aplicación de técnicas nivel estado del arte de aumento de datos. Algunos de los resultados son prometedores, al demostrar una considerable mejora respecto otras técnicas clásicas no basadas en aprendizaje automático. La caracterització d'un locutor és una de les tasques més rellevants en moltes aplicacions d'intel·ligència artificial. Tan bon punt s'afinen aquestes tecnologies i augmenta la quantitat de dades disponibles, és important adaptar-les a diferents idiomes i contexts. En aquest projecte, es proposa una xarxa neuronal per classificar el gènere, l'edat i l'accent d'un parlant de català. Vàries variacions dels blocs més convencionals seran explorades, incloent algunes tècniques punteres com un mecanisme de pooling basat en Double Multi- Head Attention. Per altra banda, es durà a terme anàlisis de dades per tal de millorar els resultats obtinguts, incloent l'aplicació de tècniques estat de l'art d'augment de dades. Alguns dels resultats són força prometedors, al demostrar una considerable millora respecte altres tècniques clàssiques no basades en l'aprenentatge automàtic. Speech characterization is one of the most relevant tasks in a lot of voice-related artificial intelligence applications. As these technologies thrive and the amount of data available increases, it is also salient their adaptation to different languages and contexts. In this project, a network to classify the gender, age and accent of a Catalan speaker through their voice is proposed. Different variations of the main models blocks are going to be explored, including some innovative techniques as the Double Multi-Head Attention pooling mechanism. In addition, some data analysis- including the application of some state-of-art voice data augmentation techniques- will be done aiming for better results. Some results show strong promise, as they indicate improvement in comparison to some classical methods not based on machine learning.

The aim of this paper is to investigate the benefit of combining both language and acoustic modelling for speaker diarization. Although conventional systems only use acoustic features, in some scenarios linguistic data contain high discriminative speaker information, even more reliable than the acoustic ones. In this study we analyze how an appropriate fusion of both kind of features is able to obtain good results in these cases. The proposed system is based on an iterative algorithm where a LSTM network is used as a speaker classifier. The network is fed with character-level word embeddings and a GMM based acoustic score created with the output labels from previous iterations. The presented algorithm has been evaluated in a Call-Center database, which is composed of telephone interview audios. The combination of acoustic features and linguistic content shows a 84.29% improvement in terms of a word-level DER as compared to a HMM/VB baseline system. The results of this study confirms that linguistic content can be efficiently used for some speaker recognition tasks. This work was partially supported by the Spanish Project DeepVoice (TEC2015-69266-P) and by the project PID2019-107579RBI00/ AEI /10.13039/501100011033.

Statistical learning has been proposed as a mechanism to structure and segment the continuous flow of information in several sensory modalities. Previous studies proposed that the medial temporal lobe, and in particular the hippocampus, may be crucial to parse the stream in the visual modality. However, the involvement of the hippocampus in auditory statistical learning, and specifically in speech segmentation is less clear. To explore the role of the hippocampus in speech segmentation based on statistical learning, we exposed seven pharmaco-resistant temporal lobe epilepsy patients to a continuous stream of trisyllabic pseudowords and recorded intracranial stereotaxic electro-encephalography (sEEG). We used frequency-tagging analysis to quantify neuronal synchronization of the hippocampus and auditory regions to the temporal structure of words and syllables of the stream. Results show that while auditory regions highly respond to syllable frequency, the hippocampus responds mostly to word frequency. These findings provide direct evidence of the involvement of the hippocampus in speech segmentation process and suggest a hierarchical organization of auditory information during speech processing., Cortex, Elsevier, 2022

Text-to-speech (TTS) systems provide fundamental reading support for people with aphasia and reading difficulties. However, artificial voices are more difficult to process than natural voices. The current study is an extended analysis of the results of a clinical experiment investigating which, among three artificial voices and a digitised human voice, is more suitable for people with aphasia and reading impairments. Such results show that the voice synthesised with Ogmios TTS, a concatenative speech synthesis system, caused significantly slower reaction times than the other three voices used in the experiment. The present study explores whether and what voice quality metrics are linked to delayed reaction times. For this purpose, the voices were analysed using an automatic assessment of intelligibility, naturalness, and jitter and shimmer voice quality parameters. This analysis revealed that Ogmios TTS, in general, performed worse than the other voices in all parameters. These observations could explain the significantly delayed reaction times in people with aphasia and reading impairments when listening to Ogmios TTS and could open up consideration about which TTS to choose for compensative devices for these patients based on the voice analysis of these parameters.

Un projecte de tecnologia de la parla en català el qual fa servir la base de dades del Common Voice, de Mozilla, en català a partir d’automatitzar cerques d’interès respecte tota la informació que emmagatzema i crear una segmentació fonètica per crear una nova veu, i per tal de permetre la creació de programes de reconeixement i síntesi de veu posteriors. Un proyecto de tecnología del habla en catalán que utiliza la base de datos del Common Voice, de Mozilla, en catalán a partir de automatizar búsquedas de interés respecto toda la información que almacena y crear una segmentación fonética para la creación de una nueva voz, y útil para permitir la creación de programas de reconocimiento y síntesis de voz posteriores. A Catalan speech technology project that uses Mozilla's Common Voice database in Catalan to automate searches of interest for all the information that it stores and create a phonetic segmentation to create a new voice, and to allow the creation of subsequent speech recognition and synthesis programs.

Deep learning has aroused over the past years as a promising technique in AI. Implementing algorithms that allow systems to learn with applications in several fields has made a difference with the conventional rule coding that had multiple limitations. Since successful experiments within this scope have taken place, the limits of the development of intelligent machines seem to have faded out. However, this is still not a flawless procedure: there is a great number of improvements ahead to be performed in order to obtain better results. One of these issues is the presence of stereotypes in large data sets, a generalized problem in many applications of Deep Learning. In the case of Neural Machine Translation, this phenomenon leads to gender-biased inaccurate translations. In this thesis, our commitment is to work on developing a modification of the attention mechanism (within the architecture of NMT) that can mitigate such bias. Attention controls the amount of context that is used in Neural Machine Translation. The motivation of this work is that the attention mechanism for different types of words, e.g. words with different amounts of dependencies, should be different. The concentration or diversion of context information both at encoding and decoding levels may differ depending on the studied biases and language. We study the effect of controlling this amount of attention both by modifying the attention mechanism at the encoding and decoding level, for self and inter-attention. We evaluate this for the case of mitigating biases in a multilingual setting on a recent NMT architecture: the Transformer. The results show a similar quality of translation as the baseline model in terms of accuracy, but outperforms the reference model in nearly all gender-bias indicators, yielding results that are encouraging for continuing exploring this line of research. El aprendizaje profundo ha surgido en los últimos años como una técnica prometedora en IA. La implementación de algoritmos que permiten que los sistemas aprendan con aplicaciones en varios campos ha marcado la diferencia con la codificación de reglas convencional que tenía múltiples limitaciones. Desde que se han llevado a cabo experimentos exitosos dentro de este ámbito, los límites del desarrollo de las máquinas inteligentes parecen haberse desvanecido. Sin embargo, este todavía no es un procedimiento impecable: hay un gran número de mejoras por realizar para obtener mejores resultados. Uno de estos problemas es la presencia de estereotipos en grandes conjuntos de datos, un problema generalizado en muchas aplicaciones de Deep Learning. En el caso de la traducción automática neuronal, este fenómeno conduce a traducciones inexactas con sesgo de género. En esta tesis, nuestro compromiso es trabajar en el desarrollo de una modificación del mecanismo de atención (dentro de la arquitectura de NMT) que pueda mitigar dicho sesgo. La atención controla la cantidad de contexto que se utiliza en la traducción automática neuronal. La motivación de este trabajo es que el mecanismo de atención para diferentes tipos de palabras, por ejemplo palabras con diferentes cantidades de dependencias, deberían ser diferentes. La concentración o el desvío de la información del contexto, tanto a nivel de codificación como de decodificación, puede diferir dependiendo de los sesgos y el lenguaje estudiados. Estudiamos el efecto de controlar esta cantidad de atención modificando el mecanismo de atención a nivel de codificación y decodificación, para la atención propia y la atención cruzada. Evaluamos esto para el caso de mitigar los sesgos en un entorno multilingüe en una arquitectura NMT reciente: el Transformer. Los resultados muestran una calidad de traducción similar al modelo de línea de base en términos de precisión, pero supera al modelo de referencia en casi todos los indicadores de sesgo de género, arrojando resultados que son alentadores para seguir explorando esta línea de investigación. L’aprenentatge profund ha sorgit durant els darrers anys com una tècnica prometedora en IA. La implementació d’algoritmes que permeten els sistemes aprendre amb aplicacions en diversos camps ha marcat la diferència amb la codificació de regles convencionals que tenia múltiples limitacions. Atès que s'han dut a terme experiments amb èxit dins d'aquest àmbit, els límits del desenvolupament de màquines intel·ligents semblen haver esvaït. Tot i això, aquest no és encara un procediment impecable: hi ha un gran nombre de millores per assolir per obtenir millors resultats. Un d’aquests problemes és la presència d’estereotips en grans conjunts de dades, un problema generalitzat en moltes aplicacions de Deep Learning. En el cas de la traducció automàtica neuronal, aquest fenomen condueix a traduccions inexactes esbiaixades en quant a gènere. En aquesta tesi, el nostre compromís és treballar en el desenvolupament d’una modificació del mecanisme d’atenció (dins de l’arquitectura de NMT) que pugui mitigar aquest biaix. L’atenció controla la quantitat de context que s’utilitza a la traducció automàtica de neurones. La motivació d’aquest treball és que el mecanisme d’atenció de diferents tipus de paraules, com per exemple, les paraules amb diferents quantitats de dependències, haurien de ser diferents. La concentració o diversificació de la informació de context tant a nivells de codificació com de descodificació pot variar en funció dels biaixos i del llenguatge estudiats. Estudiem l’efecte de controlar aquesta quantitat d’atenció modificant el mecanisme d’atenció a nivell de codificació i descodificació, per a l’interès i la interatenció. Ho avaluem per al cas de biaixos mitigadors en un entorn multilingüe sobre una arquitectura NMT recent: el transformador. Els resultats mostren una qualitat de traducció similar a la del model de referència en termes de precisió, però supera el model de referència en gairebé tots els indicadors de biaix de gènere, donant resultats que són encoratjadors per continuar explorant aquesta línia d’investigació

Speech Emotion Recognition (SER) has recently become a popular field of research because of its implications in human-computer interaction. In this study, the emotional state of the speaker is successfully predicted by using Deep Convolutional Neural Networks to automatically extract features from the spectrogram of a speech signal. Parting from a baseline model that uses a statistical approach to pooling, an alternative method is proposed by incorporating Attention mechanisms as a pooling strategy. Additionally, multi-task learning is explored as an improvement over the baseline model by assigning language recognition as an auxiliary task. The final results show a remarkable improvement in classification accuracy in respect to previous more conventional techniques, in particular Gaussian Mixture Models and i-vectors, as well as a notable improvement in performance of the proposed Attention mechanisms over statistical pooling. En las últimas décadas, Speech Emotion Recognition (SER), o el reconocimiento de emociones por voz, ha generado un fuerte interés en el ámbito del tratamiento del habla por sus implicaciones en la interacción humano-computador. En este trabajo, se consigue reconocer el estado emocional del hablante mediante redes convolucionales profundas, capaces de extraer de manera automática características contenidas en el espectrograma de la señal de voz. Partiendo de un modelo que utiliza análisis estadístico para pooling, se propone una estrategia alternativa para mejorar el rendimiento incorporando mecanismos de Atención. Como mejora añadida, se explora el campo del multi-task learning definiendo el reconocimiento del idioma como tasca auxiliar para el modelo. Los resultados obtenidos reflejan una mejora substancial en la precisión comparado con anteriores técnicas más convencionales, concretamente Gaussian Mixture Models y i-vectors, y una mejora notable en la precisión de los mecanismos de Atención respecto al pooling estadístico. En les últimes dècades, Speech Emotion Recognition (SER), o el Reconeixement d'Emocions per Veu, ha generat fort interès en l'àmbit del tractament de la parla per a les implicacions que presenta en la interacció humà-computador. En aquest treball s'aconsegueix reconèixer l'estat emocional del parlant utilitzant xarxes neuronals profundes que extreuen de manera automàtica característiques contingudes en l'espectrograma del senyal de veu. Partint d'un model que utilitza anàlisi estadística per a pooling, es proposa una estratègia alternativa per a millorar el rendiment incorporant mecanismes d'Atenció. Com a millora afegida, s'explora el camp del mulit-task learning definint el reconeixement de l'idioma com a tasca auxiliar per al model. Els resultats finals obtinguts reflecteixen una millora substancial en la precisió comparat amb anteriors mètodes, concretament respecte Gaussian Mixture Models i i-vectors, i una millora notable en la precisió dels mecanismes d'Atenció respecte el pooling estadístic.

We invited authors of selected Comments and Perspectives published in Nature Machine Intelligence in the latter half of 2019 and first half of 2020 to describe how their topic has developed, what their thoughts are about the challenges of 2020, and what they look forward to in 2021.

Models based on attention mechanisms have shown unprecedented speech recognition performance. However, they are computationally expensive and unnecessarily complex for keyword spotting, a task targeted to small-footprint devices. This work explores the application of Lambda networks, an alternative framework for capturing long-range interactions without attention, for the keyword spotting task. We propose a novel \textit{ResNet}-based model by swapping the residual blocks by temporal Lambda layers. Furthermore, the proposed architecture is built upon uni-dimensional temporal convolutions that further reduce its complexity. The presented model does not only reach state-of-the-art accuracies on the Google Speech Commands dataset, but it is 85% and 65% lighter than its Transformer-based (KWT) and convolutional (Res15) counterparts while being up to 100 times faster. To the best of our knowledge, this is the first attempt to explore the Lambda framework within the speech domain and therefore, we unravel further research of new interfaces based on this architecture., Comment: speech recognition, keyword spotting, lambda networks

In recent years the research effort of different tasks is through neural network techniques. This work continues along this line and explores its possibilities in the task of speech emotion recognition (SER). In this work, we evaluate the effect of two points of interest, Loss functions and Data Augmentation: various variations of the Softmax function and its effect on the classification accuracies are evaluated and two data augmentation techniques are also explored, one of them related to the application of the system in driving situations. En los últimos años el esfuerzo de investigación de diferentes tareas es a través de técnicas de redes neuronales. Este trabajo continúa con esa línea y explora sus posibilidades en la tarea de reconocimiento de emociones en señales de voz (SER). En este trabajo se evalúan los efectos en dos puntos de interés, las funciones de pérdida y el Aumento de Datos: Se evalúan diversas variaciones de la función Softmax y su efecto en las accuracies de clasificación, también se exploran dos técnicas de aumento de datos, una de ellas relacionada con la aplicación del sistema en situaciones de conducción. En els últims anys l'esforç d'investigació de diferents tasques és a través de tècniques de xarxes neuronals. Aquest treball continua amb aquesta línia i explora les seves possibilitats en la tasca de reconeixement d'emocions en senyals de veu (SER). En aquest treball s'avaluen els efectes en dos punts d'interès, les funcions de pèrdua i l'augment de dades: S'avaluen diverses variacions de la funció Softmax i el seu efecte en les accuracies de classificació, també s'exploren dues tècniques d'augment de dades, una d'elles relacionada amb l'aplicació de sistema en situacions de conducció.

There may be a lot of abusive behaviour in conversations between teenagers, which take place through social media. In this project, we develop classifiers to find out which texts present abuse such as violence, sexual behaviour, disorder, anxiety... For this reason, we use different classifiers that take advantage of the most recent techniques based on language modeling using deep neural networks, namely Transformers. But we find it difficult to obtain supervised data, i.e. data tagged with the feeling it contains. Luckily, thanks to the countless sources of information available on the Internet, we can get unsupervised data and examples of text on the Internet. Consequently, this project aims to explore the possibility of using unsupervised data, beyond using a supervised set to train a system in the task of sentiment classification. More specifically in short texts, in informal contexts and in Catalan and Spanish. We have a set of about 200,000 supervised samples and a set of 300,000 unsupervised samples. We will introduce and study different models of Transformers (BERT, GAN- BERT, XLM-RoBERTa and ELECTRA) and their performance in the work of sentiment analysis. To finish with, based on models that achieve a better score in the binary classification task of feelings, i.e. with ELECTRA and XLM-RoBERTa, we construct a multi- label classifier. En las conversaciones entre adolescentes, que se dan por medio de las redes sociales, pueden existir muchos comportamientos abusivos. En este proyecto desarrollamos clasificadores que detecten qué textos presentan algún abuso como violencia, conductas sexuales, desórdenes, ansiedad... Por este motivo, usamos diferentes clasificadores que aprovechan las técnicas más recientes basadas en modelado de lenguaje mediante redes neuronales profundas, concretamente Transformers. Pero nos encontramos con la dificultad de obtener datos supervisados, es decir datos etiquetados con el sentimiento que contiene. Por suerte, gracias a la infinidad de fuentes de información de las que disponemos en la red, podemos conseguir datos y ejemplos de texto no supervisado a internet. Por consiguiente, este proyecto quiere explorar la posibilidad de utilizar datos no supervisados, más allá de usar un conjunto de datos supervisado para entrenar un sistema en la tarea de clasificación de sentimientos. Más concretamente en textos cortos, en contextos informales y de lengua catalana y castellana. Contamos con un set de cerca de 200.000 muestras supervisadas y un set de 300.000 muestras no supervisadas. Introduciremos y estudiaremos diferentes modelos de Transformers (BERT, GAN-BERT, XLM-RoBERTa y ELECTRA) y su rendimiento en la tarea de análisis de sentimientos. Finalmente, a partir de los modelos que consiguen una mejor puntuación en la tarea de clasificación binaria de sentimientos, es decir con ELECTRA y XLM-RoBERTa, construimos un clasificador multi-etiqueta. En les converses entre adolescents, que es donen per mitjà de les xarxes socials, poden existir molts comportaments abusius. En aquesta tesi desenvolupem classificadors que detectin quins textos presenten algun abús com violència, conductes sexuals, desordres, ansietat... Per aquest motiu, faig servir diferents classificadors que aprofiten les tècniques més recents basades en modelat de llenguatge mitjançant xarxes neuronals profundes, concretament Transformers. Però ens trobem amb la dificultat d?obtenir dades supervisades, és a dir dades etiquetades amb el sentiment que conté. Per sort, gràcies a la infinitat de fonts d?informació de les que disposem a la xarxa, podem aconseguir dades i exemples de text no supervisat a internet. Per consegüent, aquest projecte vol explorar la possibilitat d'utilitzar dades no supervisades, més enllà de fer servir un set supervisat per entrenar un sistema en la tasca de classificació de sentiments. Més concretament en textos curts, en contextos informals i de llengua catalana i castellana. Comptem amb un set de prop de 200.000 mostres supervisades i un set de 300.000 mostres no supervisades. Introduirem i estudiarem diferents models de Transformers (BERT,GAN-BERT,XLM-Roberta i Electra) i el seu rendiment en la tasca d'anàlisi de sentiments. Finalment, a partir dels models que aconsegueixen una millor puntuació en la tasca de classificació binària de sentiments, és a dir amb Electra i XLM-Roberta, construïm un classificador multi-label

Recent advances in Deep Learning (DL) for speaker recognition have improved the performance but are constrained to the need of labels for the background data, which is difficult in prac- tice. In i-vector based speaker recognition, cosine (unsuper- vised) and PLDA (supervised) are the basic scoring techniques, with a big performance gap between the two. In this thesis we tried to fill this gap without using speaker labels in several ways. We applied Restricted Boltzmann Machine (RBM) vectors for the tasks of speaker clustering and tracking in TV broadcast shows. The experiments on AGORA database show that us- ing this approach we gain a relative improvement of 12% and 11% for speaker clustering and tracking tasks, respectively. We also applied DL techniques in order to increase the discrimina- tive power of i-vectors in speaker verification task, for which we have proposed the use of autoencoder in several ways, i.e., (1) as a pre-training for a Deep Neural Network (DNN), (2) as a near- est neighbor autoencoder for i-vectors, (3) as an average pooled nearest neighbor autoencoder. The experiments on VoxCeleb database show that we gain a relative improvement of 21%, 42% and 53%, using the three systems respectively. Finally we also proposed a self-supervised end-to-end speaker verification system. The architecture is based on a Convolutional Neural Network (CNN), trained as a siamese network with multiple branches. From the results we can see that our system shows comparable performance to a supervised baseline This work was supported by the project PID2019-107579RBI00 / AEI / 10.13039/501100011033

Aquest estudi té com a finalitat principal aconseguir que l’assistent de veu Mycroft funcioni de forma independent al núvol i que pugui operar sense necessitat d’estar connectat a internet. Aquestes necessitats impliquen que els diferents servidors que permeten el funcionament de l’assistent funcionin de forma local al dispositiu i disposin de totes les dades necessàries per operar.

Most state-of-the-art Deep Learning systems for text-independent speaker verification are based on speaker embedding extractors. These architectures are commonly composed of a feature extractor front-end together with a pooling layer to encode variable-length utterances into fixed-length speaker vectors. In this paper we present Double Multi-Head Attention (MHA) pooling, which extends our previous approach based on Self MHA. An additional self attention layer is added to the pooling layer that summarizes the context vectors produced by MHA into a unique speaker representation. This method enhances the pooling mechanism by giving weights to the information captured for each head and it results in creating more discriminative speaker embeddings. We have evaluated our approach with the VoxCeleb2 dataset. Our results show 6.09% and 5.23% relative improvement in terms of EER compared to Self Attention pooling and Self MHA, respectively. According to the obtained results, Double MHA has shown to be an excellent approach to efficiently select the most relevant features captured by the CNN-based front-ends from the speech signal. This work was supported by the Spanish project PID2019-107579RBI00 / AEI / 10.13039/501100011033.

We study the problem of estimating the overall mutual information in M independent parallel discrete-time memory-less Gaussian channels from N independent data sample pairs per channel (inputs and outputs). We focus on the case where the number of active channels L is sparse in comparison with the total number of channels (L ≪ M), for which the direct application of the maximum likelihood principle is problematic due to overfitting, especially for moderate to small N. For this regime, we show that the bias of the mutual information estimate is reduced by resorting to the minimum description length (MDL) principle. As a result, simple pre-processing based on a per-channel threshold on the empirical squared correlation coefficient is required with a fixed threshold that monotonically decreases with N as 1 - N -1/N , for N ≥ 4. The resulting improvement is shown in terms of the estimated information bias. This work is supported by projects TEC2016-76409-C2-1-R (WINTER), Ministerio de Economia y Competividad, Spanish National Research Plan, and 2017 SGR 578 - AGAUR, Catalan Government.

This project proposes a voice activated home automation system with location functionalities. The system consists of microcontrollers distributed around the home with a microphone that perform a voice activity detection in real-time. Then a communication protocol has been designed to manage the audio transmission from the microcontrollers to the controller device using WiFi. The controller can then locate the source of the audio performing trilateration with the time difference of arrival of the signals into the microphones. Time precision is needed in the voice detection and the network needs to be synchronized. Several methods are proposed and tested. Using the WiFi beacons as reference. Generating multicast UDP datagrams periodically. At the end, the synchronization is implemented using the output of a zero crossing detector connected to the mains electricity because the other methods had some limitations. Finally, a provisioning method is proposed for easier deployment. Este proyecto propone un sistema domótico activado por voz con funcionalidades de localización. El sistema consiste en microcontroladores distribuidos por el hogar con un micrófono que realizan una detección de voz en tiempo real. Después se ha diseñado un protocolo de comunicación para gestionar la transmisión de audio de los microcontroladores al dispositivo controlador utilizando WiFi. El controlador puede localizar el origen de audio haciendo una trilateración de la diferencia de los tiempos de llegada de la señal a los micrófonos. Es necesaria una gran precisión temporal en la detección de voz y la red debe estar sincronizada. Varios métodos han sido propuestos y probados. Utilizando los beacons WiFi como referencia. Generando datagramas UDP en multidifusión de forma periódica. Al final, la sincronización ha sido implementada utilizando la salida de un circuito detector de paso por cero conectado a la toma de corriente del hogar ya que los otros métodos presentan limitaciones. Finalmente, se ha propuesto un método de aprovisionamiento para un despliegue más fácil. Aquest projecte proposa un sistema domòtic activat per veu amb funcionalitats de localització. El sistema consisteix en microcontroladors distribuïts per la llar amb un micròfon que realitzen una detecció de veu en temps real. Després s?ha dissenyat un protocol de comunicació per a gestionar la transmissió d?àudio dels microcontroladors al dispositiu controlador utilitzant WiFi. El controlador pot localitzar l?origen d?àudio fent una trilateració de la diferència dels temps d?arribada de les senyals als micròfons. És necessària una gran precisió temporal a la detecció de veu i la xarxa ha d?estar sincronitzada. Diversos mètodes han estat proposats i provats. Utilitzant els beacons WiFi com a referència. Generant datagramas UDP en difusió selectiva de forma periòdica. Al final, la sincronitació ha estat implementada utilitzant la sortida d?un circuit de pas per zero connectat a la presa de corrent de la llar ja que els altres mètodes presenten limitacions. Finalment, s?ha proposat un mètode d?aprovisionament per a un desplegament més fàcil.

Speech enhancement improves recorded voice utterances to eliminate noise that might be impeding their intelligibility or compromising their quality. Typical speech enhancement systems are based on regression approaches that subtract noise or predict clean signals. Most of them do not operate directly on waveforms. In this work, we propose a generative approach to regenerate corrupted signals into a clean version by using generative adversarial networks on the raw signal. We also explore several variations of the proposed system, obtaining insights into proper architectural choices for an adversarially trained, convolutional autoencoder applied to speech. We conduct both objective and subjective evaluations to assess the performance of the proposed method. The former helps us choose among variations and better tune hyperparameters, while the latter is used in a listening experiment with 42 subjects, confirming the effectiveness of the approach in the real world. We also demonstrate the applicability of the approach for more generalized speech enhancement, where we have to regenerate voices from whispered signals.

The use of private layers is explored for a multi-task learning approach to speech emotion recognition. The methods used include LSTMs, multi-task learning, high level representation, and "in the wild" testing.

In the last years, i-vectors followed by cosine or PLDA scoringtechniques were the state-of-the-art approach in speaker veri-fication. PLDA requires labeled background data, and thereexists a significant performance gap between the two scoringtechniques. In this work, we propose to reduce this gap by us-ing an autoencoder to transform i-vector into a new speaker vec-tor representation, which will be referred to as ae-vector. Theautoencoder will be trained to reconstruct neighbor i-vectors in-stead of the same training i-vectors, as usual. These neighbori-vectors will be selected in an unsupervised manner accordingto the highest cosine scores to the training i-vectors. The evalua-tion is performed on the speaker verification trials of VoxCeleb-1 database. The experiments show that our proposed ae-vectorsgain a relative improvement of 42% in terms of EER comparedto the conventional i-vectors using cosine scoring, which fillsthe performance gap between cosine and PLDA scoring tech-niques by 92%, but without using speaker labels

The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation. This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO).

Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level. Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation. In this work we propose the use of an attention mechanism to obtain a discriminative speaker embedding given non fixed length speech utterances. Our system is based on a Convolutional Neural Network (CNN) that encodes short-term speaker features from the spectrogram and a self multi-head attention model that maps these representations into a long-term speaker embedding. The attention model that we propose produces multiple alignments from different subsegments of the CNN encoded states over the sequence. Hence this mechanism works as a pooling layer which decides the most discriminative features over the sequence to obtain an utterance level representation. We have tested this approach for the verification task for the VoxCeleb1 dataset. The results show that self multi-head attention outperforms both temporal and statistical pooling methods with a 18\% of relative EER. Obtained results show a 58\% relative improvement in EER compared to i-vector+PLDA., 4+1 pages. 4 Figures. Accepted for Interspeech 2009

The main alternatives nowadays to dealwith sequences are Recurrent Neural Net-works (RNN), Convolutional Neural Networks(CNN) architectures and the Transformer. Inthis context, RNN’s, CNN’s and Transformerhave most commonly been used as an encoder-decoder architecture with multiple layers ineach module. Far beyond this, these architec-tures are the basis for the contextual word em-beddings which are revolutionizing most natural language downstream applications. However, intermediate layer representations insequence-based architectures can be difficultto interpret. To make each layer representation within these architectures more accessible and meaningful, we introduce a web-based toolthat visualizes them both at the sentence and token level. We present three use cases. The first analyses gender issues in contextual worde mbeddings. The second and third are show-ing multilingual intermediate representations for sentences and tokens and the evolution of these intermediate representations along the multiple layers of the decoder and in the con-text of multilingual machine translation. This work is supported by a Google Faculty Research Award. This workis also supported by the Spanish Ministerio de Economía y Competitividad, the European Regional Development Fund and the Agencia Estatal de Investigación, through the post-doctoral senior grant Ramón y Cajal, contracts TEC2015-69266-P and TIN2017-88515-C2-1-R(GEN3DLIVE) (MINECO/FEDER,EU), and contract PCIN-2017-079 (AEI/MINECO).

The influence of emotional content on language processing remains unclear. Previous research conducted in English has obtained contradictory results regarding the effects of valence on word recognition. Whereas some studies indicate that valence predicts recognition latencies such that negative words are processed more slowly than positive words, other studies indicate facilitation of responses to emotional (both positive and negative) compared to neutral words. The authors examined the influence of valence and arousal on word recognition reaction time (RT) using large-scale word naming and lexical decision data-sets in Spanish. They found that linear mixed-effects model estimates revealed a valence but not an arousal effect on reading latencies. The influence of valence was better captured by a graded (RTs to positive words< neutral< negative) than by a categorical (positive< negative) valence effect. A categorical emotional versus neutral effect was not reliably observed. In an advance on previous research, the authors' analyses showed that the valence effect is substantially more prominent in lexical decision than in pronunciation. These results mirror some of those reported previously in English, adding evidence to support their validity, and demonstrating important parallels in word recognition processes in orthographically shallow as well as deep languages.(PsycINFO Database Record (c) 2018 APA, all rights reserved).

Silent Speech Interface (SSI) is a technology able to synthesize speech in the absence of any acoustic signal. It can be useful in cases like laryngectomy patients, noisy environments or silent calls. This thesis explores the particular case of SSI using ultrasound images of the tongue as input signals. A 'direct synthesis' approach based on Deep Neural Networks and Mel-generalized cepstral coefficients is proposed. This document is an extension of Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface". Several deep learning models, such as the basic Feed-forward Neural Networks, Convolutional Neural Networks and Recurrent Neural Networks are presented and discussed. A denoising pre-processing based on a Deep Convolutional Autoencoder has also been studied. A considerable number of experiments using a set of different deep learning architectures and an extensive hyperperameter optimization study have been realized. The different experiments have been testing and rating several objective and subjective quality measures. According to the experiments, an architecture based on a CNN and bidirectional LSTM layers has shown the best results in both objective and subjective terms. Silent Speech Interface (SSI) is a technology able to synthesize speech in the absence of any acoustic signal. It can be useful in cases like laryngectomy patients, noisy environments or silent calls. This thesis explores the particular case of SSI using ultrasound images of the tongue as input signals. A 'direct synthesis' approach based on Deep Neural Networks and Mel-generalized cepstral coefficients is proposed. This document is an extension of Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface". Several deep learning models, such as the basic Feed-forward Neural Networks, Convolutional Neural Networks and Recurrent Neural Networks are presented and discussed. A denoising pre-processing based on a Deep Convolutional Autoencoder has also been studied. A considerable number of experiments using a set of different deep learning architectures and an extensive hyperperameter optimization study have been realized. The different experiments have been testing and rating several objective and subjective quality measures. According to the experiments, an architecture based on a CNN and bidirectional LSTM layers has shown the best results in both objective and subjective terms. Silent Speech Interface (SSI) és una tecnologia capaç de sintetitzar veu partint únicament de senyals no-acústiques. Pot tenir gran utilitat en casos com pacients de laringectomia, ambients sorollosos o trucades silencioses. Aquesta tèsis explora el cas particular de SSI utilitzant imatges de la llengua captades amb ultrasons com a senyals d'entrada. Es proposa un enfocament de 'síntesis directa' basat en Xarxes Neuronals Profundes i coeficients Mel-generalized cepstral. Aquest document és una extensió del treball de Csapó et al. "DNN-based Ultrasound-to-Speech Conversion for a Silent Speech Interface" . Diversos models de xarxes neuronals són presentats i discutits, com les bàsiques xarxes neuronals directes, xarxes neuronals convolucionals o xarxes neuronals recurrents. També s'ha estudiat un pre-processat reductor de soroll basat en un Autoencoder convolucional profund. S'ha portat a terme un nombre considerable d'experiments utilitzant diverses arquitectures de Deep Learning, així com un extens estudi d'optimització d'hyperparàmetres. Els diferents experiments han estat evaluar i qualificar a partir de diferentes mesures de qualitat objectives i subjectives. Els millors resultats, tant en termes objectius com subjectius, els ha presentat una arquitectura basada en una CNN i capes bidireccionals de LSTMs.

HighlightsOverview of hybrid machine translation guided by the rule-based paradigm.Overview of hybrid machine translation guided by the corpus-based paradigm.Review of works on machine translation applications with hybrid components. This survey on hybrid machine translation (MT) is motivated by the fact that hybridization techniques have become popular as they attempt to combine the best characteristics of highly advanced pure rule or corpus-based MT approaches. Existing research typically covers either simple or more complex architectures guided by either rule or corpus-based approaches. The goal is to combine the best properties of each type.This survey provides a detailed overview of the modification of the standard rule-based architecture to include statistical knowledge, the introduction of rules in corpus-based approaches, and the hybridization of approaches within this last single category. The principal aim here is to cover the leading research and progress in this field of MT and in several related applications.

Alzheimer’s disease (AD) is the most prevalent form of degenerative dementia; it has a high socio-economic impact in Westerncountries. The purpose of our project is to contribute to earlier diagnosis of AD and allow better estimates of its severity by usingautomatic analysis performed through new biomarkers extracted through non-invasive intelligent methods. The method selectedis based on speech biomarkers derived from the analysis of spontaneous speech (SS). Thus the main goal of the present work isfeature search in SS, aiming at pre-clinical evaluation whose results can be used to select appropriate tests for AD diagnosis. Thefeature set employed in our earlier work offered some hopeful conclusions but failed to capture the nonlinear dynamics of speechthat are present in the speech waveforms. The extra information provided by the nonlinear features could be especially useful whentraining data is limited. In this work, the fractal dimension (FD) of the observed time series is combined with linear parameters inthe feature vector in order to enhance the performance of the original system while controlling the computational cost.© 2014 Elsevier Ltd. All rights reserved.

Most speech synthesis systems require a linguistic module to produce the features that drive the speech generation module. In this project, system will be designed using a deep architecture and automatically learned to produce either linguistic features or speech from the raw letter representation. Recently, neural networks have become the state of the art for speech synthesis from raw text tasks and they are actually representing a powerful force in the industry. In this project, we present an end-to-end deep learning-based TTS system, able to generate a voice signal from characters. In order to fulfil this task we developed a re-implementation of the Uncodicional-SampleRNN neural vocoder, in order to be conditioned under an adaptation of MUSA, which predicts vocoder parameters from text. Recientemente, las redes neuronales se han convertido en el estado del arte para las tareas de síntesis del habla y actualmente representan una fuerza poderosa en la industria. En este proyecto, presentamos un sistema de conversión de texto a voz (Text-to-Speech) basado en aprendizaje profundo (Deep learning), capaz de generar una señal de voz a partir de caracteres. Para realizar dicha tarea desarrollamos una adaptación de MUSA, encargado de realizar una predicción de los parámetros del vocoder a partir del texto para, posteriormente, condicionar una reimplementación del vocoder neuronal Uncodicional-SampleRNN. Recentment, les xarxes neuronals s'han convertit en l'estat de l'art per a les tasques de síntesis de la parla i actualment representen una força poderosa en la indústria. En aquest projecte, presentem un sistema de conversió de text a veu (Text-to-Speech) basat en aprenentatge profund (Deep learning), capaç de generar un senyal de veu a partir de caràcters. Per realitzar aquesta tasca desenvolupem una adaptació de MUSA, encarregat de realitzar una predicció dels paràmetres del vocoder a partir del text, per condicionar posteriorment una reimplementació del vocoder neuronal Uncodicional-SampleRNN.

Speech Recognition (reconocimiento de voz) es la tarea que pretende indentificar palabras habladas y convertirlas a texto. Este trabajo de fin de grado se centra en utilizar técnicas de deep learning para construir un sistema de Speech Recognition entrenándolo end-to-end. Como paso preliminar, hacemos un resumen de los métodos más relevantes llevados a cabo los últimos años. A continuación estudiamos uno de los trabajos más recientes en este área que propone un modelo sequence to sequence con atención entrenado end-to-end. Después, reproducimos satisfactoriamente el modelo y lo avaluamos con la base de datos TIMIT. Analizamos los parecidos y diferencias entre la implementación propuesta y el trabajo teórico original. Y finalmente, experimentamos y contrastamos el modelo utilizando diferentes parámetros (e.g. numero de neuronas por capa, la tasa de aprendizaje -learning rate y los batch sizes) y reducimos el Phoneme Error Rate cerca del 12% relativo. Speech recognition is the task aiming to identify words in spoken language and convert them into text. This bachelor's thesis focuses on using deep learning techniques to build an end-to-end Speech Recognition system. As a preliminary step, we overview the most relevant methods carried out over the last several years. Then, we study one of the latest proposals for this end-to-end approach that uses a sequence to sequence model with attention-based mechanisms. Next, we successfully reproduce the model and test it over the TIMIT database. We analyze the similarities and differences between the current implementation proposal and the original theoretical work. And finally, we experiment and contrast using different parameters (e.g. number of layer units, learning rates and batch sizes) and reduce the Phoneme Error Rate in almost 12% relative. Speech Recognition (reconeixement de veu) és la tasca que pretén indentificar paraules del llenguatge parlat i convertir-les a text. Aquest treball de fi de grau es centra en utilitzar tècniques de deep learning per construir un sistema d'Speech Recognition entrenant-lo end-to-end. Com a pas preliminar, fem un resum dels mètodes més rellevants duts a terme els últims anys. A continuació, estudiem un dels treballs més recents en aquesta àrea que proposa un model sequence to sequence amb l?atenció entrenat end-to-end. Després, reproduim satisfactòiament el model i l'avaluem amb la base de dades TIMIT. Analitzem les semblances i diferències entre l'implementació proposada i el treball teòric original. I finalment, experimentem i contrastem el model utilitzant diferents paràmetres (e.g. nombre de neurones per capa, la taxa d'aprenentatge -learning rate- i els batch sizes) i reduim el Phoneme Error Rate gairebé un 12% relatiu.

Treball de fi de grau en Sistemes Audiovisuals Tutora: Mireia Farrús i Cabeceran La interacció persona-màquina ha augmentat exponencialment durant els últims anys. Un exemple d’aquest creixement es pot veure en el camp del processament de la parla, i en concret en el de la síntesi de veu. Aquest treball va sobre el tractament de la síntesi de veu dins el marc del projecte KRISTINA, orientat al sector mèdic, on s’utilitzarà veu sintetitzada perquè els pacients amb dificultats de comprensió de la llengua autòctona del país on viuen s’hi puguin comunicar. Els sintetitzadors de veu (o Text-To-Speech (TTS) systems, en anglès) acostumen a ser monòtons i amb poca naturalitat. Un dels objectius d’aquest treball és aconseguir generar una millor veu per a l’avatar del projecte KRISTINA, dotant-la de més naturalitat i expressivitat, de manera que s’acosti al màxim a una veu humana. Hem treballat la prosòdia a nivell de paràgraf i a nivell d’estructura comunicativa, utilitzant diferents sintetitzadors de veu. Mitjançant l’anàlisi i l’aplicació de diversos llenguatges d’etiquetes de veu utilitzats per a modificar determinades característiques prosòdiques relacionades amb l’entonació, la durada i la intensitat, hem aconseguit millores en la naturalitat i l’expressivitat de la veu sintetitzada. The human-computer interaction has increased exponentially in recent years. An example of this growth can be seen in the field of speech processing, specifically in speech synthesis. This work is about the treatment of speech synthesis within the framework of KRISTINA, a project oriented in the medical sector, which develop a conversational avatar to facilitate communication with patients with difficulties in understanding the native language of the country where they live. The majority of Text-to-Speech (TTS) voice synthesizers tend to be monotonous and with few naturalness, and one of the aims of this work is to create a better voice for KRISTINA avatar, providing it with more naturalness and expressiveness, approaching as close as possible to a human voice. We have worked prosody paragraph level and in terms of communicative structure, by using several speech synthesizers. By analyzing and applying different speech synthesis markup languages used to modify certain prosodic characteristics related to intonation, duration and intensity, we achieved improvements in naturalness and expressiveness of the synthesized voice.

The following document will drive the reader through the whole development of a project consisting in the investigation of the feasibility and development of an access control security system that uses voice biometrics as its cornerstone to make rooms as safe and controlled as possible in a two-factor authentication approach using open software. For that, an initial decision is presented, in which the technology that is going to be used to make the Speaker Recognition functionality possible is chosen. A detailed explanation of it is included so that the reader will know and understand the basics to follow the project development. With this, the document will go step by step through the different stages that I have faced and the requirements that came up and I had to adapt to, including the integration of the system that is nowadays up and running at IIT. The User Interface of the final application is presented together with an explanation of every step that users will have to follow to use the system correctly. Last, an estimation of the costs associated to this project is presented, which will give the reader an idea of how much starting this project from scratch would cost if the same hardware is used. Also, for those developers that would like to get some of this functionality in their systems, a link to a GitHub repository where they can find the code is also given.

In this project we present a first attempt at a Voice Conversion system based on Deep Learning in which the alignment between the training data is intrinsic to the model. Our system is structured in three main blocks. The first performs a vocoding of the speech (we have used Ahocoder for this task) and a normalization of the data. The second and main block consists of a Sequence-to-Sequence model. It consists of an RNN-based encoder-decoder structure with an Attention Mechanism. Its main strengts are the ability to process variable-length sequences, as well as aligning them internallly. The third block of the system performs a denormalization and reconstructs the speech signal. For the development of our system we have used the Voice Conversion Challenge 2016 dataset, as well as a part of the TC-STAR dataset. Unfortunately we have not obtained the results we expected. At the end of this thesis we present them and discuss some hypothesis to explain the reasons behind them. En este proyecto presentamos un primer intento en la realización de un sistema de Conversión de Voz basado en Aprendizaje Profundo (\emph{Deep Learning}) en el cual el alineamiento de los datos de entrenamiento es intrínseco al modelo. Nuestro sistema está estructurado en tres bloques principales. El primer bloque codifica la señal de voz en parámetros (\emph{vocoding}). Hemos elegido el \emph{vocoder} Ahocoder para esta tarea. Este bloque también normaliza los parámetros codificados. El segundo bloque consiste en un modelo \emph{Sequence-to-Sequence}. Este modelo está formado por una estructura codificador-decodificador basada en Redes Neuronales Recurrentes (RNN) con un Mecanismo de Atención. Sus puntos fuertes son la capacidad de procesar secuencias de longitud variable, a la vez que las alinea internamente. El tercer bloque del sistema desnormaliza los parámetros, y reconstruye la señal de voz a partir de ellos. Para el desarrollo del modelo hemos usado el conjunto de datos (\emph{dataset}) del \emph{Voice Conversion Challenge} 2016. También hemos usado una parte del conjunto TC-STAR. Desafortunadamente no hemos obtenido los resultados que esperábamos. Al final de esta tesis los presentamos y proponemos varias hipótesis que los explican. En aquest projecte presentem un primer itent en la realització d'un sistema de Conversió de Veu basat en Aprenentatge Profund (Deep Learning) en el qual l'alineament entre les dades d'entrenament sigui intrínsec al model. El nostre sistema s'estructura en tres blocs principals. El primer bloc codifica la veu en paràmetres (\emph{vocoding}). Hem usat el codificador Ahocoder per a aquesta tasca. A més a més, aquest primer bloc normalitza les dades. El segon bloc consisteix en un model \emph{Sequence-to-Sequence}. Consisteix en una estructura codificador-decodificador basada en Xarxes Neuronals Recurrents (RNN) amb un Mecanisme d'Atenció (\emph{Attention Mechanism}). Els punts forts d'aquest model són la capacitat per a tractar seqüències de durada variable, alhora que les alinea internament. El tercer bloc del sistema desnormalitza les seqüències i reconstrueix els senyals de veu. Per a desenvolupar el sistema hem usat el conjunt de dades del \emph{Voice Conversion Challenge} 2016. Hem fet servir també una part del conjunt TC-STAR. Desafortunadament no hem obtingut els resultats que esperàvem. Al final d'aquesta tesis presentem aquests resultats i plantegem algunes hipòtesis que els expliquen.

The transcription of spontaneous speech still poses a challenge to state-of-the-art methods for automatic speech recognition. The present thesis describes the comprehensive development of a large vocabulary continuous speech recognition system for the transcription of Catalan broadcast news and conversions and evolves towards novel approaches for analysis and modelling of acoustic reduction in spontaneous speech. It emphasises initially on various conventional methods for acoustic analysis, acoustic and language modelling and hypothesis search. Improvements over the original single-pass baseline system are mainly attained by domain and speaking style emphasising interpolation of individually estimated language models, linear discriminating projection of acoustic observations that improves the phonetic class separability, speaker normalisation of the acoustic observations, speaker adaptive training and acoustic model adaptation in a multi-pass system approach. The analysis of acoustic reduction initially emphasises on context independent vowel and consonant specific spectral and temporal properties whose parameters display statistically significant differences between the phoneme prototypes in spontaneous speech and their canonical realisations in planned speech. The introduction of the feature space analysis provides the general means to reveal these differences in conventional acoustic observations for automatic speech recognition. It displays statistically significant differences context-independently but also in a syllable context between adjacent phonemes suggesting particular reduction patterns. The analysis furthermore challenges the often suggested coherence between the co-occurring reduction of spectral and temporal properties. The modelling of acoustic reduction first emphasises on segment conditioned discriminating variables and variability class dependent models and variability class specific adaptation of the original acoustic model. It introduces phoneme rate as means to analyse temporal properties and feature space reduction ratio as means to analyse the reduction of spectral properties in conventional feature space for large vocabulary continuous speech recognition as discriminating variables. These variables are clustered and determine the classes for segment conditioned variability class dependent models and their scoring during the hypothesis search in recognition. Both approaches displays no significant performance improvement. Furthermore the modelling advances towards segment constituent predictability dependent models that introduce predictability as discriminating variable for variability class dependent models relying on the fundamental coherence between predictability and acoustic reduction that is suggested through the principle of least effort and the redundancy theory. It thereby emphasises on word and phoneme predictability. This approach displays no significant performance improvement. Planned speech is apparently antagonising the principle of least effort. Thus, a prior segment conditioned analysis of acoustic reduction may indicate its average degree of reduction, while their within-segment variation may indicate whether it exhibits sufficient relaxation of the speaking style to adopt the principle of least effort. Thus, segments exhibiting small within-segment variation may be modelled separately from those of large within-segment variation, whereas modelling the latter by word, syllable or phoneme predictability dependent models may provide a research perspective., La transcripció de converses espontànies encara suposa un repte per als mètodes actuals de reconeixement automàtic de veu. Aquesta tesi descriu el desenvolupament d'un sistema de reconeixement de veu continu de vocabulari gran per a la transcripció de converses i notícies emeses en català i condueix cap a noves aproximacions per a l'anàlisi i modelat de la reducció acústica en converses espontànies. Es centra inicialment en diversos mètodes convencionals per a l'anàlisi acústica, modelat acústic i del llenguatge i en la cerca d'hipòtesis. Les millores respecte el sistema original d'única passada són principalment degudes al domini i l'estil en la parla posant èmfasi en la interpolació de models de llenguatge, discriminació lineal i projecció d'observacions acústiques, entrenament adaptat al locutor per millorar la separació de les classes fonètiques, normalització de les observacions acústiques, i adaptació del model acústic en una sistema de múltiples passades. L'anàlisi de reducció acústica posa inicialment èmfasi en les propietats espectrals i temporals independents de vocals i consonant específiques, els paràmetres de les quals mostren diferències estadísticament significatives entre els prototips de fonemes en la conversa espontània i la seva realització canònica en el discurs planejat. La introducció de l'anàlisi del espai de característiques proporciona els mitjans generals per a revelar aquestes diferències en observacions acústiques convencionals per al reconeixement automàtic de veu. Mostra diferències estadísticament significatives independents de context però també entre fonemes adjacents en el context de síl·laba suggerint patrons de reducció particulars. A més, l'anàlisi desafia la, sovint suggerida, coherència entre les reducció simultànies de les propietats espectrals i temporals. El modelat de la reducció acústica primer fa èmfasi en variables discriminants de cada segment, models dependents de la variabilitat de la classe i l'adaptació del model acústic original. Introdueix la taxa de fonemes com a mitjà d'analitzar propietats temporals i la proporció de la reducció del espai de característiques com a mitjà d'analitzar la reducció dels propietats espectrals en el espai de característiques convencional per al reconeixement de veu continu de vocabulari gran com a variables discriminants. Aquestes variables s'agrupen i determinen les classes per a models dependents de la variabilitat de cada segment i la seva puntuació durant el reconeixement i cerca d'hipòtesi. Ambdues aproximacions no mostren una millora significativa en el rendiment. A més a més, les tècniques de modelat es dirigeixen cap a models dependents de la predicibilitat del segment que introdueixen la predicibilitat com a variable discriminant per a models dependents de la classe de variabilitat basats en la coherència fonamental entre predicibilitat i reducció acústica que es suggereix pel principi del mínim esforç i la teoria de la redundància. Per tant, emfatitza la predicibilitat de les paraules i dels fonemes. Aquesta aproximació no suposa cap millora significativa de rendiment. El discurs planejat és aparentment antagònic amb el principi del mínim esforç. Per tant, un anàlisi previ condicionat al segment de la reducció acústica pot indicar el seu grau mig de reducció, mentre la variació intra-segmental pot indicar si exhibeix prou relaxació en l'estil de parlar per adoptar el principi del mínim esforç. Per tant, segments amb poca variació intra-segmental poden ser modelats apart dels que tenen gran variació intra-segmental, mentre que modelar aquestes darreres mitjançant models dependents de predicibilitat de paraula, síl·laba o fonema poden aportar una perspectiva viable de recerca.

Conversion from text to speech relies on the accurate mapping from linguistic to acoustic symbol sequences, for which current practice employs recurrent statistical models such as recurrent neural networks. Despite the good performance of such models (in terms of low distortion in the generated speech), their recursive structure with intermediate affine transformations tends to make them slow to train and to sample from. In this work, we explore two different mechanisms that enhance the operational efficiency of recurrent neural networks, and study their performance&ndash, speed trade-off. The first mechanism is based on the quasi-recurrent neural network, where expensive affine transformations are removed from temporal connections and placed only on feed-forward computational directions. The second mechanism includes a module based on the transformer decoder network, designed without recurrent connections but emulating them with attention and positioning codes. Our results show that the proposed decoder networks are competitive in terms of distortion when compared to a recurrent baseline, whilst being significantly faster in terms of CPU and GPU inference time. The best performing model is the one based on the quasi-recurrent mechanism, reaching the same level of naturalness as the recurrent neural network based model with a speedup of 11.2 on CPU and 3.3 on GPU.

Restricted Boltzmann Machines (RBMs) have shown success in both the front-end and backend of speaker verification systems. In this paper, we propose applying RBMs to the front-end for the tasks of speaker clustering and speaker tracking in TV broadcast shows. RBMs are trained to transform utterances into a vector based representation. Because of the lack of data for a test speaker, we propose RBM adaptation to a global model. First, the global model&mdash, which is referred to as universal RBM&mdash, is trained with all the available background data. Then an adapted RBM model is trained with the data of each test speaker. The visible to hidden weight matrices of the adapted models are concatenated along with the bias vectors and are whitened to generate the vector representation of speakers. These vectors, referred to as RBM vectors, were shown to preserve speaker-specific information and are used in the tasks of speaker clustering and speaker tracking. The evaluation was performed on the audio recordings of Catalan TV Broadcast shows. The experimental results show that our proposed speaker clustering system gained up to 12% relative improvement, in terms of Equal Impurity (EI), over the baseline system. On the other hand, in the task of speaker tracking, our system has a relative improvement of 11% and 7% compared to the baseline system using cosine and Probabilistic Linear Discriminant Analysis (PLDA) scoring, respectively.

Alzheimer's disease is the most prevalent form of progressive degenerative dementia; it has a high socio-economic impact in Western countries. Therefore it is one of the most active research areas today. Alzheimer's is sometimes diagnosed by excluding other dementias, and definitive confirmation is only obtained through a post-mortem study of the brain tissue of the patient. The work presented here is part of a larger study that aims to identify novel technologies and biomarkers for early Alzheimer's disease detection, and it focuses on evaluating the suitability of a new approach for early diagnosis of Alzheimer’s disease by non-invasive methods. The purpose is to examine, in a pilot study, the potential of applying Machine Learning algorithms to speech features obtained from suspected Alzheimer sufferers in order help diagnose this disease and determine its degree of severity. Two human capabilities relevant in communication have been analyzed for feature selection: Spontaneous Speech and Emotional Response. The experimental results obtained were very satisfactory and promising for the early diagnosis and classification of Alzheimer’s disease patients.

The Speech Emotion Recognition project verses about the design and implementation of an emotion recognition system by analyzing the characteristics of the voice signal, based on the use of the Ivectors technique, which is considered the state of the art in the area in treatment. The use of this technique is compared with two baseline systems (Gaussian Mixture Models and Speaker factors) and it is shown that the degree of recognition improves. The aim of this project is to recognize, with as much precision as possible, a set of 7 emotions, available in the INTERFACE database, partially developed by UPC. To achieve this, we design a series of experiments that are used to test the performance of the system under different situations. El proyecto Speech Emotion Recognition trata del diseño y la implementación de un sistema de reconocimiento de emociones mediante el análisis de las características de la señal de voz, basado en el uso de la técnica de Ivectors, que se considera el estado del arte del área en cuestión. El uso de esta técnica se compara con dos sistemas base (Gaussian Mixture Modelos y Speaker factores) y se demuestra que el grado de reconocimiento mejora. El objetivo de este proyecto es el de reconocer con la máxima precisión posible un conjunto de 7 emociones, disponibles en la base de datos INTERFACE, desarrollada parcialmente por la UPC. Para lograrlo, se diseñan una serie de experimentos que se utilizan para probar el rendimiento del sistema bajo diferentes casuísticas. El projecte Speech Emotion Recognition tracta del disseny i la implementació d'un sistema de reconeixement d'emocions mitjançant l'anàlisi de les característiques de la senyal de veu, basat en l'ús de la tècnica d'Ivectors, que es considera l'estat de l'art de l'àrea en qüestió. L'ús d'aquesta tècnica es compara amb dos sistemes base (Gaussian Mixture Models i Speaker factors) i es demostra que el grau de reconeixement millora. L'objectiu d'aquest projecte és el de reconèixer amb la màxima precisió possible un conjunt de 7 emociones, disponibles en la base de dades INTERFACE, desenvolupada parcialment per la UPC. Per a assolir-lo, es dissenyen un seguit d'experiments que s'utilitzen per probar el rendiment del sistema sota diferents casuístiques.

Speech synthesis is the task of generating speech using computers. Due to the limitations of classical techniques, these systems are normally not suitable for applications that would benefit from expressiveness in the speech, such as audiobook reading. In this project, we attempt to develop a text-to-speech speech synthesizer that is capable of reacting to the semantic content of the input text to produce expressive speech. The system is based on the Socrates text-to-speech framework developed in the VEU research lab at UPC and the Keras deep learning library. La sintesis de voz consiste en utilizar ordenadores para generar voz humana. Debido a las limitaciones de las técnicas clásicas, estos sitemas normalmente no son adecuados para aplicaciones que requieren voz expresiva como en la lectura automática de audiolibros. En este proyecto, tratamos de desarrollar un sintetizador de voz capaz de reaccionar al contenido semántico del texto para producir voz expresiva. El sistema está basado en el framework de síntesis de voz Socrates, desarrollado en el grupo VEU de la UPC, y en la librería de deep learning Keras. La síntesi de veu consisteix en fer servir ordinadors per generar veu humana. Degut a les limitacions de les tècniques clàssiques, aquests sistemes normalment no són adequats per aplicacions que requireixen veu expressiva com és el cas de la lectura de audiollibres automàtica. En aquest projecte, desenvolupem un sintetitzador de veu capaç de reaccionar al contingut semàntic del text per produir veu expressiva. El sistema està basat en el framework de síntesi de veu Socrates, desenvolupat al grup de recerca VEU de la UPC, i en la llibreria de deep learning Keras.

Speaker diarization has received several research attentions over the last decade. Among the different domains of speaker diarization, diarization in meeting domain is the most challenging one. It usually contains spontaneous speech and is, for example, susceptible to reverberation. The appropriate selection of speech features is one of the factors that affect the performance of speaker diarization systems. Mel Frequency Cepstral Coefficients (MFCC) are the most widely used short-term speech features in speaker diarization. Other factors that affect the performance of speaker diarization systems are the techniques employed to perform both speaker segmentation and speaker clustering. In this thesis, we have proposed the use of jitter and shimmer long-term voice-quality features both for Gaussian Mixture Modeling (GMM) and i-vector based speaker diarization systems. The voice-quality features are used together with the state-of-the-art short-term cepstral and long-term speech ones. The long-term features consist of prosody and Glottal-to-Noise excitation ratio (GNE) descriptors. Firstly, the voice-quality, prosodic and GNE features are stacked in the same feature vector. Then, they are fused with cepstral coefficients at the score likelihood level both for the proposed Gaussian Mixture Modeling (GMM) and i-vector based speaker diarization systems. For the proposed GMM based speaker diarization system, independent HMM models are estimated from the short-term and long-term speech feature sets. The fusion of the short-term descriptors with the long-term ones in speaker segmentation is carried out by linearly weighting the log-likelihood scores of Viterbi decoding. In the case of speaker clustering, the fusion of the short-term cepstral features with the long-term ones is carried out by linearly fusing the Bayesian Information Criterion (BIC) scores corresponding to these feature sets. For the proposed i-vector based speaker diarization system, the speaker segmentation is carried out exactly the same as in the previously mentioned GMM based speaker diarization system. However, the speaker clustering technique is based on the recently introduced factor analysis paradigm. Two set of i-vectors are extracted from the speaker segmentation hypothesis. Whilst the first i-vector is extracted from short-term cepstral features, the second one is extracted from the voice quality, prosody and GNE descriptors. Then, the cosine-distance and Probabilistic Linear Discriminant Analysis (PLDA) scores of i-vectors are linearly weighted to obtain a fused similarity score. Finally, the fused score is used as speaker clustering distance. We have also proposed the use of delta dynamic features for speaker clustering. The motivation for using deltas in clustering is that delta dynamic features capture the transitional characteristics of the speech signal which contain speaker specific information. This information is not captured by the static cepstral coefficients. The delta features are used together with the short-term static cepstral coefficients and long-term speech features (i.e., voice-quality, prosody and GNE) both for GMM and i-vector based speaker diarization systems. The experiments have been carried out on Augmented Multi-party Interaction (AMI) meeting corpus. The experimental results show that the use of voice-quality, prosody, GNE and delta dynamic features improve the performance of both GMM and i-vector based speaker diarization systems., La diarización del altavoz ha recibido varias atenciones de investigación durante la última década. Entre los diferentes dominios de la diarización del hablante, la diarización en el dominio del encuentro es la más difícil. Normalmente contiene habla espontánea y, por ejemplo, es susceptible de reverberación. La selección apropiada de las características del habla es uno de los factores que afectan el rendimiento de los sistemas de diarización de los altavoces. Los Coeficientes Cepstral de Frecuencia Mel (MFCC) son las características de habla de corto plazo más utilizadas en la diarización de los altavoces. Otros factores que afectan el rendimiento de los sistemas de diarización del altavoz son las técnicas empleadas para realizar tanto la segmentación del altavoz como el agrupamiento de altavoces. En esta tesis, hemos propuesto el uso de jitter y shimmer características de calidad de voz a largo plazo tanto para GMM y i-vector basada en sistemas de diarización de altavoces. Las características de calidad de voz se utilizan junto con el estado de la técnica a corto plazo cepstral y de larga duración de habla. Las características a largo plazo consisten en la prosodia y los descriptores de relación de excitación Glottal-a-Ruido (GNE). En primer lugar, las características de calidad de voz, prosódica y GNE se apilan en el mismo vector de características. A continuación, se fusionan con coeficientes cepstrales en el nivel de verosimilitud de puntajes tanto para los sistemas de diarización de altavoces basados ¿¿en el modelo Gaussian Mixture Modeling (GMM) como en los sistemas basados ¿¿en i-vector. . Para el sistema de diarización de altavoces basado en GMM propuesto, se calculan modelos HMM independientes a partir de cada conjunto de características. En la segmentación de los altavoces, la fusión de los descriptores a corto plazo con los de largo plazo se lleva a cabo mediante la ponderación lineal de las puntuaciones log-probabilidad de decodificación Viterbi. En la agrupación de altavoces, la fusión de las características cepstrales a corto plazo con las de largo plazo se lleva a cabo mediante la fusión lineal de las puntuaciones Bayesian Information Criterion (BIC) correspondientes a estos conjuntos de características. Para el sistema de diarización de altavoces basado en un vector i, la fusión de características se realiza exactamente igual a la del sistema basado en GMM antes mencionado. Sin embargo, la técnica de agrupación de altavoces se basa en el paradigma de análisis de factores recientemente introducido. Dos conjuntos de i-vectores se extraen de la hipótesis de segmentación de altavoz. Mientras que el primer vector i se extrae de características espectrales a corto plazo, el segundo se extrae de los descriptores de calidad de voz apilados, prosódicos y GNE. A continuación, las puntuaciones de coseno-distancia y Probabilistic Linear Discriminant Analysis (PLDA) entre i-vectores se ponderan linealmente para obtener una puntuación de similitud fundida. Finalmente, la puntuación fusionada se utiliza como distancia de agrupación de altavoces. También hemos propuesto el uso de características dinámicas delta para la agrupación de locutores. La motivación para el uso de deltas en la agrupación es que las características dinámicas delta capturan las características de transición de la señal de voz que contienen información específica del locutor. Esta información no es capturada por los coeficientes cepstrales estáticos. Las características delta se usan junto con los coeficientes cepstrales estáticos a corto plazo y las características de voz a largo plazo (es decir, calidad de voz, prosodia y GNE) tanto para sistemas de diarización de altavoces basados en GMM como en sistemas i-vector. Los resultados experimentales sobre AMI muestran que el uso de calidad vocal, prosódica, GNE y dinámicas delta mejoran el rendimiento de los sistemas de diarización de altavoces basados en GMM e i-vector.

In this paper, we describe the TALP- UPC participation in the News Task for German-English and Finish-English. Our primary submission implements a fully character to character neural machine translation architecture with an additional rescoring of a n-best list of hypothesis us- ing a forced back-translation to the source sentence. This model gives consistent im- provements on different pairs of languages for the language direction with the low- est performance while keeping the qual- ity in the direction with the highest perfor- mance. Additional experiments are reported for multilingual character to character neural machine translation, phrase-based trans- lation and the additional Turkish-English language pair.

Literature review on prosody reveals the lack of corpora for prosodic studies in Catalan and Spanish. In this paper, we present a corpus intended to fill this gap. The corpus comprises two distinct data-sets, a news subcorpus and a dialogue subcorpus, the latter containing either conversational or task-oriented speech. More than 25 h were recorded by twenty eight speakers per language. Among these speakers, eight were professional (four radio news broadcasters and four advertising actors). The entire material presented here has been transcribed, aligned with the acoustic signal and prosodically annotated. Two major objectives have guided the design of this project: (i) to offer a wide coverage of representative real-life communicative situations which allow for the characterization of prosody in these two languages; and (ii) to conduct research studies which enable us to contrast the speakers different speaking styles and discursive practices. All material contained in the corpus is provided under a Creative Commons Attribution 3.0 Unported License.

With the increasingly global nature of our everyday interactions, the need for multilingual technologies to support efficient and efective information access and communication cannot be overemphasized. Computational modeling of language has been the focus of Natural Language Processing, a subdiscipline of Artificial Intelligence. One of the current challenges for this discipline is to design methodologies and algorithms that are cross-language in order to create multilingual technologies rapidly. The goal of this JAIR special issue on Cross-Language Algorithms and Applications (CLAA) is to present leading research in this area, with emphasis on developing unifying themes that could lead to the development of the science of multi- and cross-lingualism. In this introduction, we provide the reader with the motivation for this special issue and summarize the contributions of the papers that have been included. The selected papers cover a broad range of cross-lingual technologies including machine translation, domain and language adaptation for sentiment analysis, cross-language lexical resources, dependency parsing, information retrieval and knowledge representation. We anticipate that this special issue will serve as an invaluable resource for researchers interested in topics of cross-lingual natural language processing.

This work presents a study on the suitability of prosodic andacoustic features, with a special focus on i-vectors, in expressivespeech analysis and synthesis. For each utterance of two dif-ferent databases, a laboratory recorded emotional acted speech,and an audiobook, several prosodic and acoustic features are ex-tracted. Among them, i-vectors are built not only on the MFCCbase, but also on F0, power and syllable durations. Then, un-supervised clustering is performed using different feature com-binations. The resulting clusters are evaluated calculating clus-ter entropy for labeled portions of the databases. Additionally,synthetic voices are trained, applying speaker adaptive training,from the clusters built from the audiobook. The voices are eval-uated in a perceptual test where the participants have to edit anaudiobook paragraph using the synthetic voices.The objective results suggest that i-vectors are very use-ful for the audiobook, where different speakers (book charac-ters) are imitated. On the other hand, for the laboratory record-ings, traditional prosodic features outperform i-vectors. Also,a closer analysis of the created clusters suggest that differentspeakers use different prosodic and acoustic means to conveyemotions. The perceptual results suggest that the proposed i-vector based feature combinations can be used for audiobookclustering and voice training.

Recogniton of peple by speech and face in TV shows. Participation in the international competition Meiaeval 2016. The Project described in this document falls within the topic of person recognition in TV recordings by mean of multimodal systems. It has been developed as collaboration with image and audio processing groups in the signal theory department in UPC. Thus, it is a project about the development of a person recognition system in TV Broadcast videos implemented for the participation in the Whorkshop Mediaeval 2016. The aim of the competition consist in find the names of the people that appears and talks in each shot of different videos of a given database. This discovery should be done in totally unsupervised manner using only the information in each shot as image, audio or text. For this purpose, it has been proposed to implement the fusion of three monomodal algorithms. These technologies will process the information present in the text, image and audio in independent manner. Information after monomodal technologies will be fused with objective of create e a multimodal algorithm able to tag the shots in the database. Finally, this thesis is centered in the monomodal audio algorithm development for which it has been proposed a tracking system based in i-vectors. El proyecto descrito en este documento se enmarca en el tópico del descubrimiento de personas en vídeos de televisión mediante sistemas multimodales. Forma parte de una colaboración con los grupos de procesado de audio e imagen del departamento de Teoría de señal. Así pues, se trata del desarrollo de un sistema de descubrimiento de personas en un entorno de televisión broadcast para la participación en un Workshop denominado Mediaeval 2016. El reto de la competición consiste en encontrar los nombres de las personas que aparecen hablando en cada escena de los diferentes shows de un conjunto de videos. Dicho descubrimiento debe realizarse de manera totalmente no supervisada, utilizando únicamente la información presente en cada escena, como la imagen, el audio o el texto. Para ello, se propone utilizar la fusión de tres algoritmos monomodales que procesen de manera independiente el texto, la imagen y el audio, para lograr un sistema multimodal capaz de etiquetar cada escena dada una base de datos. Finalmente, el presente proyecto se centrará en el desarrollo del algoritmo de audio a nivel monomodal, para el cual se ha propuesto la implementación de un sistema de tracking basado en i-vectors.

Nowadays, biometrics is considered as a promising solution in the market of security and personal verification. Applications such as financial transactions, law enforcement or network management security are already benefitting from this technology. Among the different biometric modalities, speaker verification represents an accurate and efficient way of authenticating a person’s identity by analyzing his/her voice. This identification method is especially suitable in real-life scenarios or when a remote recognition over the phone is required. The processing of a signal of voice, in order to extract its unique features, that allows distinguishing an individual to confirm or deny his/ her identity is, usually, a process characterized by a high computational cost. This complexity imposes that many systems, based on microprocessor clocked at hundreds of MHz, are unable to process samples of voice in real-time. This drawback has an important effect, since in general, the response time needed by the biometric system affects its acceptability by users. The design based on FPGA (Field Programmable Gate Arrays) is a suited way to implement systems that require a high computational capability and the resolution of algorithms in real-time. Besides, these devices allow the design of complex digital systems with outstanding performance in terms of execution time. This paper presents the implementation of a MFCC (Mel-Frequency Cepstrum Coefficients)—SVM (Support Vector Machine) speaker verification system based on a low-cost FPGA. Experimental results show that our system is able to verify a person’s identity as fast as a high-performance microprocessor based on a Pentium IV personal computer.