Search

The extensive use of Deep Neural Networks (DNNs) encourages people to design domain-specific hardware called Artificial Intelligence (AI) processors. The novel hardware makes optimizations challenging without a proper performance model that reveals working details and performance implications. This paper presents a performance model, Verrocchio, for Huawei DaVinci AI Core, which predicts the execution time of real-world DaVinci kernels. We propose specially-crafted micro-benchmarks to identify contention source, runtime behaviors, and bandwidth sharing, which significantly determine performance. Since DaVinci Core adopts a binary semaphore mechanism for synchronization, Verrocchio views each instruction as a discrete event and manages its execution time based on the programming logic. For evaluation, Verrocchio achieves average error rates of 2.62% and 2.30% in sample kernels for single-core and double-core execution. We demonstrate an optimizing process of matrix multiplications with Verrocchio, achieving speedups of 1.70× for operators and 1.53× for applications and error rates of 5.06% and 5.25%. • Detailed dissections of Huawei DaVinci AI Core, a novel AI processor. • Benchmarking the DaVinci Core bandwidth contention, the key performance factor. • Performance model for accurate execution time prediction of kernel program. • Demonstration of DaVinci kernel optimization and prediction accuracy evaluation. [ABSTRACT FROM AUTHOR]

The current methods of treating organic waste suffer from limited resource usage and low product value. Research and development of value-added products emerges as an unavoidable trend for future growth. Electro-fermentation (EF) is a technique employed to stimulate cell proliferation, expedite microbial metabolism, and enhance the production of value-added products by administering minute voltages or currents in the fermentation system. This method represents a novel research direction lying at the crossroads of electrochemistry and biology. This article documents the current progress of EF for a range of value-added products, including gaseous fuels, organic acids, and other organics. It also presents novel value-added products, such as 1,3-propanediol, 3-hydroxypropionic acid, succinic acid, acrylic acid, and lysine. The latest research trends suggest a focus on EF for cogeneration of value-added products, studying microbial community structure and electroactive bacteria, exploring electron transfer mechanisms in EF systems, developing effective methods for nutrient recovery of nitrogen and phosphorus, optimizing EF conditions, and utilizing biosensors and artificial neural networks in this area. In this paper, an analysis is conducted on the challenges that currently exist regarding the selection of conductive materials, optimization of electrode materials, and development of bioelectrochemical system (BES) coupling processes in EF systems. The aim is to provide a reference for the development of more efficient, advanced, and value-added EF technologies. Overall, this paper aims to provide references and ideas for the development of more efficient and advanced EF technology. [ABSTRACT FROM AUTHOR]

In recent years many amazing deep learning models have been developed, but in the process of practical applications, people often find that these deep learning models have high requirements for hardware storage space and computing power. In Artificial Intelligent of Things (AIoT) scenario, the computing power of the edge or terminal side are relatively limited, therefore, most conventional deep learning models are difficult to be deployed into AIoT devices. It is significant to explore the different performance under different scales of deep learning models. In this paper, we mainly propose a method to analyze the impact of deep learning models with various sizes through various experiments. We employ slimmable network as a Neural Archtecture Search (NAS) tool to realize various model size freely, and evaluate them on the indicators of flops, robustness and accuracy. The experimental results show the variation of flops, robustness and accuracy with the various model sizes, which help understand the impact on performance of deep learning models with different scales in AIoT systems. • Interpreting robustness of deep learning models with different model complexity. • Employing slimmable network to compress neural network. • Understanding the relevance between robustness and accuracy of deep learning models. [ABSTRACT FROM AUTHOR]

Flood modelling and forecasting can enhance our understanding of flood mechanisms and facilitate effective management of flood risk. Conventional flood hazard and risk assessments usually consider one driver at a time, whether it is ocean, fluvial or pluvial, without considering the compound nature of flood events. In this paper, we developed a novel approach for modelling and forecasting compound coastal-fluvial floods using a two-step framework. In step one, a hydrodynamic model is used to simulate floodwater propagation; while in step two, machine learning (ML) models are used to generate flood forecasts. The architecture of hydrodynamic-ML forecasting system incorporates a hydrodynamic model covering a specific domain, with individual ML models trained for each pixel. In total 7 ML models including: Support Vector Regression (SVR), Support Vector Machine (SVM), Radial Basis Function (RBF), Linear Regression (LR), Gaussian Process Regression (GPR), Decision Tree (DT), and Artificial Neural Network (ANN) were applied in this study. Forecasting compound floods is achieved using two sets of inputs: timeseries of river discharges in the upstream fluvial section and downstream ocean water levels in the coastal areas. The accuracy of the flood forecasting system is demonstrated for Cork City, Ireland; and modelling performance was evaluated using several statistical tools. Results show that the proposed models can provide reliable estimates of flood inundation and associated water depths. Overall, the RBF model exhibits the best performance. Despite the complexity of compound multi-driver floods, this study shows that the coupled hydrodynamic-ML approach can forecast coastal-fluvial flood with limited hydraulic and hydrological input data. This system overcomes the limitations of traditional hydrodynamic model-based systems where trade-offs between the always competing numerical model accuracy and computational time prohibit the model to be used for short-term flood forecasting. Once trained, the ML component of the coupled system can perform flood forecasting in near real-time, potentially integrating into a flood early warning system. Accurate flood forecasting has a wide range of positive societal impacts, including improved flood preparedness, increased confidence, better resource allocation, reduced flood damage, and potentially even flood prevention. • Novel technique to improve compound coastal-fluvial flood modelling. • Flood modelling technique based on seven Machine Learning methods. • Approach is successfully implemented, and results are evaluated over Cork, Ireland. • The results demonstrate minimal quantitative errors for the proposed models. [ABSTRACT FROM AUTHOR]

• A novel approach for single channel speech enhancement is presented, using the colored spectrograms. • Using deep neural network for spectrogram denoising and regression neural network for translating the colors of denoised spectrograms to short time Fouier transform (STFT) magnitudes. • Output is comparable to the state-of-the-art baseline models at a much reduced computational cost. Speech enhancement concerns the processes required to remove unwanted background sounds from the target speech to improve its quality and intelligibility. In this paper, a novel approach for single-channel speech enhancement is presented using colored spectrograms. We propose the use of a deep neural network (DNN) architecture adapted from the pix2pix generative adversarial network (GAN) and train it over colored spectrograms of speech to denoise them. After denoising, the colors of spectrograms are translated to magnitudes of short-time Fourier transform (STFT) using a shallow regression neural network. These estimated STFT magnitudes are later combined with the noisy phases to obtain an enhanced speech. The results show an improvement of almost 0.84 points in the perceptual evaluation of speech quality (PESQ) and 1 % in the short-term objective intelligibility (STOI) over the unprocessed noisy data. The gain in quality and intelligibility over the unprocessed signal is almost equal to the gain achieved by the baseline methods used for comparison with the proposed model, but at a much reduced computational cost. The proposed solution offers a comparative PESQ score at almost 10 times reduced computational cost than a similar baseline model that has generated the highest PESQ score trained on grayscaled spectrograms, while it provides only a 1 % deficit in STOI at 28 times reduced computational cost when compared to another baseline system based on convolutional neural network-GAN (CNN-GAN) that produces the most intelligible speech. [ABSTRACT FROM AUTHOR]

Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and back-end models may lead to a local minimum, which theoretically prevents the whole system from achieving the best optimization. Although some methods have been proposed for jointly optimizing the two models, such as the generalized end-to-end (GE2E) model and NPLDA E2E model, most of these methods have not fully investigated how to model the intra-relationship between multiple enrollment utterances. In this paper, we propose a new E2E joint method for speaker verification especially designed for the practical scenario of multiple enrollment utterances. To leverage the intra-relationship among multiple enrollment utterances, our model comes equipped with frame-level and utterance-level attention mechanisms. Additionally, focal loss is utilized to balance the importance of positive and negative samples within a mini-batch and focus on the difficult samples during the training process. We also utilize several data augmentation techniques, including conventional noise augmentation using MUSAN and RIRs datasets and a unique speaker embedding-level mixup strategy for better optimization. • Extend attention back-end to end-to-end for multi-enrollment speaker verification. • Introduce focal loss, suggest speaker embedding mixup for fine-tuning end-to-end model. • Analyze the effect of the number of enrollment utterances. • Analyze the embedding-level attention weights for speech genres. [ABSTRACT FROM AUTHOR]

The rapid development of Deep Neural Networks (DNNs) lays solid foundations for Internet of Things systems. However, mobile devices with limited processing capacity and short battery life confront the difficulties of executing complex DNNs. To satisfy different Quality of Service requirements, a feasible solution is offloading DNN layers to edge nodes and the cloud. The energy-efficient offloading problem for DNN-based applications with the deadline and budget constraints in the edge-cloud environment is still an open and challenging issue. To this end, this paper proposes a Hybrid Chaotic Evolutionary Algorithm (HCEA) incorporating diversification and intensification strategies and a DVFS-enabled version of it (HCEA-DVFS). The Archimedes Optimization Algorithm-based diversification strategy exploits global and local guiding information to improve population diversity during the updating process and employs Metropolis acceptance rule of Simulated Annealing to avoid premature convergence. The Genetic Algorithm-based chaotic intensification strategy is designed to enhance the local search capability of HCEA. Moreover, the Dynamic Voltage Frequency Scaling-enabled adjustment strategies can be embedded into HCEA to further reduce energy consumption by resetting frequency levels and reallocating DNN layers. Experimental results over four DNN-based applications demonstrate that HCEA-DVFS can reduce more energy consumption under different deadlines, budgets, and workloads on average by 7.93, 9.68, 11.02, 11.84, and 19.38 percent in comparison with HCEA, PSO-GA, MCEA, AOA, and Greedy, respectively. • An AOA-based diversification strategy is designed to exploit global and local guiding information for population updating. • A chaotic GA-based intensification strategy is developed to enhance local search capability and introduce randomness. • A DVFS-enabled adjustment strategy is presented to save energy by resetting frequency levels and reallocating DNN layers. • HCEA and HCEA-DVFS outperform baselines in energy consumption optimization under budget and deadline constraints. [ABSTRACT FROM AUTHOR]

• We propose novel training algorithms for vocoder-free text-to-speech synthesis using STFT spectra based on generative adversarial networks (GANs). • We demonstrate that using GANs with the original-frequency-resolution amplitude spectra degrades the synthetic speech quality. • We show that the proposed low-frequency-resolution GANs improves the synthetic speech quality. • We also show that using the inverse mel scale for the proposed algorithm further improves the synthetic speech quality. This paper proposes novel training algorithms for vocoder-free text-to-speech (TTS) synthesis based on generative adversarial networks (GANs) that compensate for short-term Fourier transform (STFT) amplitude spectra in low/multi frequency resolution. Vocoder-free TTS using STFT amplitude spectra can avoid degradation of synthetic speech quality caused by the vocoder-based parameterization used in conventional TTS. Our previous work for the vocoder-based TTS proposed a method for incorporating the GAN-based distribution compensation into acoustic model training to improve synthetic speech quality. This paper extends the algorithm to the vocoder-free TTS and propose a GAN-based training algorithm using low-frequency-resolution amplitude spectra to overcome the difficulty in modeling complicated distribution of the high-dimensional spectra. In the proposed algorithm, amplitude spectra are transformed into low-frequency-resolution amplitude spectra by applying an average pooling function along with a frequency axis; then the GAN-based distribution compensation is performed in the low-frequency-resolution domain. Because the low-frequency-resolution amplitude spectra approximately emulate filter banks, the proposed algorithm is expected to improve synthetic speech quality by reducing differences in spectral envelopes of natural and synthetic speech. Furthermore, various frequency scales that are related to human speech perception (e.g., mel and inverse mel frequency scales) can be introduced to the proposed training algorithm by applying an frequency warping function to amplitude spectra. This paper also proposes a GAN-based training algorithm using multi-frequency-resolution amplitude spectra that uses both low- and original-frequency-resolution amplitude spectra to reduce the differences in not only spectral envelopes but also fine structures. Experimental results demonstrate that (1) GANs using low-frequency-resolution amplitude spectra improve speech quality and work robustly against the settings of the frequency resolution and hyperparameters, (2) in comparison among low-, original-, and multi-frequency-resolution amplitude spectra, the use of low-frequency-resolution ones work best improve the synthetic speech quality, and (3) the use of the inverse mel frequency scale for obtaining low-frequency-resolution amplitude spectra further improves synthetic speech quality. [ABSTRACT FROM AUTHOR]

In many speaker verification tasks, the quality of speaker embedding is an important factor in affecting speaker verification systems. Advanced speaker embedding extraction networks aim to capture richer speaker features through the multi-branch network architecture. Recently, speaker verification systems based on transformer encoders have received much attention, and many satisfactory results have been achieved because transformer encoders can efficiently extract the global features of the speaker (e.g., MFA-Conformer). However, the large number of model parameters and computational latency are common problems faced by the above approaches, which make them difficult to apply to resource-constrained edge terminals. To address this issue, this paper proposes an effective, lightweight transformer model (MCA-former) with multi-scale convolutional self-attention (MCA), which can perform multi-scale modeling and channel modeling in the temporal direction of the input with low computational cost. In addition, in the inference phase of the model, we further develop a systematic re-parameterization method to convert the multi-branch network structure into the single-path topology, effectively improving the inference speed. We investigate the performance of the MCA-former for speaker verification under the VoxCeleb1 test set. The results show that the MCA-based transformer model is more advantageous in terms of the number of parameters and inference efficiency. By applying the re-parameterization, the inference speed of the model is increased by about 30%, and the memory consumption is significantly improved. • Designing a lightweight multi-scale convolutional self-attention module. • An efficient transformer encoder for speaker verification. • Using the re-parameterization method improves the model's inference efficiency. [ABSTRACT FROM AUTHOR]

Specific flood volume is an important criterion for evaluating the performance of sewer networks. Currently, mechanistic models - MCMs (e.g., SWMM) are usually used for its prediction, but they require the collection of detailed information about the characteristics of the catchment and sewer network, which can be difficult to obtain, and the process of model calibration is a complex task. This paper presents a methodology for developing simulators to predict specific flood volume using machine learning methods (DNN - Deep Neural Network, GAM - Generalized Additive Model). The results of Sobol index calculations using the GSA method were used to select the ML model as an alternative to the MCM model. It was shown that the DNN model can be used for flood prediction, for which high agreement was obtained between the results of GSA calculations for rainfall data, catchment and sewer network characteristics, and calibrated SWMM parameters describing land use and sewer retention. Regression relationships (polynomials and exponential functions) were determined between Sobol indices (retention depth of impervious area, correction factor of impervious area, Manning's roughness coefficient of sewers) and sewer network characteristics (unit density of sewers, retention factor - the downstream and upstream of retention ratio) obtaining R2 = 0. 55–0.78. The feasibility of predicting sewer network flooding and modernization with the DNN model using a limited range of input data compared to the SWMM was shown. The developed model can be applied to the management of urban catchments with limited access to data and at the stage of urban planning. • ML simulator of specific flood volume as an alternative to the SWMM model. • Multi-criteria selection of machine learning method for flooding simulation. • Sobol indices prediction in GSA via empirical models. • Correlation between Sobol indices and characteristic of the catchment and sewer. [ABSTRACT FROM AUTHOR]

In this paper, we present an approach that combines data-driven and physical modelling for predicting the runoff occurrence and volume at catchment scale. With that aim, we first estimated the runoff volume from recorded storms aided by the Green-Ampt infiltration model. Then, we used machine learning algorithms, namely LightGBM (LGBM) and Deep Neural Network (DNN), to predict the outputs of the physical model fed on a set of atmospheric variables (relative humidity, temperature, atmospheric pressure, and wind velocity) collected before or immediately after the beginning of the storm. Results for a small urban catchment in Madrid show DNN performed better in predicting the runoff occurrence and volume. Moreover, enriching the input primary atmospheric variables with auxiliary variables (e.g., storm intensity data recorded during the first hour, or rain volume and intensity estimates obtained from auxiliary regression methods) largely increased the model performance. We show in this manuscript data-driven algorithms shaped by physical criteria can be successfully generated by allowing the data-driven algorithm learn from the output of physical models. It represents a novel approach for physics-informed data-driven algorithms shifting from common practices in hydrological modelling through machine learning. • Data-driven and physical modelling are merged to predict runoff. • Data-driven algorithm fed on weather variables before the storm starts. • Light GBM and Deep Neural Networks were coupled with Green-Ampt model. [ABSTRACT FROM AUTHOR]

Physical exertion is a stress condition that affects how we normally produce speech. It alters both the temporal and spectral pattern of speech characteristics. Therefore, speech utterances can be used as a cost-effective telehealth solution to detect whether a person is under physical exertion. This paper deals with the detection of shortness of breath (or out-of-breath) condition from speech under physical load by using an multi-task learning (MTL) framework. The primary classification targets are the neutral and the out-of-breath classes. Naturally these are binary targets and do not reflect the actual extent of exertion. This leads to the creation of a novel auxiliary target learning in collaboration with a pre-trained expert system. The targets here indicate the level of exertion under physical load. The MTL framework for both the convolutional neural network (CNN) and CNN-long short-term memory (CLSTM) network performs nearly 1.5% (F1-score) better than the single task learning (STL) framework. In addition to that, out-of-breath speech has more influence on the lower frequency spectrum. Warped spectrograms are given as input to the networks, enabling the deep networks to focus on lower spectral regions. The non-linear frequency warping is achieved by Mel-scale transformation and constant-Q-transform (CQT). CQT, being less dependent on fixed-window size, shows at least 6.57% (F1-score) improvement over Mel-based inputs. The MTL framework, combined with the warped spectrum, performs better in classifying out-of-breath speech from neutral. • Aims to detect speech-based out-of-breath (or physical load) conditions. • Efficiently determines the extent of physical exertion using speech utterances. • Lower frequency-centric approach for better detection of out-of-breath condition. • Multi-task learning further improves the performance by combing the above two steps. [ABSTRACT FROM AUTHOR]

Several cities in the developing world, of which the capital city of India, New Delhi, is an example, often experience air quality in which pollutant levels go way above the levels considered hazardous for human health. To bring down the air quality to within permissible limits quickly, the measures typically taken involve shutting down certain high-polluting activities for some time to enable the air quality to recover temporarily. This paper presents a first-ever model based on artificial neural networks to forecast the extent of reduction in air quality parameters that can be achieved and the time period within which a change can be experienced when the source of the emissions is cut off temporarily. The model is based on the extensive data on the extent of reduction in air quality parameters that occurred during the lockdown that was imposed during the COVID-19 pandemic. The non-linear autoregressive exogenous network-based model chosen for the purpose employs the hour since stopping of emissions, relative humidity, wind speed, wind direction, and ambient temperature as input parameters to predict the rate of change of PM 2.5 with respect to the concentration at the start of the stopping of the emissions. Air quality data from a key monitoring station in New Delhi was used to develop the model. The model predicted the rate of drop in PM 2.5 with an R and MSE of 0.0044 and 0.9736, respectively, while training and 0.0095 and 0.9583 while testing. The model was then tested with data from 19 other stations in New Delhi, and accuracy of the model was found to be exceptionally accurate, with the correlation between the measured and the predicted PM 2.5 levels ranging from 0.74 to 0.94 and the MSE ranging from 0.0110 to 1.0746. Thus, the model can be employed to determine the number of hours of temporary stoppage of emissions required for the PM 2.5 concentration to reach safe levels. The methodology of development of the model can be extrapolated to construct models tailored for use in other parts of the world as well. • A novel ANN -based model is reported to predict rate of drop of PM 2.5 • Model has been trained on extensive dataset from the covid lockdown in Delhi. • Duration and efficacy of emergency measures to reduce emissions can be estimated. • The model uses easy-to-measure meteorological parameters as inputs. • Model has been validated for 19 stations of various land-use types in Delhi. [ABSTRACT FROM AUTHOR]

Super-enhancer (SE) is a cluster of active typical enhancers (TE) with high levels of the Mediator complex, master transcriptional factors, and chromatin regulators. SEs play a key role in the control of cell identity and disease. Traditionally, scientists used a variety of high-throughput data of different transcriptional factors or chromatin marks to distinguish SEs from TEs. This kind of experimental methods are usually costly and time-consuming. In this paper, we proposed a model DeepSE, which is based on a deep convolutional neural network model, to distinguish the SEs from TEs. DeepSE represent the DNA sequences using the dna2vec feature embeddings. With only the DNA sequence information, DeepSE outperformed all state-of-the-art methods. In addition, DeepSE can be generalized well across different cell lines, which implied that cell-type specific SEs may share hidden sequence patterns across different cell lines. The source code and data are stored in GitHub (https://github.com/QiaoyingJi/DeepSE). • A computational model for detecting super enhancers from typical enhancers is proposed. • The model can be generalized to different cell lines, even for cell-type specific SEs. • DNA sequence is informative to identify genome elements. [ABSTRACT FROM AUTHOR]

The bi-directional flow of energy and information in the smart grid makes it possible to record and analyze the electricity consumption profiles of consumers. Because of the increasing rate of inflation over the past few years, people started looking for means to use electricity illegally, termed as electricity theft. Many data analytics techniques are proposed in the literature for electricity theft detection (ETD). These techniques help in the detection of suspected illegal consumers. However, the existing approaches have a low ETD rate either due to improper handling of the imbalanced class problem in a dataset or the selection of inappropriate classifier. In this paper, a robust big data analytics technique is proposed to resolve the aforementioned concerns. Firstly, adaptive synthesis (ADASYN) is applied to handle the imbalanced class problem of data. Secondly convolutional neural network (CNN) and long-short term memory (LSTM) integrated deep siamese network (DSN) are proposed to discriminate the features of both honest and fraudulent consumers. Specifically, the task of feature extraction from weekly energy consumption profiles is handed over to the CNN module while the LSTM module performs the sequence learning. Finally, the DSN contemplates on the shared features provided by the CNN-LSTM and applies final judgment. The data analytics is performed on different train–test ratios of the real-time smart meters' data. The simulation results validate the proposed model's effectiveness in terms of high area under the curve, F 1 -Score, precision and recall. • A robust big data analytic technique for electricity theft detection in smart grid is proposed. • We propose an adaptive synthesis to handle imbalanced class problem. • A deep siamese network with underpinned integrated networks of convolution neural network and long-short term memory is proposed to discriminate both features of fair and fraud consumers. • Simulation results validate the proposed model in terms of high area under the curve, low false positive rate and low false negative rate. [ABSTRACT FROM AUTHOR]

An increasing amount of malicious code causes harm on the internet by threatening user privacy as one of the primary sources of network security vulnerabilities. The detection of malicious code is becoming increasingly crucial, and current methods of detection require much improvement. This paper proposes a method to advance the detection of malicious code using convolutional neural networks (CNNs) and intelligence algorithm. The CNNs are used to identify and classify grayscale images converted from executable files of malicious code. Non-dominated Sorting Genetic Algorithm II (NSGA-II) is then employed to deal with the data imbalance of malware families. A series of experiments are designed for malware image data from Vision Research Lab. The experimental results demonstrate that the proposed method is effective, maintaining higher accuracy and less loss. • A technique for converting a malware binary to an image was introduced. • In this paper, a method based on CNN is used to identify and classify the malicious codes. • An effective data equilibrium approach based on the NSGA-II was designed. • The proposed method was demonstrated through the extensive experiments. [ABSTRACT FROM AUTHOR]

In this paper we address the problem of constructing a feature extractor which combines Mallat's scattering transform framework with time-frequency (Gabor) representations. To do this, we introduce a class of frames, called uniform covering frames , which includes a variety of semi-discrete Gabor systems. Incorporating a uniform covering frame with a neural network structure yields the Fourier scattering transform S F and the truncated Fourier scattering transform. We prove that S F propagates energy along frequency decreasing paths and its energy decays exponentially as a function of the depth. These quantitative estimates are fundamental in showing that S F satisfies the typical scattering transform properties, and in controlling the information loss due to width and depth truncation. We introduce the fast Fourier scattering transform algorithm, and illustrate the algorithm's performance. The time-frequency covering techniques developed in this paper are flexible and give insight into the analysis of scattering transforms. [ABSTRACT FROM AUTHOR]

Highlights • DSTC6: Dialog Challenge to improve performance of end-to-end dialog systems using Neural Network models and dialog breakdown detection. • Track 1, End-to-End Goal Oriented Dialog Learning: selection of the best system response. Hybrid Code Network and Memory Network were the best models. • Track 2, End-to-End Conversation Modeling: system response generation. 78.5% of the automatically generated sentences were rated as acceptable responses by humans. • Track 3, Dialogue Breakdown Detection. The submitted systems performed as well as humans in detecting dialog breakdown, for both English and Japanese data-sets. This paper describes the experimental setups and the evaluation results of the sixth Dialog System Technology Challenges (DSTC6) aiming to develop end-to-end dialogue systems. Neural network models have become a recent focus of investigation in dialogue technologies. Previous models required training data to be manually annotated with word meanings and dialogue states, but end-to-end neural network dialogue systems learn to directly output natural-language system responses without needing training data to be manually annotated. Thus, this approach allows us to scale up the size of training data and cover more dialog domains. In addition, dialogue systems require a meta-function to avoid deploying inappropriate responses generated by themselves. To challenge such issues, the DSTC6 consists of three tracks, (1). End-to-End Goal Oriented dialogue Learning to select system responses, (2). End-to-End Conversation Modeling to generate system responses using Natural Language Generation (NLG) and (3). Dialogue Breakdown Detection. Since each domain has different issues to be addressed to develop dialogue systems, we targeted restaurant retrieval dialogues to fill slot-value in Track 1, customer services on Twitter by combining goal-oriented dialogues and ChitChat in Track 2 and human-machine dialogue data for ChitChat in Track 3. DSTC6 had 141 people declaring their interests and 23 teams submitted their final results. 18 scientific papers were presented in the wrap-up workshop. We find the blending end-to-end trainable models associated to meaningful prior knowledge performs the best for the restaurant retrieval for Track 1. Indeed, Hybrid Code Network and Memory Network have been the best models for this task. In Track 2, 78.5% of the system responses automatically generated by the best system were rated better than acceptable by humans and this achieves 89% of the number of the human responses rated in the same class. In Track3, the dialogue breakdown detection technologies performed as well as human agreements, in both data-sets of English and Japanese. [ABSTRACT FROM AUTHOR]

Highlights • Different from previous studies that use a single label as the gold-standard to decide dialogue breakdown, we focused on inherent subjectivity when people regard dialogue breakdowns and variationality due to the design of conversational agents. • We propose methods for dialogue breakdown detection that first cluster annotators based on their label distributions, and then ensemble detectors trained by different sets of annotators. • Two kinds of neural networks are used as detectors. We assume that breakdowns occur locally (word and phrase levels) and/or globally (sentence and context levels). The convolutional neural network (CNN) is applied to detect former type of breakdowns, and long short-term memory (LSTM) detects the latter type of breakdowns. • Our best model achieves 63.6% F-score on breakdown detection, which is 5.6% higher than the baseline using the conditional random fields. • This is an extended version of our workshop paper submitted to the Dialogue Breakdown Detection Challange 3 (DBDC3). We newly conducted additional experiments that confirm effectiveness of clustering annotators based on their annotation distributions. In addition, detailed error analysis revealed characteristics of the CNN and LSTM on the dialogue breakdown detection task. Abstract Dialogue breakdown is a significant problem in conversational agents. Timely breakdown detection helps the agents quickly recover from mistakes, minimizing the impact on user experience. In this paper, we focus on two problems: variations in determining a response that breakdowns a conversation i.e. , subjectivity, and variations in breakdown types due to designs of conversational agents, i.e. , variationality. To address the subjectivity, which decreases the agreement rate among annotators, our methods detect a dialogue breakdown by ensembling detectors trained by different sets of annotators that are grouped using a clustering algorithm. To address the variationality, our methods apply two types of detector architectures to capture global and local breakdowns. The long short-term memory detector considers the global context and the convolutional neural networks detector is sensitive to the local characteristics. The ensemble of all detectors makes a final judgment. The results of the Japanese task in the Dialogue Breakdown Detection Challenge 3 (DBDC3) confirm that our approach significantly outperforms the baseline, which uses the conventional conditional random field. Detailed error analysis reveals that our encoders based on a convolutional neural network and a long short-term memory have different characteristics. It also confirms the effects of annotator clustering. [ABSTRACT FROM AUTHOR]

Highlights • An advanced end to end conversation system for the 6-th edition of Dialog System Technology Challenge (DSTC6). • Applying sequence adversarial training with extension of the objective function to improve both objective and subjective evaluation metrics. • Minimum Bayes risk (MBR) based system combination of multiple neural conversation models. • Example-based response selection using an embedding-based context similarity. • Thorough evaluation of three different neural conversation models, training techniques, and decoding strategies using a help-desk dialog task in DSTC6. Abstract This paper presents adversarial training and decoding methods for neural conversation models that can generate natural responses given dialog contexts. In our prior work, we built several end-to-end conversation systems for the 6th Dialog System Technology Challenges (DSTC6) Twitter help-desk dialog task. These systems included novel extensions of sequence adversarial training, example-based response extraction, and Minimum Bayes-Risk based system combination. In DSTC6, our systems achieved the best performance in most objective measures such as BLEU and METEOR scores and decent performance in a subjective measure based on human rating. In this paper, we provide a complete set of our experiments for DSTC6 and further extend the training and decoding strategies more focusing on improving the subjective measure, where we combine responses of three adversarial models. Experimental results demonstrate that the extended methods improve the human rating score and outperform the best score in DSTC6. [ABSTRACT FROM AUTHOR]

• A novel voice conversion technique for body-conducted unvoiced speech is proposed based on Taylor-Boltzmann Machines. The proposed algorithm was the integration of Taylor Series with Boltzmann machine Deep Neural Networks. • A body conductive microphone referred as the non audible murmur (NAM) microphone is used to capture the non-audible murmur from the neck part behind the ear. • The classifier was trained with the features likes "pitch chroma, spectral centroid, and spectral skewness" that are derived from the incoming body conducted low voice verbal communication signal. • In the initial training phase, a Taylor series approximation DNN model is trained from a collection of stereo data which encompasses the NAM speech of same phonemes represented as log-power spectral features. • Moreover, in the recognition phase, a well trained Taylor-DNN model that is based on Boltzmann Machine intakes the speech features and generates the "log-power-spectra features" and it aids in recognizing the speech waveform. • The paper may gain high interest from the readers and experts of Speech processing, Voice Conversion Techniques, body-conducted unvoiced speech. Communication becomes effectual while the verbal communication signal reaches the destination with profound characteristics. An application based on human voice attracts the researcher's attention. It leads the researcher to proceed towards the speech conversion for low voice humans having murmured voices as their attitude. The researchers introduced a technique for improving the body-conducted unvoiced speech for silent speech communication using Taylor-Boltzmann machine-trained optimized Deep Neural Network (DNN). The projected model includes two main stages: the training stage as well as recognition stage. In the training stage, the following functionalities are undergone: (i) pre-processing, (ii) feature extraction, and (iii) deep learning-based speech enhancement. Initially, the collected input Non-Audible Murmur (NAM) speech signal is subjected to the pre-processing stage, wherein the improved wiener filtering is employed for suppressing the noise in the input signal. From the pre-processed speech signal, the most relevant characteristics such as "spectral skewness, spectral chroma, spectral centroid and improved Mel Frequency Cepstral Coefficients (MFCC)" are extracted, and they are fused together. Then, using these fused features, the Taylor series with optimized DNN and Boltzmann Machine (Taylor-DNN-BM) model is trained. The final enhanced speech is acquired from the Taylor- optimized DNN-BM. The recognition phase encapsulates the feature extraction and deep learning-based speech enhancement stage. The input speech data enters into the feature extraction phase, wherein the characteristics such as "spectral skewness, spectral chroma, spectral centroid and improved (MFCC)" are retrieved. After that, these features are fused and fed as input to the Taylor-optimized DNN-BM model. From Taylor- optimized DNN-BM, the enhanced speech signal is acquired. In DNN, the weight function is fine-tuned using the newly projected Self Improved Flower Pollination Algorithm (SI-FPA) algorithm. This SI-FPA model is the conceptual improvement of the Flower Pollination Algorithm (FPA). Finally, the projected model is compared with the existing models to validate its efficiency in terms of speech enhancement. [ABSTRACT FROM AUTHOR]

This paper discusses how automatic speech recognition systems are and could be designed, in order to best exploit the discriminative information encoded in human speech. This contrasts with many recent machine learning approaches that apply general recognition architectures to signals to identify, with little concern for the nature of the input. The implicit assumption has often been that training can automatically discover the useful properties that exist in signals, with minimal manual intervention. These approaches may be suitable for some tasks such as image recognition, where the diversity of visual input is vast; e.g., an image may be any (natural or synthetic) scene that a camera views. We first examine what makes speech special, i.e., a natural signal from a complex tube, driven by a source that is quasi-periodic and/or noisy, aiming to communicate a wide variety of information, using the different vocal systems of human speakers. Then, we view how pertinent features are extracted from speech via efficient means, related to the objectives of communication. We see how to reliably and efficiently identify the different units of oral language. We learn from the history of attempts to do ASR, e.g., why they succeeded and how improved methods exploited the increasing availability of data and computer power (in particular, deep neural networks). Finally, we suggest ways to render ASR both more accurate and efficient. This work is aimed at both newcomers to ASR and experts, in terms of presenting issues broadly, but without mathematical or algorithmic details, which are readily found in the references. [ABSTRACT FROM AUTHOR]

Sexism, a form of oppression based on one's sex, manifests itself in numerous ways and causes enormous suffering. In view of the growing number of experiences of sexism reported online, automatically classifying these recollections can aid in the battle against sexism by allowing gender studies researchers and government officials involved in policymaking to conduct more effective analyses. This paper investigates the 23-class fine-grained, multi-label classification of accounts (reports) of sexism. Moreover, we propose a knowledge-based cascaded multi-task framework for fine-grained multi-label sexism classification. We leverage several supporting tasks, including homogeneous and heterogeneous auxiliary tasks. Homogeneous tasks are set up without incurring any manual labeling cost and heterogeneous tasks are set up which have a high correlation with the accounts of sexism. Unlabeled accounts of sexism are utilized through unsupervised learning to help construct our multi-task setup. In addition, we incorporate a knowledge module within the framework to infuse external knowledge features into the learning process. Further, we investigate transfer learning that employs weakly labeled accounts of sexism and transfers the learning to the multi-label sexism classification. We also devise objective functions that exploit label correlations in the training data explicitly. Multiple proposed multi-task methods outperform the state-of-the-art multi-label sexism classification across five standard metrics. • Propose a knowledge-based cascaded multi-task neural framework for the multi-label sexism classification. • Explore two types of auxiliary tasks: (a) Homogeneous tasks and (b) Heterogeneous tasks • Trained the model with weakly labeled data and transfered the knowledge to the sexism classification task. • Proposed loss functions aimed at tapping label correlations in the multi-label data explicitly. [ABSTRACT FROM AUTHOR]

Neural networks activated by the rectified linear unit (ReLU) play a central role in the recent development of deep learning. The topic of approximating functions from Hölder spaces by these networks is crucial for understanding the efficiency of the induced learning algorithms. Although the topic has been well investigated in the setting of deep neural networks with many layers of hidden neurons, it is still open for shallow networks having only one hidden layer. In this paper, we provide rates of uniform approximation by these networks. We show that ReLU shallow neural networks with m hidden neurons can uniformly approximate functions from the Hölder space W ∞ r ([ − 1 , 1 ] d) with rates O ((log ⁡ m) 1 2 + d m − r d d + 2 d + 4 ) when r < d / 2 + 2. Such rates are very close to the optimal one O (m − r d ) in the sense that d + 2 d + 4 is close to 1, when the dimension d is large. [ABSTRACT FROM AUTHOR]

Constructing neural networks for function approximation is a classical and longstanding topic in approximation theory. In this paper, we aim at constructing deep neural networks with three hidden layers using a sigmoidal activation function to approximate smooth and sparse functions. Specifically, we prove that the constructed deep nets with controllable magnitude of free parameters can reach the optimal approximation rate in approximating both smooth and sparse functions. In particular, we prove that neural networks with three hidden layers can avoid the phenomenon of saturation, i.e., the phenomenon that for some neural network architectures, the approximation rate stops improving for functions of very high smoothness. [ABSTRACT FROM AUTHOR]

Chinese chess is an ancient game in which the chess situation is the information ensemble of chess pieces’ spatial locations and interrelations. The situation evaluation plays an extremely important role in the policy decisions of Chinese chess game. However, the situation evaluation is too complex for human to cover every detail with naked eyes. The deep stochastic weight assignment network (DSWAN) proposed in this paper to classify the situations in advantages and disadvantages can solve the above problem. DSWAN is a type of multi-layer perception (MLP) which is divided into two main components: unsupervised feature extractor formed by multi-layer auto encoder and supervised classifier trained by stochastic weight assignment network (SWAN). We summarize a series of chess situation features by gathering specialized knowledges of Chinese chess, and these features are proved valid to estimate the situation. Another highlight of this paper is that the auto encoder is constrained by L 1∕2 regularization. By doing so, it can bring more sparsity to data set, make situation features more representative and ease the overfitting trouble of SWAN. [ABSTRACT FROM AUTHOR]

Recently, the deep learning schemes have been well investigated for improving the driving safety and efficiency in the transportation systems. In this paper, a probabilistic model named as CPGN (Collision Prediction model based on GA-optimized Neural Network) for decision-making in the rear-end collision avoidance system is proposed, targeting modeling the impact of important influential factors of collisions on the occurring probability of possible accidents in the Internet of Vehicles (IoV). The decision on how to serve the chauffeur is determined by a typical deep learning model, i.e., the BP neural network through evaluating the possible collision risk with V2I (Vehicle-to-Infrastructure) communication, V2V (Vehicle-to-Vehicle) communication and GPS infrastructure supporting. The proper structure of our BP neural network model is deeply learned with training data generated from VISSIM with multiple influential factors considered. In addition, since the selection of the connection coefficient array and thresholds of the neural network has great randomness, a local optimization issue is readily occurring during the modeling procedure. To overcome this problem and consider the ability to efficiently find out a global optimization, this paper chooses the genetic algorithm to optimize the coefficient array and thresholds of proposed neural network. For the purpose of enhancing the convergence speed of the proposed model, we further adjust the studying rate according to the relationship between the actual and predicated values of two adjacent iterations. Simulation results demonstrate that the proposed collision risk evaluation framework could offer rationale estimations to the possible collision risk in car-following scenarios for the next discrete monitoring interval. [ABSTRACT FROM AUTHOR]

Accurate, fast forecasting of hydro-meteorological time series is presently a major challenge in drought and flood mitigation. This paper proposes a hybrid approach, wavelet de-noising (WD) and Rank-Set Pair Analysis (RSPA), that takes full advantage of a combination of the two approaches to improve forecasts of hydro-meteorological time series. WD allows decomposition and reconstruction of a time series by the wavelet transform, and hence separation of the noise from the original series. RSPA, a more reliable and efficient version of Set Pair Analysis, is integrated with WD to form the hybrid WD-RSPA approach. Two types of hydro-meteorological data sets with different characteristics and different levels of human influences at some representative stations are used to illustrate the WD-RSPA approach. The approach is also compared to three other generic methods: the conventional Auto Regressive Integrated Moving Average (ARIMA) method, Artificial Neural Networks (ANNs) (BP-error Back Propagation, MLP-Multilayer Perceptron and RBF-Radial Basis Function), and RSPA alone. Nine error metrics are used to evaluate the model performance. Compared to three other generic methods, the results generated by WD-REPA model presented invariably smaller error measures which means the forecasting capability of the WD-REPA model is better than other models. The results show that WD-RSPA is accurate, feasible, and effective. In particular, WD-RSPA is found to be the best among the various generic methods compared in this paper, even when the extreme events are included within a time series. [ABSTRACT FROM AUTHOR]

In this paper an unsupervised clustering algorithm is developed for acoustic data in the context of speech recognition tasks. One of the key features of the algorithm is scalability to large data sets. Specifically, given the unlabeled training and test sets, the class-labels of the utterances are obtained in an automatic manner. The extracted labels may correspond to the speakers in the speech corpus if the data is relatively clean. The proposed scheme is attractive from an industrial perspective as it alleviates the need to store the speaker-labels manually, saving considerable amount of human efforts and expenses. The core of the algorithm comprises a three-stage architecture that processes the input data one after the other, while each stage is designed to perform a well-defined and specific task. In more detail, the first-pass involves a bottom-up clustering mechanism, the second-pass comprises a cluster splitting operation and the third-pass consists of a cluster refining process. Each of the stages allows for data parallelization using multiple CPUs that leads to faster computation. Two alternative forms of the algorithm are presented – the first considers Gaussian distributions and the other i-Vectors – to facilitate the clustering. Although the algorithm may find applications in various realms of speech recognition, in this paper, the effectiveness of the schemes are evaluated by means of speaker adaptive training (SAT) and speaker-aware training of DNN-HMM acoustic models. In particular, experiments are conducted on the Switchboard task to extract the speaker-labels for the utterances in the training and test sets. It is shown that the SAT DNN-HMM trained using the Gaussian based scheme yields a 7.2% relative improvement in the ASR accuracy over the speaker independent DNN-HMM, whereas the i-Vector approach provides an additional improvement, amounting to a 10.8% relative gain overall. The standard SAT DNN-HMM developed using the ground-truth speaker-labels is found to be only 2.7% relative better than the proposed scheme. Similar observation is made as with speaker-aware training. The analysis of computational complexity, conducted stage by stage, demonstrates the scalability of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Peer-Led Team Learning (PLTL) is a learning methodology where a peer-leader co-ordinate a small-group of students to collaboratively solve technical problems. PLTL have been adopted for various science, engineering, technology and maths courses in several US universities. This paper proposed and evaluated a speech system for behavioral analysis of PLTL groups. It could help in identifying the best practices for PLTL. The CRSS-PLTL corpus was used for evaluation of developed algorithms. In this paper, we developed a robust speech activity detection (SAD) by fusing the outputs of a DNN-based pitch extractor and an unsupervised SAD based on voicing measures. Robust speaker diarization system consisted of bottleneck features (from stacked autoencoder) and informed HMM-based joint segmentation and clustering system. Behavioral characteristics such as participation, dominance, emphasis, curiosity and engagement were extracted by acoustic analyses of speech segments belonging to all students. We proposed a novel method for detecting question inflection and performed equal error rate analysis on PLTL corpus. In addition, a robust approach for detecting emphasized speech regions was also proposed. Further, we performed exploratory data analysis for understanding the distortion present in CRSS-PLTL corpus as it was collected in naturalistic scenario. The ground-truth Likert scale ratings were used for capturing the team dynamics in terms of student’s responses to a variety of evaluation questions. Results suggested the applicability of proposed system for behavioral analysis of small-group conversations such as PLTL, work-place meetings etc. . [ABSTRACT FROM AUTHOR]

Speech difficulties are common in children and teenagers, but they can also occur in adults as a result of physical problems. A speech disorder is a situation in which an individual struggles to produce or construct the spoken sounds necessary for interpersonal communication. As a result, it could be challenging to comprehend the person's speech. Articulation abnormalities are typical speech problems. In this situation, automatic speech recognition (ASR) technology may be used to detect and further rectify such deficiencies. The first attempts to detect speech abnormalities were made in the early 1970s, and they appear to have followed the same path as those on the ASR. These early experiments did rely heavily on signal processing techniques. As time goes on, more ideas from ASR technology are being incorporated into systems that deal with speech impairments. Many traditional techniques are executed in the ASR system. In this paper, we developed an automatic speech recognition technology based on deep learning techniques. In this paper, we research alternative extraction and classification methods of electroencephalography (EEG) to help diagnose speech disorders (SD). The EEG data is prepared before degradation into numerous EEG sub-strands with a discrete wavelet transformation to eliminate unimportant errors. For sharpening signals, the Eigenvector crack curvature wavelet method was used. A hyper-similarity abnormality coder is used for feature extraction in the EEG recording and to detect synchronization between EEG channels, which may show abnormalities in communication. The recovered functions are then categorized using the Deep Residual–encoder–based VGG net CNN Classification Method. Thus, the techniques proposed to produce the most promising outcome aren't the suggested technique attained better classification accuracy when compared to the traditional methodologies. [ABSTRACT FROM AUTHOR]

Identifying the family of malware can determine their malicious intent and attack patterns, which helps to efficiently analyze large numbers of malware variants. Methods based on traditional machine learning often require a lot of time and resources in feature engineering. Virtually all existing static analysis methods based on malware visualization are derived from grayscale images, while a single low-order feature representation may be detrimental to discovering hidden features in a malware family. Based on these problems, this paper proposes an effective malware classification framework (MalFCS) based on malware visualization and automated feature extraction. MalFCS includes mainly three modules: malware visualization, feature extraction, and classification. First, we visualize malware binaries as entropy graphs based on structural entropy. Second, we present a feature extractor based on deep convolutional neural networks to extract patterns shared by a family from entropy graphs automatically. Finally, we propose an SVM classifier to classify malware based on the extracted features. We evaluate the proposed MalFCS over two widely studied benchmark datasets, i.e., Malimg and Microsoft. Experimental results show that compared with the state-of-the-art methods, MalFCS can obtain excellent classification performance with accuracy of 0.997 and 1, respectively, achieving the state-of-the-art performance. • MalFCS integrates visualization, automated feature extraction and classification. • Visualizing malware as entropy graphs based on structural entropy. • Using convolutional neural networks to extract family patterns from entropy graphs. • Achieving an accuracy of 0.997 and 1 on Malimg and Microsoft dataset, respectively. [ABSTRACT FROM AUTHOR]

The main contribution of this paper is to show efficient implementations of the convolution-pooling in the GPU, in which the pooling follows the multiple convolution. Since the multiple convolution and the pooling operations are performed alternately in earlier stages of many Convolutional Neural Networks (CNNs), it is very important to accelerate the convolution-pooling. Our new GPU implementation uses two techniques, (1) convolution interchange with direct sum, and (2) conversion to matrix multiplication. By these techniques, the computational and memory access cost are reduced. Further the convolution interchange is converted to matrix multiplication, which can be computed by cuBLAS very efficiently. Experimental results using Tesla V100 GPU show that our new GPU implementation compatible with cuDNN for the convolution-pooling is expected 2.90 times and 1.43 times faster for fp32 and fp16 than the multiple convolution and then the pooling by cuDNN, respectively. the most popular library of primitives to implement the CNNs in the GPU. • Efficient GPU implementations for the convolution-pooling have been presented. • The convolution-pooling is a frequently used operations in convolutional neural networks. • Our GPU implementation is cuDNN-compatible and so users can use it easily to accelerate the CNN. [ABSTRACT FROM AUTHOR]

• A series of neural adversarial learning methods are proposed for speaker recognition. • Deep learning is implemented for PLDA speaker recognition with i -vectors. • Adversarial manifold learning is proposed to build a subspace model with neighbor embeddings in latent variables. • Multiobjective learning is performed for minimax optimization. • Adversarial augmentation learning is proposed to compensate the imbalanced data problem in speaker recognition. This paper presents the adversarial learning approaches to deal with various tasks in speaker recognition based on probabilistic discriminant analysis (PLDA) which is seen as a latent variable model for reconstruction of i -vectors. The first task aims to reduce the dimension of i -vectors based on an adversarial manifold learning where the adversarial neural networks of generator and discriminator are merged to preserve neighbor embedding of i -vectors in a low-dimensional space. The generator is trained to fool the discriminator with the generated samples in latent space. A PLDA subspace model is constructed by jointly minimizing a PLDA reconstruction error, a manifold loss for neighbor embedding and an adversarial loss caused by the generator and discriminator. The second task of adversarial learning is developed to tackle the imbalanced data problem. A PLDA based generative adversarial network is trained to generate new i -vectors to balance the size of training utterances across different speakers. An adversarial augmentation learning is proposed for robust speaker recognition. In particular, the minimax optimization is performed to estimate a generator and a discriminator where the class conditional i -vectors produced by generator could not be distinguished from real i -vectors via discriminator. A multiobjective learning is realized for a specialized neural model with the cosine similarity between real and fake i -vectors as well as the regularization for Gaussianity. Experiments are conducted to show the merit of adversarial learning in subspace construction and data augmentation for PLDA-based speaker recognition. [ABSTRACT FROM AUTHOR]

• Novel deep domain architectures based on CNN for spoofing detection are proposed. • A comparison of features using a CNN as a back-end classifier. • An extensive cross-database evaluation on ASVspoof and BTAS databases. • Shows that deep domain adaptation in both supervised and unsupervised settings improve the spoofing detection performance. With the increasing use of voice as a biometric, it has become imperative to develop countermeasures to thwart malicious spoofing attacks on speaker recognition systems. Even though there has been a significant research effort over the last few years dedicated to the development of countermeasures to detect and deflect spoofing attacks, the problem is far from being solved. While a deep learning technique has been successfully applied in anti-spoofing research, it suffers from a data scarcity issue where large amounts of labeled training data are required to build a robust model. In this paper, we investigate a domain adaptation approach of deep architectures in both a supervised setting where we use labeled data, and in an unsupervised setting where we assume unlabeled data when transferring knowledge from the source to target domain. Specifically, we employ convolutional neural networks (CNNs) as back-end classifiers for spoofed speech detection. For supervised domain adaptation, we propose joint neural network training while allowing the weights to be shared between the source and target streams, and an additional domain regularizer. In the unsupervised domain adaptation scenario, the weights are not shared in order to explicitly model the domain shift. However, the weights are related by weight regularizers to take into account the difference between the two domains. We conduct extensive cross-database (domain mismatch) experiments using ASVspoof 2015 and BTAS 2016 datasets to demonstrate the generalization capability of the proposed deep domain architectures for spoofing detection. Experimental results reveal that the proposed architectures can generalize across databases for both supervised and unsupervised adaptation scenarios. [ABSTRACT FROM AUTHOR]

Keyphrases of a given document represent its main topic and they are used as a simple method to represent the document. Statistical and graph-based models as unsupervised approaches have been mainly studied. The statistical models have some difficulty in extracting keyphrases from a single document because most statistical ones generally require statistical information from a large corpus. On the other hand, graph-based models can extract keyphrases by only using the information from a single document; nevertheless, they have some drawbacks. The scores of the edges can be biased because a single document does not contain sufficient information to score the edges of a graph and this influences the score of the nodes. In this paper, we propose an effective combination method of a statistical model, C-value method, and a graph-based model to overcome the drawbacks of each model. A new scoring method for keyphrase candidates is developed by the graph-based model and the scores calculated by the new method are applied to the modified C-value method to estimate the final importance scores of the keyphrase candidates. Subsequently, the proposed model is evaluated using two datasets, SemEval 2010 and Inspec, and its results outperformed the state-of-the-art model among unsupervised models and the existing graph-based ranking models. [ABSTRACT FROM AUTHOR]

• Examine an LSTM neural tagging model for Named Entity Recognition (NER). • Demonstrate the capability of the proposed model with both word and character levels and how it can handle out-of-vocabulary words. • Convolutional Neural Network can be one of the promising technologies that can successfully extract character-level representations. • Compare different word vector models for the task of Arabic NER. • Conduct experimental evaluation that shows that our model performance outperforms state-of-the-art Arabic NER systems on many standard benchmarks. Named Entity Recognition (NER) is a significant information extraction task since it is an important component of many natural language processing applications, such as Information Retrieval, Question Answering and Speech Recognition. The complexity and morphological richness of the Arabic language is the main reason why most existing Arabic NER systems rely strongly on hand-crafted feature engineering. In this paper, we propose to augment the existing LSTM neural tagging model for Arabic NER with a Convolutional Neural Network (CNN) for the extraction of relevant character-level features. By operating on the character-level, the proposed model is able to handle out-of-vocabulary words. Our results show that character CNN is able to outperform the previously used character-level Bi-directional Long Short-Term Memory Networks (BiLSTM) in many settings. Moreover, our observations indicate that CNNs tend to perform better than BiLSTM on relatively longer tokens. In addition, we conduct a comparison of four different pre-trained word vector models for Arabic NER and results show that a Skip-Gram Word2vec model, pre-trained on a subset of the Arabic Gigaword corpus, is generally sufficient to obtain acceptable Arabic NER performance. [ABSTRACT FROM AUTHOR]

Most of the existing deep models are deep neural networks. Recently, the deep forest opens a door towards an alternative to deep neural networks for many tasks and has attracted more and more attention. At the same time, the deep forest model becomes widely used in many real-world applications. However, the existing deep forest system is inefficient and lacks scalability. In this paper, we present ForestLayer, which is an efficient and scalable deep forest system built on distributed task-parallel platforms. First, to improve the computing concurrency and reduce the communication overhead, we propose a fine-grained sub-forest based task-parallel algorithm. Next, we design a novel task splitting mechanism to reduce the training time without decreasing the accuracy of the original method. To further improve the performance of ForestLayer, we propose three system-level optimization techniques, including lazy scan, pre-pooling, and partial transmission. Besides the systematic optimization, we also propose a set of high-level programming APIs to improve the ease-of-use of ForestLayer. Finally, we have implemented ForestLayer on the distributed task-parallel platform Ray. The experimental results reveal that ForestLayer outperforms the existing deep forest system gcForest with 7 × to 20. 9 × speedup on a range of datasets. In addition, ForestLayer outperforms TensorFlow-based implementation on most of the datasets, while achieving better predictive performance. Furthermore, ForestLayer achieves good scalability and load balance. • ForestLayer is an efficient and scalable system for the training of deep forests. • Better performance than gcForest with 7x to 20.9x speedup on a range of datasets. • Outperform tfForest and achieves better predictive performance. • Achieve near linear scalability and good load balance. • High ease-of-use with a set of high-level programming APIs. [ABSTRACT FROM AUTHOR]

This paper investigated the uptake of CIP and OFL in single and multicomponent adsorptive systems using modified carbon nanotubes (CNTs) as adsorbent material. The characterization analyses of the pre- and post-process material by XPS, TG/DTG, FT-IR, SEM/EDS, and XRD helped in the elucidation of the mechanisms, indicating greater involvement of n-n and π -π interactions. In the kinetic studies, the simple systems with CIP and OFL were similar, both showed equilibrium time around 20/30 min and increased adsorptive capacity with increasing initial drug concentration. In the multicomponent system, different fractions of CIP and OFL were tested and the time to reach equilibrium also varied between 20 and 30 min. In general, the adsorption capacity of CIP is slightly lower than that of OFL under the conditions tested. The selectivity analysis of the system showed that the selectivity's of the two drugs are identical in equimolar fractions. The mathematical modeling of the kinetic data indicated that in monocomponent systems, the model of pseudo-second order (PSO) adequately described both CIP and OFL kinetics. Furthermore, with the implementation of Artificial Neural Networks (ANN), it was possible to obtain a more assertive prediction of the behavior of single and binary systems. [Display omitted] • Investigation of single and competitive adsorption of ciprofloxacin and ofloxacin. • Carbon nanotubes were functionalized with iron nanoparticles via the green route. • Ofloxacin showed a greater adsorption preference than ciprofloxacin. • The main adsorption mechanisms were hydrogen bonds, π-π, and n-π interactions. • Artificial neural network successfully predicted single and binary kinetic data. [ABSTRACT FROM AUTHOR]

We introduce a set of benchmark corpora of conversational English speech derived from the Switchboard-I and Fisher datasets. Traditional automatic speech recognition (ASR) research requires considerable computational resources and has slow experimental turnaround times. Our goal is to introduce these new datasets to researchers in the ASR and machine learning communities in order to facilitate the development of novel speech recognition techniques on smaller but still acoustically rich, diverse, and hence interesting corpora. We select these corpora to maximize an acoustic quality criterion while limiting the vocabulary size (from 10 words up to 10,000 words), where both “acoustic quality” and vocabulary size are adeptly measured via various submodular functions. We also survey numerous submodular functions that could be useful to measure both “acoustic quality” and “corpus complexity” and offer guidelines on when and why a scientist may wish use to one vs. another. The corpora selection process itself is naturally performed using various state-of-the-art submodular function optimization procedures, including submodular level-set constrained submodular optimization (SCSC/SCSK), difference-of-submodular (DS) optimization, and unconstrained submodular minimization (SFM), all of which are fully defined herein. While the focus of this paper is on the resultant speech corpora, and the survey of possible objectives, a consequence of the paper is a thorough empirical comparison of the relative merits of these modern submodular optimization procedures. We provide baseline word recognition results on all of the resultant speech corpora for both Gaussian mixture model (GMM) and deep neural network (DNN)-based systems, and we have released all of the corpora definitions and Kaldi training recipes for free in the public domain. [ABSTRACT FROM AUTHOR]

Highlights • Automatic metric for evaluating natural language generated sentences for dialog systems. • Integration of adequacy and fluency information to jointly evaluate semantic and syntactic information. • Higher correlation (up to 12.8%) with human evaluations than the best current objective metrics at a system and sentence level. End-to-end dialog systems are gaining interest due to the recent advances of deep neural networks and the availability of large human–human dialog corpora. However, in spite of being of fundamental importance to systematically improve the performance of this kind of systems, automatic evaluation of the generated dialog utterances is still an unsolved problem. Indeed, most of the proposed objective metrics shown low correlation with human evaluations. In this paper, we evaluate a two-dimensional evaluation metric that is designed to operate at sentence level, which considers the syntactic and semantic information carried along the answers generated by an end-to-end dialog system with respect to a set of references. The proposed metric, when applied to outputs generated by the systems participating in track 2 of the DSTC-6 challenge, shows a higher correlation with human evaluations (up to 12.8% relative improvement at the system level) than the best of the alternative state-of-the-art automatic metrics currently available. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have recently been successful in many applications and have become a popular approach for speech recognition. Training a DNN model for speech recognition is computationally expensive due to the model large number of parameters. Pre-training improves DNN modeling. However, DNN learning is challenging if pre-training is inefficient. This paper introduces a new framework for pre-training that utilizes label information in lower layers (layers near input) for better recognition. The proposed pre-training method dynamically inserts discriminative information not only in the last layer but also in other layers. In this algorithm, the lower layers achieve more generative information while the higher layers achieve more discriminative information. In addition, this method uses speaker information by employing the Subspace Gaussian Mixture Model (SGMM), which improves recognition accuracy. Experimental results on TIMIT, MNIST, Switchboard, and English Broadcast News datasets show that this approach significantly outperforms current state-of-the-art methods such as the Deep Belief Network and the Deep Boltzmann Machine. Moreover, the proposed algorithm has minimal memory requirements. [ABSTRACT FROM AUTHOR]

Abstract In this paper, we introduce the Task Dependent Recurrent Entity Network (TDREN) to solve Dialogue System Technology Challenges 6 (DSTC 6) track 1. Traditionally, there have been methods such as collecting the intent of the user in a conversation directly using rules. We design an end-to-end structure that properly models the restaurant pre-related user preferences that appear in the dialogue and gives appropriate responses. We perform experiments on the TDREN and achieved 97.7% at precision 1. We propose a new artificial neural network structure and recurrent cell for modeling user preference information. Then, we show that task-oriented dialogue modeling experiment results using the structure and the recurrent cell. [ABSTRACT FROM AUTHOR]

Former comparisons of human speech recognition (HSR) and automatic speech recognition (ASR) have shown that humans outperform ASR systems in nearly all speech recognition tasks. However, recent progress in ASR has led to substantial improvements of recognition accuracy, and it is therefore unclear how large the task-dependent human-machine gap still remains. This paper investigates this gap between HSR and ASR based on deep neural networks (DNNs) in different acoustic conditions, with the aim of comparing differences and identifying processing strategies that should be considered in ASR. We find that DNN-based ASR reaches human performance for single-channel, small-vocabulary tasks in the presence of speech-shaped noise and in multi-talker babble noise, which is an important difference to previous human-machine comparisons: The speech reception threshold, i.e., the signal-to-noise ratio with 50% word recognition rate is at about −7 to −8 dB both for HSR and ASR. However, in more complex spatial scenes with diffuse noise and moving talkers, the SRT gap amounts to approximately 12 dB. Based on cross comparisons that use oracle knowledge (e.g., the speakers’ true position), incorrect responses are attributed to localization errors or missing pitch information to distinguish between speakers with different gender. In terms of the SRT, localization errors and missing spectral information amount to 2.1 and 3.2 dB, respectively. The comparison hence identifies specific components in ASR that can profit from learning from auditory signal processing. [ABSTRACT FROM AUTHOR]

Viewing dialogue management as a reinforcement learning task enables a system to learn to act optimally by maximising a reward function. This reward function is designed to induce the system behaviour required for the target application and for goal-oriented applications, this usually means fulfilling the user’s goal as efficiently as possible. However, in real-world spoken dialogue system applications, the reward is hard to measure because the user’s goal is frequently known only to the user. Of course, the system can ask the user if the goal has been satisfied but this can be intrusive. Furthermore, in practice, the accuracy of the user’s response has been found to be highly variable. This paper presents two approaches to tackling this problem. Firstly, a recurrent neural network is utilised as a task success predictor which is pre-trained from off-line data to estimate task success during subsequent on-line dialogue policy learning. Secondly, an on-line learning framework is described whereby a dialogue policy is jointly trained alongside a reward function modelled as a Gaussian process with active learning. This Gaussian process operates on a fixed dimension embedding which encodes each varying length dialogue. This dialogue embedding is generated in both a supervised and unsupervised fashion using different variants of a recurrent neural network. The experimental results demonstrate the effectiveness of both off-line and on-line methods. These methods enable practical on-line training of dialogue policies in real-world applications. [ABSTRACT FROM AUTHOR]

This paper addresses the issue of language model adaptation for Recurrent Neural Network Language Models ( rnnlm s), which have recently emerged as a state-of-the-art method for language modeling in the area of speech recognition. Curriculum learning is an established machine learning approach that achieves better models by applying a curriculum, i.e., a well-planned ordering of the training data, during the learning process. Our contribution is to demonstrate the importance of curriculum learning methods for adapting rnnlm s and to provide key insights on how they should be applied. rnnlm s model language in a continuous space and can theoretically exploit word-dependency information over arbitrarily long distances. These characteristics give rnnlm s the ability to learn patterns robustly with relatively little training data, implying that they are well suited for adaptation. In this paper, we focus on two related challenges facing language models: within-domain adaptation and limited-data within-domain adaptation . We propose three types of curricula that start with general data, i.e., characterizing the domain as a whole, and move towards specific data, i.e., characterizing the sub-domain targeted for adaptation. Effectively, these curricula result in a model that can be considered to represent an implicit interpolation between general data and sub-domain-specific data. We carry out an extensive set of experiments that investigates how adapting rnnlm s using curriculum learning can improve their performance. Our first set of experiments addresses the within-domain adaptation challenge, i.e., creating models that are adapted to specific sub-domains that are part of a larger, heterogeneous domain of speech data. Under this challenge, all training data is available to the system at the time when the language model is trained. First, we demonstrate that curriculum learning can be used to create effective sub-domain-adapted rnnlm s. Second, we show that a combination of sub-domain-adapted rnnlm s can be used if the sub-domain of the target data is unknown at test time. Third, we explore the potential of applying combinations of sub-domain-adapted rnnlm s to data for which sub-domain information is unknown at training time and must be inferred. Our second set of experiments addresses limited-data within-domain adaptation, i.e., adapting an existing model trained on a large set of data using a smaller amount of data from the target sub-domain. Under this challenge, data from the target sub-domain is not available at the time when the language model is trained, but rather becomes available little by little over time. We demonstrate that the implicit interpolation carried out by applying curriculum learning methods to rnnlm s outperforms conventional interpolation and has the potential to make more of less adaptation data. [ABSTRACT FROM AUTHOR]

• In this paper, an optimal approach for text feature Selection , we work on text categorization and propose a statistical-based feature selection method (MFX) that considers all documents from the same category as one extended document, and chooses the most discriminative terms that are frequent and common across all documents of the same category, but rarely present in other categories. MFX is language independent and backed up with a mathematical formulation that finds the optimal number of features that guarantees accurate text categorization. Experimental results show the superiority of MFX over the state-of-the-art existing techniques. This work is very significant and timely given its applicability in applications such as spam filtering, opinion mining and topic spotting, among others. Traditionally, feature selection is conducted by first deriving a candidate list of features, then ranking and selecting the top features based on predefined threshold. These methods are highly dependent on the choice of the threshold, and therefore lead to sub-optimal text categorization results. In this paper, we address the selection problem by suggesting a one-step method designed to optimally select the subset of features. The selection is formulated mathematically as an optimization problem with the objective of maximizing classification accuracy while simultaneously deriving and choosing the most discriminative features. Our method, MFX, is applicable to many of the conventional methods, with two distinguishing aspects. First, it is based on considering all documents from the same category as one extended document, instead of analyzing individual documents. Second, it considers choosing the most discriminative terms that are frequent and common across all documents of the same category, and minimally present in other categories. Moreover, MFX is language-independent. It was tested on the well-known benchmark Reuters RCV1 dataset. To showcase its language independence, MFX was also tested on Arabic datasets extracted from Arabic news sources. The results indicated that MFX always performed similar to or better than other well-known feature selection methods. MFX with a Support Vector Machine (SVM) classifier was also shown to outperform recent text classification algorithms based on neural networks and word embeddings. [ABSTRACT FROM AUTHOR]

This paper examines the individual and combined impacts of various front-end approaches on the performance of deep neural network (DNN) based speech recognition systems in distant talking situations, where acoustic environmental distortion degrades the recognition performance. Training of a DNN-based acoustic model consists of generation of state alignments followed by learning the network parameters. This paper first shows that the network parameters are more sensitive to the speech quality than the alignments and thus this stage requires improvement. Then, various front-end robustness approaches to addressing this problem are categorised based on functionality. The degree to which each class of approaches impacts the performance of DNN-based acoustic models is examined experimentally. Based on the results, a front-end processing pipeline is proposed for efficiently combining different classes of approaches. Using this front-end, the combined effects of different classes of approaches are further evaluated in a single distant microphone-based meeting transcription task with both speaker independent (SI) and speaker adaptive training (SAT) set-ups. By combining multiple speech enhancement results, multiple types of features, and feature transformation, the front-end shows relative performance gains of 7.24% and 9.83% in the SI and SAT scenarios, respectively, over competitive DNN-based systems using log mel-filter bank features. [ABSTRACT FROM AUTHOR]

In this paper, we present a survey on the application of recurrent neural networks to the task of statistical language modeling. Although it has been shown that these models obtain good performance on this task, often superior to other state-of-the-art techniques, they suffer from some important drawbacks, including a very long training time and limitations on the number of context words that can be taken into account in practice. Recent extensions to recurrent neural network models have been developed in an attempt to address these drawbacks. This paper gives an overview of the most important extensions. Each technique is described and its performance on statistical language modeling, as described in the existing literature, is discussed. Our structured overview makes it possible to detect the most promising techniques in the field of recurrent neural networks, applied to language modeling, but it also highlights the techniques for which further research is required. [ABSTRACT FROM AUTHOR]

Deep Neural Networks (DNN) have become a powerful, and extremely popular mechanism, which has been widely used to solve problems of varied complexity, due to their ability to make models fitted to non-linear complex problems. Despite its well-known benefits, DNNs are complex learning models whose parametrisationand architecture are made usually by hand. This paper proposes a new Evolutionary Algorithm, named EvoDeep , devoted to evolve the parameters and the architecture of a DNN in order to maximise its classification accuracy, as well as maintaining a valid sequence of layers. This model is tested against a widely used dataset of handwritten digits images. The experiments performed using this dataset show that the Evolutionary Algorithm is able to select the parameters and the DNN architecture appropriately, achieving a 98.93% accuracy in the best run. [ABSTRACT FROM AUTHOR]