Search

This paper describes the experimental setups and the evaluation results of the sixth Dialog System Technology Challenges (DSTC6) aiming to develop end-to-end dialogue systems. Neural network models have become a recent focus of investigation in dialogue technologies. Previous models required training data to be manually annotated with word meanings and dialogue states, but end-to-end neural network dialogue systems learn to directly output natural-language system responses without needing training data to be manually annotated. Thus, this approach allows us to scale up the size of training data and cover more dialog domains. In addition, dialogue systems require a meta-function to avoid deploying inappropriate responses generated by themselves. To challenge such issues, the DSTC6 consists of three tracks, (1). End-to-End Goal Oriented dialogue Learning to select system responses, (2). End-to-End Conversation Modeling to generate system responses using Natural Language Generation (NLG) and (3). Dialogue Breakdown Detection. Since each domain has different issues to be addressed to develop dialogue systems, we targeted restaurant retrieval dialogues to fill slot-value in Track 1, customer services on Twitter by combining goal-oriented dialogues and ChitChat in Track 2 and human-machine dialogue data for ChitChat in Track 3. DSTC6 had 141 people declaring their interests and 23 teams submitted their final results. 18 scientific papers were presented in the wrap-up workshop. We find the blending end-to-end trainable models associated to meaningful prior knowledge performs the best for the restaurant retrieval for Track 1. Indeed, Hybrid Code Network and Memory Network have been the best models for this task. In Track 2, 78.5% of the system responses automatically generated by the best system were rated better than acceptable by humans and this achieves 89% of the number of the human responses rated in the same class. In Track3, the dialogue breakdown detection technologies performed as well as human agreements, in both data-sets of English and Japanese.

This paper presents adversarial training and decoding methods for neural conversation models that can generate natural responses given dialog contexts. In our prior work, we built several end-to-end conversation systems for the 6th Dialog System Technology Challenges (DSTC6) Twitter help-desk dialog task. These systems included novel extensions of sequence adversarial training, example-based response extraction, and Minimum Bayes-Risk based system combination. In DSTC6, our systems achieved the best performance in most objective measures such as BLEU and METEOR scores and decent performance in a subjective measure based on human rating. In this paper, we provide a complete set of our experiments for DSTC6 and further extend the training and decoding strategies more focusing on improving the subjective measure, where we combine responses of three adversarial models. Experimental results demonstrate that the extended methods improve the human rating score and outperform the best score in DSTC6.

This paper gives an in-depth presentation of the multi-microphone speech recognition system we submitted to the 3rd CHiME speech separation and recognition challenge (CHiME-3) and its extension. The proposed system takes advantage of recurrent neural networks (RNNs) throughout the model from the front-end speech enhancement to the language modeling. Three different types of beamforming are used to combine multi-microphone signals to obtain a single higher-quality signal. The beamformed signal is further processed by a single-channel long short-term memory (LSTM) enhancement network, which is used to extract stacked mel-frequency cepstral coefficients (MFCC) features. In addition, the beamformed signal is processed by two proposed noise-robust feature extraction methods. All features are used for decoding in speech recognition systems with deep neural network (DNN) based acoustic models and large-scale RNN language models to achieve high recognition accuracy in noisy environments. Our training methodology includes multi-channel noisy data training and speaker adaptive training, whereas at test time model combination is used to improve generalization. Results on the CHiME-3 benchmark show that the full set of techniques substantially reduced the word error rate (WER). Combining hypotheses from different beamforming and robust-feature systems ultimately achieved 5.05% WER for the real-test data, an 84.7% reduction relative to the baseline of 32.99% WER and a 44.5% reduction from our official CHiME-3 challenge result of 9.1% WER. Furthermore, this final result is better than the best result (5.8% WER) reported in the CHiME-3 challenge.

Deep bidirectional RNNs (DBRNNs) are applied to ASR error detection and accuracy estimation.DBRNNs take longer bidirectional context of input feature vectors into account.DBRNNs model highly nonlinear relationships between input feature vectors and output labels.DBRNNs greatly outperform CRFs and other structures of neural networks. Recurrent neural networks (RNNs) have recently been applied as the classifiers for sequential labeling problems. In this paper, deep bidirectional RNNs (DBRNNs) are applied to error detection in automatic speech recognition (ASR), which is a sequential labeling problem. We investigate three types of ASR error detection tasks, i.e. confidence estimation, out-of-vocabulary word detection and error type classification. We also estimate ASR accuracy, i.e. percent correct and word accuracy, from the error type classification results. Experimental results using English and Japanese lecture speech corpora show that the DBRNNs greatly outperform conditional random fields (CRFs) and the other NN structures, i.e. deep feedforward NNs (DNNs) and deep unidirectional RNNs (DURNNs). These performance improvements are because the DBRNNs can take the longer bidirectional context of input feature vectors into account and can model highly nonlinear relationships between the input feature vectors and output labels. In detailed analyses, the DBRNNs show a better generalization ability than the CRFs. These results are thanks to the ability of the DBRNNs to represent (embed) the words in a low-dimensional continuous value vector space. In addition, the superiority of the DBRNNs to the DNNs and DURNNs indicates that the average length of the context of the input feature vectors required for ASR error detection is only a few time steps, however, it will change (lengthen) depending on the situation.

Research on noise robust speech recognition has mainly focused on dealing with relatively stationary noise that may differ from the noise conditions in most living environments. In this paper, we introduce a recognition system that can recognize speech in the presence of multiple rapidly time-varying noise sources as found in a typical family living room. To deal with such severe noise conditions, our recognition system exploits all available information about speech and noise; that is spatial (directional), spectral and temporal information. This is realized with a model-based speech enhancement pre-processor, which consists of two complementary elements, a multi-channel speech-noise separation method that exploits spatial and spectral information, followed by a single channel enhancement algorithm that uses the long-term temporal characteristics of speech obtained from clean speech examples. Moreover, to compensate for any mismatch that may remain between the enhanced speech and the acoustic model, our system employs an adaptation technique that combines conventional maximum likelihood linear regression with the dynamic adaptive compensation of the variance of the Gaussians of the acoustic model. Our proposed system approaches human performance levels by greatly improving the audible quality of speech and substantially improving the keyword recognition accuracy.

The purpose of this paper is to describe the development of a speaker adaptation method that improves speech recognition performance regardless of the amount of adaptation data. For that purpose, we propose the consistent employment of a maximum a posteriori (MAP)-based Bayesian estimation for both feature space normalization and model space adaptation. Namely, constrained structural maximum a posteriori linear regression (CSMAPLR) is first performed in a feature space to compensate for the speaker characteristics, and then, SMAPLR is performed in a model space to capture the remaining speaker characteristics. A prior distribution stabilizes the parameter estimation especially when the amount of adaptation data is small. In the proposed method, CSMAPLR and SMAPLR are performed based on the same acoustic model. Therefore, the dimension-dependent variations of feature and model spaces can be similar. Dimension-dependent variations of the transformation matrix are explained well by the prior distribution. Therefore, by sharing the same prior distribution between CSMAPLR and SMAPLR, their parameter estimations can be appropriately regularized in both spaces. Experiments on large vocabulary continuous speech recognition using the Corpus of Spontaneous Japanese (CSJ) and the MIT OpenCourseWare corpus (MIT-OCW) confirm the effectiveness of the proposed method compared with other conventional adaptation methods with and without using speaker adaptive training.

This paper focuses on discriminative language models (DLMs) for large vocabulary speech recognition tasks. To train such models, we usually use a large number of hypotheses generated for each utterance by a speech recognizer, namely an n-best list or a lattice. Since the data size is large, we usually need a high-end machine or a large-scale distributed computation system consisting of many computers for model training. However, it is still unclear whether or not such a large number of sentence hypotheses are necessary. Furthermore, we do not know which kinds of sentences are necessary. In this paper, we show that we can generate a high performance model using small subsets of the n-best lists by choosing samples properly, i.e., we describe a sample selection method for DLMs. Sample selection reduces the memory footprint needed for holding training samples and allows us to train models in a standard machine. Furthermore, it enables us to generate a highly accurate model using various types of features. Specifically, experimental results show that even training using two samples in each list can provide an accurate model with a small memory footprint.

In a real environment, acoustic and language features often vary depending on the speakers, speaking styles and topic changes. To accommodate these changes, speech recognition approaches that include the incremental tracking of changing environments have attracted attention. This paper proposes a topic tracking language model that can adaptively track changes in topics based on current text information and previously estimated topic models in an on-line manner. The proposed model is applied to language model adaptation in speech recognition. We use the MIT OpenCourseWare corpus and Corpus of Spontaneous Japanese in speech recognition experiments, and show the effectiveness of the proposed method.

A dependency structure interprets modification relationships between words and is often recognized as an important element in semantic information analysis. With conventional approaches for extracting this dependency structure, it is assumed that the complete sentence is known before the analysis starts. For spontaneous speech data, however, this assumption is not necessarily correct since sentence boundaries are not marked in the data and it is not easy to detect them correctly. Although sentence boundaries can be detected before dependency analysis, this cascaded implementation is not suitable for online processing since it delays the responses of the application. In this paper, we propose a sequential dependency analysis method for online spontaneous speech processing. The proposed method enables us to analyze incomplete sentences sequentially and detect sentence boundaries simultaneously. The analyzer can be trained using parsed data based on the maximum entropy principle. Experimental results using spontaneous lecture speech from the Corpus of Spontaneous Japanese show that our proposed method achieves online processing with an accuracy equivalent to that of offline processing in which boundary detection and dependency analysis are cascaded.

Conformations of two dodecameric porphyrin wheels adsorbed on a Cu(1 0 0) were probed by using scanning tunneling microscopy (STM). Whereas a wheel consisting of six meso – meso linked diporphyrins was detected as uniform ring structure, several different images with three discrete molecular heights were detected for a wheel consisting of six meso – meso , β-β,β-β triply-linked planar diporphyrins. These results indicate that the former has a conformation similar to that in a free space, while the latter has various conformations with respect to orientation of planar diporphyrin units toward the metal surface. Several discrete STM images of the latter have been interpreted in terms of possible eight conformations, which vary as to relative orientation of neighboring diporphyrin units.

We have explored a variety of covalently and non-covalently assembled cyclic porphyrin arrays mainly as biomimetic models of light harvesting antenna in photosynthetic systems. The key reaction is Ag(I)-promoted coupling reaction of 5,15-diaryl zinc(II) porphyrin that provides a meso–meso linked diporphyrin. An advantage of this coupling reaction is its extremely easy extension to higher porphyrin arrays, since longer porphyrin arrays have practically the same reactivity as that of the monomer. On the basis of this strategy, we have prepared cyclic porphyrin arrays including directly meso–meso linked porphyrin rings CZ4 – CZ8 , large porphyrin wheels C12ZA and C24ZB , and three-dimensional porphyrin boxes D1 – D3 . Efficient excitation energy transfer along these cyclic porphyrin arrays has been revealed by the time-resolved transient absorption and fluorescence measurements.

Deletion of 9p21 has frequently been observed in human bladder carcinomas. A candidate target suppressor gene, p16, was recently identified within this deleted region. In this study, we therefore investigated the loss of heterozygosity (LOH) of the p16 gene which is located on mouse chromosome 4, as well as its expression in mouse bladder carcinomas. We also studied the effects of normal cell contamination on LOH analysis using xenografts in CD-1(ICR) nude mice from B6C3F1 bladder carcinomas. We could not detect any LOH at the p16 locus in the mouse primary bladder carcinomas and xenografts. Surprisingly, overexpression of p16 was found in all primary mouse bladder carcinomas. Using microsatellite polymorphisms, a distinction could be made between PCR products derived from B6C3F1 and CD-1(ICR) nude mice. It was thereby confirmed that effects of normal cell contamination on LOH analysis are negligible when only tumor tissue is carefully sampled. The results suggest that abnormalities of p16 expression may be involved in mouse bladder carcinogenesis, but that gene deletion is not involved.

The effects of space restriction and water-immersion stress on epithelial cell proliferation in the digestive tract, with special attention to the esophagus, stomach and duodenum, in 8-week-old SD male rats were examined. Histological assessment revealed spotted hemorrhagic lesions in the fundus of the glandular stomach, accompanied by statistically increased 5-bromo-2'-deoxyuridine (BrdU) labeling index in the fundic and pyloric regions. Furthermore, biochemical analysis demonstrated an increased activity of ornithine decarboxylase (ODC) and spermidine/spermine N1-acetyltransferase (SAT), known as key late-limiting enzymes of the polyamine pathway, in the gastric fundus. The stress may induce a remarkable increase in expression of c-fos, c-jun and c-myc mRNAs in both fundic and pyloric regions of the glandular stomach. There were no remarkable changes in the esophagus. These results indicate that space restriction and water-immersion stress induced cell proliferation in the glandular stomach through overexpression of proto-oncogenes and increased ODC and SAT activities that might be related to the promotion of gastric carcinogenesis.