Search

In a unified fashion, we establish Lieb-Schultz-Mattis (LSM) theorems and their generalizations in systems with long-range interactions. We show that, for a quantum spin chain, if the interactions decay fast enough as their ranges increase and the Hamiltonian has an anomalous symmetry, the Hamiltonian cannot have a unique gapped symmetric ground state. If the Hamiltonian contains only 2-spin interactions, these theorems hold when the interactions decay faster than $1/r^2$, with $r$ the distance between the two interacting spins. Moreover, any pure state with an anomalous symmetry, which may not be a ground state of any natural Hamiltonian, must be long-range entangled. The symmetries we consider include on-site internal symmetries combined with lattice translation symmetries, and they can also extend to purely internal but non-on-site symmetries. Moreover, these internal symmetries can be discrete or continuous. We explore the applications of the theorems through various examples., Comment: 4.5 pages + supplemental material

Antiferromagnets have attracted significant attention in the field of magnonics, as promising candidates for ultralow-energy carriers for information transfer for future computing. The role of crystalline orientation distribution on magnon transport has received very little attention. In multiferroics such as BiFeO$_3$ the coupling between antiferromagnetic and polar order imposes yet another boundary condition on spin transport. Thus, understanding the fundamentals of spin transport in such systems requires a single domain, a single crystal. We show that through Lanthanum(La) substitution, a single ferroelectric domain can be engineered with a stable, single-variant spin cycloid, controllable by an electric field. The spin transport in such a single domain displays a strong anisotropy, arising from the underlying spin cycloid lattice. Our work shows a pathway to understand the fundamental origins of spin transport in such a single domain multiferroic., Comment: 15 pages, 9 figure

We introduce a synthetic Mach-Zehnder interferometer for digitized quantum computing devices to probe fractional exchange statistics of anyonic excitations that appear in quantum spin liquids. Employing an IonQ quantum computer, we apply this scheme to the toric ladder, a quasi-one-dimensional reduction of the toric code. We observe interference patterns resulting from the movement of `electric' excitations in the presence and absence of `magnetic' ones. We model the noise in IonQ via depolarizing Lindbladian dynamics, and find quantitative agreement with the measurements obtained from the quantum device. The synthetic Mach-Zehnder interferometer can thus also serve as an effective means to probe the coherence length and time scales of multi-qubit noisy quantum devices., Comment: 5 pages, 2 figures

Spin waves in magnetic materials are promising information carriers for future computing technologies due to their ultra-low energy dissipation and long coherence length. Antiferromagnets are strong candidate materials due, in part, to their stability to external fields and larger group velocities. Multiferroic aniferromagnets, such as BiFeO$_3$ (BFO), have an additional degree of freedom stemming from magnetoelectric coupling, allowing for control of the magnetic structure, and thus spin waves, with electric field. Unfortunately, spin-wave propagation in BFO is not well understood due to the complexity of the magnetic structure. In this work, we explore long-range spin transport within an epitaxially engineered, electrically tunable, one-dimensional (1D) magnonic crystal. We discover a striking anisotropy in the spin transport parallel and perpendicular to the 1D crystal axis. Multiscale theory and simulation suggests that this preferential magnon conduction emerges from a combination of a population imbalance in its dispersion, as well as anisotropic structural scattering. This work provides a pathway to electrically-reconfigurable magnonic crystals in antiferromagnets.

The emergence of multi-petawatt laser facilities is expected to push forward the maximum energy gain that can be achieved in a single stage of a LWFA to tens of GeV, which begs the question - is it likely to impact particle physics by providing a truly compact particle collider? Colliders have very stringent requirements on beam energy, acceleration efficiency and beam quality. In this article, we propose a LWFA scheme that can for the first time simultaneously achieve hitherto unrealized acceleration efficiency from the laser to the electron beam of >20% and a sub-one percent energy spread using a stepwise plasma structure and a nonlinearly chirped laser pulse. Three-dimensional high-fidelity simulations show that the nonlinear chirp can effectively mitigate the laser waveform distortion and lengthen the acceleration distance. This combined with an inter-stage rephasing process in the stepwise plasma can triple the beam energy gain compared to that in a uniform plasma for a fixed laser energy thereby dramatically increasing the efficiency. A dynamic beam loading effect can almost perfectly cancel the energy chirp that arises during the acceleration, leading to the sub-percent energy spread. This scheme is highly scalable and can be applied to peta-watt LWFA scenarios. Scaling laws are obtained that suggest electron beams with energy gain of >100 GeV, charge of 2 nC, and with an energy spread <1% can be realized with a high laser pulse to particle beam energy transfer efficiency in a LWFA driven by a peta-watt laser, which could be the basis for a proof of concept of one arm of a future electron-positron collider.

A key challenge in antiferromagnetic spintronics is the control of spin configuration on nanometer scales applicable to solid-state technologies. Bismuth ferrite (BiFeO3) is a multiferroic material that exhibits both ferroelectricity and canted antiferromagnetism at room temperature, making it a unique candidate in the development of electric-field controllable magnetic devices. The magnetic moments in BiFeO3 are arranged into a spin cycloid, resulting in unique magnetic properties which are tied to the ferroelectric order. Previous understanding of this coupling has relied on average, mesoscale measurements to infer behavior. Using nitrogen vacancy-based diamond magnetometry, we show that the spin cycloid can be deterministically controlled with an electric field. The energy landscape of the cycloid is shaped by both the ferroelectric degree of freedom and strain-induced anisotropy, restricting the magnetization changes to specific ferroelectric switching events. This study provides understanding of the antiferromagnetic texture in BiFeO3 and paves new avenues for designing magnetic textures and spintronic devices., Comment: 28 pages, 4 main text figures, 8 supp figures

Recent advances in programmable quantum devices brought to the fore the intriguing possibility of using them to realise and investigate topological quantum spin liquid phases. This new and exciting direction brings about important research questions on how to probe and determine the presence of such exotic, highly entangled phases. One of the most promising tools is investigating the behaviour of the topological excitations, and in particular their fractional statistics. In this work we put forward a generic route to achieve this, and we illustrate it in the specific case of $\mathbb{Z}_2$ topological spin liquids implemented with the aid of combinatorial gauge symmetry. We design a convenient architecture to study signatures of fractional statistics via quasiparticle interferometry, and we assess its robustness to diagonal and off-diagonal disorder, as well as to dephasing -- effects that are generally pervasive in noisy quantum programmable devices. A useful counterpart of our scheme is that it provides a clear test of the `quantumness' of these devices, since the signatures that we are looking for crucially hinge on quantum coherence and quantum interference effects in the system., Comment: 16 pages, 13 figures

In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to segment the audio and then transcribe each segmentation. These two stages are addressed separately by speaker change detection (SCD) and automatic speech recognition (ASR). Most previous SCD systems rely solely on speaker information and ignore the importance of speech content. In this paper, we propose a novel SCD system that considers both cues of speaker difference and speech content. These two cues are converted into token-level representations by the continuous integrate-and-fire (CIF) mechanism and then combined for detecting speaker changes on the token acoustic boundaries. We evaluate the performance of our approach on a public real-recorded meeting dataset, AISHELL-4. The experiment results show that our method outperforms a competitive frame-level baseline system by 2.45% equal coverage-purity (ECP). In addition, we demonstrate the importance of speech content and speaker difference to the SCD task, and the advantages of conducting SCD on the token acoustic boundaries compared with conducting SCD frame by frame.

Plasma wakefield acceleration in the nonlinear blowout regime has been shown to provide high acceleration gradients and high energy transfer efficiency while maintaining great beam quality for electron acceleration. In contrast, research on positron acceleration in this regime is still in a preliminary stage. We find that an on-axis electron filament can be self-consistently formed and maintained by loading an intense positron beam at the back of the electron beam driven blowout cavity. Via an analytic model and fully nonlinear simulations, we show this coaxial electron filament not only can focus the positron beam but changes the loaded longitudinal wakefield in a distinctly different way from electron beam loading in the blowout regime. Using simulations, we demonstrate that a high charge positron beam can be accelerated with tens of percent energy transfer from wake to positrons, percent level induced energy spread and several mm$\cdot$mrad normalized emittance, while significantly depleting the energy of the electron drive beam. This concept can be extended to simultaneous acceleration of electron and positron beams and high transformer ratio positron acceleration as well.

Computational guidance is an emerging and accelerating trend in aerospace guidance and control. Combining machine learning and convex optimization, this paper presents a real-time computational guidance method for the 6-degrees-of-freedom powered landing guidance problem. The powered landing guidance problem is formulated as an optimal control problem, which is then transformed into a convex optimization problem. Instead of brutally using the neural networks as the controller, we use neural networks to improve the state-of-the-art sequential convex programming (SCP) algorithm. Based on the deep neural network, an initial trajectory generator is designed to provide a satisfactory initial guess for the SCP algorithm. Benefitting from designing the initial trajectory generator as a sequence model predictor, the proposed data-driven SCP architecture is capable of improving the performance of any state-of-the-art SCP algorithm in various applications, not just powered landing guidance. The simulation results show that the proposed method can precisely guide the vehicle to the landing site. Moreover, through Monte Carlo tests, the proposed method can averagely save 40.8% of the computation time compared with the SCP method, while ensuring higher terminal states accuracy. The proposed computational guidance scheme is suitable for real-time applications., Comment: 13 pages, 10 figures

Although the development of spintronic devices has advanced significantly over the past decade with the use of ferromagnetic materials, the extensive implementation of such devices has been limited by the notable drawbacks of these materials. Antiferromagnets claim to resolve many of these shortcomings leading to faster, smaller, more energy-efficient, and more robust electronics. Antiferromagnets exhibit many desirable properties including zero net magnetization, imperviousness to external magnetic fields, intrinsic high-frequency dynamics with a characteristic precession frequency on the order of terahertz (THz), and the ability to serve as passive exchange-bias materials in multiple magnetoresistance (MR)- based devices. In this Perspective article, we will discuss the fundamental physics of magnetic structures in antiferromagnets and their interactions with external stimuli such as spin current, voltage, and magnons. A discussion on the challenges lying ahead is also provided along with an outlook of future research directions of these systems., Comment: Insufficient discussions on electrical manipulation and detection of antiferromagnetic order

Nowadays, most methods in end-to-end contextual speech recognition bias the recognition process towards contextual knowledge. Since all-neural contextual biasing methods rely on phrase-level contextual modeling and attention-based relevance modeling, they may encounter confusion between similar context-specific phrases, which hurts predictions at the token level. In this work, we focus on mitigating confusion problems with fine-grained contextual knowledge selection (FineCoS). In FineCoS, we introduce fine-grained knowledge to reduce the uncertainty of token predictions. Specifically, we first apply phrase selection to narrow the range of phrase candidates, and then conduct token attention on the tokens in the selected phrase candidates. Moreover, we re-normalize the attention weights of most relevant phrases in inference to obtain more focused phrase-level contextual representations, and inject position information to better discriminate phrases or tokens. On LibriSpeech and an in-house 160,000-hour dataset, we explore the proposed methods based on a controllable all-neural biasing method, collaborative decoding (ColDec). The proposed methods provide at most 6.1% relative word error rate reduction on LibriSpeech and 16.4% relative character error rate reduction on the in-house dataset over ColDec., Comment: Accepted by ICASSP 2022

We propose a novel positron beam loading regime in a hollow plasma channel that can efficiently accelerate $e^+$ beam with high gradient and narrow energy spread. In this regime, the $e^+$ beam coincides with the drive $e^-$ beam in time and space and their net current distribution determines the plasma wakefields. By precisely shaping the beam current profile and loading phase according to explicit expressions, three-dimensional Particle-in-Cell (PIC) simulations show that the acceleration for $e^+$ beam of $\sim$nC charge with $\sim$GV/m gradient, $\lesssim$0.5% induced energy spread and $\sim$50% energy transfer efficiency can be achieved simultaneously. Besides, only tailoring the current profile of the more tunable $e^-$ beam instead of the $e^+$ beam is enough to obtain such favorable results. A theoretical analysis considering both linear and nonlinear plasma responses in hollow plasma channels is proposed to quantify the beam loading effects. This theory agrees very well with the simulation results and verifies the robustness of this beam loading regime over a wide range of parameters.

In this paper, we propose an Omni-perception Pre-Trainer (OPT) for cross-modal understanding and generation, by jointly modeling visual, text and audio resources. OPT is constructed in an encoder-decoder framework, including three single-modal encoders to generate token-based embeddings for each modality, a cross-modal encoder to encode the correlations among the three modalities, and two cross-modal decoders to generate text and image respectively. For the OPT's pre-training, we design a multi-task pretext learning scheme to model multi-modal resources from three different data granularities, \ie, token-, modality-, and sample-level modeling, through which OPT learns to align and translate among different modalities. The pre-training task is carried out on a large amount of image-text-audio triplets from Open Images. Experimental results show that OPT can learn strong image-text-audio multi-modal representations and achieve promising results on a variety of cross-modal understanding and generation tasks.

End-to-end models have achieved impressive results on the task of automatic speech recognition (ASR). For low-resource ASR tasks, however, labeled data can hardly satisfy the demand of end-to-end models. Self-supervised acoustic pre-training has already shown its amazing ASR performance, while the transcription is still inadequate for language modeling in end-to-end models. In this work, we fuse a pre-trained acoustic encoder (wav2vec2.0) and a pre-trained linguistic encoder (BERT) into an end-to-end ASR model. The fused model only needs to learn the transfer from speech to language during fine-tuning on limited labeled data. The length of the two modalities is matched by a monotonic attention mechanism without additional parameters. Besides, a fully connected layer is introduced for the hidden mapping between modalities. We further propose a scheduled fine-tuning strategy to preserve and utilize the text context modeling ability of the pre-trained linguistic encoder. Experiments show our effective utilizing of pre-trained modules. Our model achieves better recognition performance on CALLHOME corpus (15 hours) than other end-to-end models.

There are several domains that own corresponding widely used feature extractors, such as ResNet, BERT, and GPT-x. These models are usually pre-trained on large amounts of unlabeled data by self-supervision and can be effectively applied to downstream tasks. In the speech domain, wav2vec2.0 starts to show its powerful representation ability and feasibility of ultra-low resource speech recognition on the Librispeech corpus, which belongs to the audiobook domain. However, wav2vec2.0 has not been examined on real spoken scenarios and languages other than English. To verify its universality over languages, we apply pre-trained models to solve low-resource speech recognition tasks in various spoken languages. We achieve more than 20% relative improvements in six languages compared with previous work. Among these languages, English achieves a gain of 52.4%. Moreover, using coarse-grained modeling units, such as subword or character, achieves better results than fine-grained modeling units, such as phone or letter.

End-to-end (E2E) models have achieved promising results on multiple speech recognition benchmarks, and shown the potential to become the mainstream. However, the unified structure and the E2E training hamper injecting contextual information into them for contextual biasing. Though contextual LAS (CLAS) gives an excellent all-neural solution, the degree of biasing to given context information is not explicitly controllable. In this paper, we focus on incorporating context information into the continuous integrate-and-fire (CIF) based model that supports contextual biasing in a more controllable fashion. Specifically, an extra context processing network is introduced to extract contextual embeddings, integrate acoustically relevant context information and decode the contextual output distribution, thus forming a collaborative decoding with the decoder of the CIF-based model. Evaluated on the named entity rich evaluation sets of HKUST/AISHELL-2, our method brings relative character error rate (CER) reduction of 8.83%/21.13% and relative named entity character error rate (NE-CER) reduction of 40.14%/51.50% when compared with a strong baseline. Besides, it keeps the performance on original evaluation set without degradation., Comment: Accepted by ICASSP 2021

Wav2vec 2.0 is a recently proposed self-supervised framework for speech representation learning. It follows a two-stage training process of pre-training and fine-tuning, and performs well in speech recognition tasks especially ultra-low resource cases. In this work, we attempt to extend self-supervised framework to speaker verification and language identification. First, we use some preliminary experiments to indicate that wav2vec 2.0 can capture the information about the speaker and language. Then we demonstrate the effectiveness of wav2vec 2.0 on the two tasks respectively. For speaker verification, we obtain a new state-of-the-art result, Equal Error Rate (EER) of 3.61% on the VoxCeleb1 dataset. For language identification, we obtain an EER of 12.02% on 1 second condition and an EER of 3.47% on full-length condition of the AP17-OLR dataset. Finally, we utilize one model to achieve the unified modeling by the multi-task learning for the two tasks., Comment: Self-supervised, speaker verification, language identification, multi-task learning, wav2vec 2.0

Plasma wakefield acceleration in the blowout regime is particularly promising for high-energy acceleration of electron beams because of its potential to simultaneously provide large acceleration gradients and high energy transfer efficiency while maintaining excellent beam quality. However, no equivalent regime for positron acceleration in plasma wakes has been discovered to-date. We show that after a short propagation distance, an asymmetric electron beam drives a stable wakefield in a hollow plasma channel that can be both accelerating and focusing for a positron beam. A high charge positron bunch placed at a suitable distance behind the drive bunch can beam-load or flatten the longitudinal wakefield and enhance the transverse focusing force, leading to high-efficiency and narrow energy spread acceleration of the positrons. Three-dimensional quasi-static particle-in-cell (PIC) simulations show that over 30% energy extraction efficiency from the wake to the positrons and 1% level energy spread can be simultaneously obtained, and further optimization is feasible.

Prediction of events such as part replacement and failure events plays a critical role in reliability engineering. Event stream data are commonly observed in manufacturing and teleservice systems. Designing predictive models for individual units based on such event streams is challenging and an under-explored problem. In this work, we propose a non-parametric prognostic framework for individualized event prediction based on the inhomogeneous Poisson processes with a multivariate Gaussian convolution process (MGCP) prior on the intensity functions. The MGCP prior on the intensity functions of the inhomogeneous Poisson processes maps data from similar historical units to the current unit under study which facilitates sharing of information and allows for analysis of flexible event patterns. To facilitate inference, we derive a variational inference scheme for learning and estimation of parameters in the resulting MGCP modulated Poisson process model. Experimental results are shown on both synthetic data as well as real-world data for fleet based event prediction., Comment: 9 pages, 9 figures

We build and probe a $\mathbb{Z}_2$ spin liquid in a programmable quantum device, the D-Wave DW-2000Q. Specifically, we observe the classical 8-vertex and 6-vertex (spin ice) states and transitions between them. To realize this state of matter, we design a Hamiltonian with combinatorial gauge symmetry using only pairwise-qubit interactions and a transverse field, i.e., interactions which are accessible in this quantum device. The combinatorial gauge symmetry remains exact along the full quantum annealing path, landing the system onto the classical 8-vertex model at the endpoint of the path. The output configurations from the device allows us to directly observe the loop structure of the classical model. Moreover, we deform the Hamiltonian so as to vary the weights of the 8 vertices and show that we can selectively attain the classical 6-vertex (ice) model, or drive the system into a ferromagnetic state. We present studies of the classical phase diagram of the system as function of the 8-vertex deformations and effective temperature, which we control by varying the relative strengths of the programmable couplings, and we show that the experimental results are consistent with theoretical analysis. Finally, we identify additional capabilities that, if added to these devices, would allow us to realize $\mathbb{Z}_2$ quantum spin liquids on which to build topological qubits., Comment: 6 pages, 5 figures

End-to-end models are gaining wider attention in the field of automatic speech recognition (ASR). One of their advantages is the simplicity of building that directly recognizes the speech frame sequence into the text label sequence by neural networks. According to the driving end in the recognition process, end-to-end ASR models could be categorized into two types: label-synchronous and frame-synchronous, each of which has unique model behaviour and characteristic. In this work, we make a detailed comparison on a representative label-synchronous model (transformer) and a soft frame-synchronous model (continuous integrate-and-fire (CIF) based model). The results on three public dataset and a large-scale dataset with 12000 hours of training data show that the two types of models have respective advantages that are consistent with their synchronous mode., Comment: 4 pages, 2 figures

Recently, end-to-end (E2E) models become a competitive alternative to the conventional hybrid automatic speech recognition (ASR) systems. However, they still suffer from speaker mismatch in training and testing condition. In this paper, we use Speech-Transformer (ST) as the study platform to investigate speaker aware training of E2E models. We propose a model called Speaker-Aware Speech-Transformer (SAST), which is a standard ST equipped with a speaker attention module (SAM). The SAM has a static speaker knowledge block (SKB) that is made of i-vectors. At each time step, the encoder output attends to the i-vectors in the block, and generates a weighted combined speaker embedding vector, which helps the model to normalize the speaker variations. The SAST model trained in this way becomes independent of specific training speakers and thus generalizes better to unseen testing speakers. We investigate different factors of SAM. Experimental results on the AISHELL-1 task show that SAST achieves a relative 6.5% CER reduction (CERR) over the speaker-independent (SI) baseline. Moreover, we demonstrate that SAST still works quite well even if the i-vectors in SKB all come from a different data source other than the acoustic training set.

This paper proposes a novel approach to pre-train encoder-decoder sequence-to-sequence (seq2seq) model with unpaired speech and transcripts respectively. Our pre-training method is divided into two stages, named acoustic pre-trianing and linguistic pre-training. In the acoustic pre-training stage, we use a large amount of speech to pre-train the encoder by predicting masked speech feature chunks with its context. In the linguistic pre-training stage, we generate synthesized speech from a large number of transcripts using a single-speaker text to speech (TTS) system, and use the synthesized paired data to pre-train decoder. This two-stage pre-training method integrates rich acoustic and linguistic knowledge into seq2seq model, which will benefit downstream automatic speech recognition (ASR) tasks. The unsupervised pre-training is finished on AISHELL-2 dataset and we apply the pre-trained model to multiple paired data ratios of AISHELL-1 and HKUST. We obtain relative character error rate reduction (CERR) from 38.24% to 7.88% on AISHELL-1 and from 12.00% to 1.20% on HKUST. Besides, we apply our pretrained model to a cross-lingual case with CALLHOME dataset. For all six languages in CALLHOME dataset, our pre-training method makes model outperform baseline consistently.

Clifford circuits are insufficient for universal quantum computation or creating $t$-designs with $t\ge 4$. While the entanglement entropy is not a telltale of this insufficiency, the entanglement spectrum is: the entanglement levels are Poisson-distributed for circuits restricted to the Clifford gate-set, while the levels follow Wigner-Dyson statistics when universal gates are used. In this paper we show, using finite-size scaling analysis of different measures of level spacing statistics, that in the thermodynamic limit, inserting a single T $(\pi/8)$ gate in the middle of a random Clifford circuit is sufficient to alter the entanglement spectrum from a Poisson to a Wigner-Dyson distribution., Comment: 14 pages, 8 figures; Submission to SciPost

Recently there has been an increasing interest in the multivariate Gaussian process (MGP) which extends the Gaussian process (GP) to deal with multiple outputs. One approach to construct the MGP and account for non-trivial commonalities amongst outputs employs a convolution process (CP). The CP is based on the idea of sharing latent functions across several convolutions. Despite the elegance of the CP construction, it provides new challenges that need yet to be tackled. First, even with a moderate number of outputs, model building is extremely prohibitive due to the huge increase in computational demands and number of parameters to be estimated. Second, the negative transfer of knowledge may occur when some outputs do not share commonalities. In this paper we address these issues. We propose a regularized pairwise modeling approach for the MGP established using CP. The key feature of our approach is to distribute the estimation of the full multivariate model into a group of bivariate GPs which are individually built. Interestingly pairwise modeling turns out to possess unique characteristics, which allows us to tackle the challenge of negative transfer through penalizing the latent function that facilitates information sharing in each bivariate model. Predictions are then made through combining predictions from the bivariate models within a Bayesian framework. The proposed method has excellent scalability when the number of outputs is large and minimizes the negative transfer of knowledge between uncorrelated outputs. Statistical guarantees for the proposed method are studied and its advantageous features are demonstrated through numerical studies.

End-to-end models have been showing superiority in Automatic Speech Recognition (ASR). At the same time, the capacity of streaming recognition has become a growing requirement for end-to-end models. Following these trends, an encoder-decoder recurrent neural network called Recurrent Neural Aligner (RNA) has been freshly proposed and shown its competitiveness on two English ASR tasks. However, it is not clear if RNA can be further improved and applied to other spoken language. In this work, we explore the applicability of RNA in Mandarin Chinese and present four effective extensions: In the encoder, we redesign the temporal down-sampling and introduce a powerful convolutional structure. In the decoder, we utilize a regularizer to smooth the output distribution and conduct joint training with a language model. On two Mandarin Chinese conversational telephone speech recognition (MTS) datasets, our Extended-RNA obtains promising performance. Particularly, it achieves 27.7% character error rate (CER), which is superior to current state-of-the-art result on the popular HKUST task., Comment: To appear in Interspeech 2018

Sequence-to-sequence attention-based models integrate an acoustic, pronunciation and language model into a single neural network, which make them very suitable for multilingual automatic speech recognition (ASR). In this paper, we are concerned with multilingual speech recognition on low-resource languages by a single Transformer, one of sequence-to-sequence attention-based models. Sub-words are employed as the multilingual modeling unit without using any pronunciation lexicon. First, we show that a single multilingual ASR Transformer performs well on low-resource languages despite of some language confusion. We then look at incorporating language information into the model by inserting the language symbol at the beginning or at the end of the original sub-words sequence under the condition of language information being known during training. Experiments on CALLHOME datasets demonstrate that the multilingual ASR Transformer with the language symbol at the end performs better and can obtain relatively 10.5\% average word error rate (WER) reduction compared to SHL-MLSTM with residual learning. We go on to show that, assuming the language information being known during training and testing, about relatively 12.4\% average WER reduction can be observed compared to SHL-MLSTM with residual learning through giving the language symbol as the sentence start token., Comment: arXiv admin note: text overlap with arXiv:1805.06239

The choice of modeling units is critical to automatic speech recognition (ASR) tasks. Conventional ASR systems typically choose context-dependent states (CD-states) or context-dependent phonemes (CD-phonemes) as their modeling units. However, it has been challenged by sequence-to-sequence attention-based models, which integrate an acoustic, pronunciation and language model into a single neural network. On English ASR tasks, previous attempts have already shown that the modeling unit of graphemes can outperform that of phonemes by sequence-to-sequence attention-based model. In this paper, we are concerned with modeling units on Mandarin Chinese ASR tasks using sequence-to-sequence attention-based models with the Transformer. Five modeling units are explored including context-independent phonemes (CI-phonemes), syllables, words, sub-words and characters. Experiments on HKUST datasets demonstrate that the lexicon free modeling units can outperform lexicon related modeling units in terms of character error rate (CER). Among five modeling units, character based model performs best and establishes a new state-of-the-art CER of $26.64\%$ on HKUST datasets without a hand-designed lexicon and an extra language model integration, which corresponds to a $4.8\%$ relative improvement over the existing best CER of $28.0\%$ by the joint CTC-attention based encoder-decoder network., Comment: arXiv admin note: substantial text overlap with arXiv:1804.10752

Sequence-to-sequence attention-based models have recently shown very promising results on automatic speech recognition (ASR) tasks, which integrate an acoustic, pronunciation and language model into a single neural network. In these models, the Transformer, a new sequence-to-sequence attention-based model relying entirely on self-attention without using RNNs or convolutions, achieves a new single-model state-of-the-art BLEU on neural machine translation (NMT) tasks. Since the outstanding performance of the Transformer, we extend it to speech and concentrate on it as the basic architecture of sequence-to-sequence attention-based model on Mandarin Chinese ASR tasks. Furthermore, we investigate a comparison between syllable based model and context-independent phoneme (CI-phoneme) based model with the Transformer in Mandarin Chinese. Additionally, a greedy cascading decoder with the Transformer is proposed for mapping CI-phoneme sequences and syllable sequences into word sequences. Experiments on HKUST datasets demonstrate that syllable based model with the Transformer performs better than CI-phoneme based counterpart, and achieves a character error rate (CER) of \emph{$28.77\%$}, which is competitive to the state-of-the-art CER of $28.0\%$ by the joint CTC-attention based encoder-decoder network., Comment: accepted by INTERSPEECH2018