Search

For most automatic speech recognition systems, many unacceptable hypothesis errors still make the recognition results absurd and difficult to understand. In this paper, we introduce the grammar information to improve the performance of the grammatical deviation distance and increase the readability of the hypothesis. The reinforcement of word embedding with grammar embedding is presented to intensify the grammar expression. An auxiliary text-to-grammar task is provided to improve the performance of the recognition results with the downstream task evaluation. Furthermore, the multiple evaluation methodology of grammar is used to explore an expandable usage paradigm with grammar knowledge. Experiments on the small open-source Mandarin speech corpus AISHELL-1 and large private-source Mandarin speech corpus TRANS-M tasks show that our method can perform very well with no additional data. Our method achieves relative character error rate reductions of 3.2% and 5.0%, a relative grammatical deviation distance reduction of 4.7% and 5.9% on AISHELL-1 and TRANS-M tasks, respectively. Moreover, the grammar-based mean opinion score of our method is about 4.29 and 3.20, significantly superior to the baseline of 4.11 and 3.02.

Sign language recognition is a conversion of sign language into text or speech, bridging the communication between the hearing and society. Recently, sequence-to-sequence video to text (S2VT) models has been employed in the field of sign language recognition as an effective method. However, more than 20 million parameters trained in S2VT models will result in a huge consumption in memory and computational resources, making it hard to be applied in mobile devices. In order to overcome this issue, we proposed to employ tensor-train decomposition in S2VT models to reduce the parameters. First, the impact of parameters of tensor-train factorization on the model performance was investigated systematically. After that, we applied tensor-train decomposition in different layers of a S2VT model to establish 6 tensor-train S2VT models for Chinese sign language recognition. The experimental results demonstrated that when the fully-connected layer and the first LSTM layer in S2VT was represented with tensor-train format, the model could obtain the best performance, remaining high accuracy and reducing parameters and memory significantly. The proposed tensor-train S2VT models can also be applied in other sequence-to-sequence problems to improve the performance.

The sign language signal has hierarchically related information over short and long distances. Due to the intricate temporal correlation of input sequences, Chinese sign language recognition (SLR) has a modeling challenge. The conventional encoders based on recurrent networks cannot discover and leverage the hierarchical structure of sign language well. In this paper, we propose a novel encoder-decoder method based on boundary adaptive learning for Chinese SLR. The hierarchical structure of sign language signal can be encoded by the boundary-adaptive encoder (BAE) in the proposed method. In order to improve efficiency in modeling long sign language sequences, the window attention model based on location is utilized in the decoding phase, which can generate more effective weight coefficients. Besides, we use sign language subword units to realize both isolated and continuous Chinese SLR in the same sequence learning framework in our method. Theoretical analysis and experimental results demonstrate the effectiveness and superiority of the proposed method.

Sign language recognition(SLR) is a multidisciplinary research topic in pattern recognition and computer vision. Due to large amount of data from the continuous frames of sign language videos, selecting representative data to eliminate irrelevant information has always been a challenging problem in data preprocessing of sign language samples. In recent years, skeletal data emerged as a new type of data but received insufficient attention. Meanwhile, due to the increasing diversity of sign language features, making full use of them has also been an important research topic. In this paper, we improve keyframe-centered clips (KCC) sampling to get a new kind of sampling method called optimized keyframe-centered clips (OptimKCC) sampling to select key actions from sign language videos. Besides, we design a new kind of skeletal feature called Multi-Plane Vector Relation (MPVR) to describe the video samples. Finally, combined with the attention mechanism, we also use Attention-Based networks to distribute weights to the temporal features and the spatial features extracted from skeletal data. We implement comparison experiments on our own and the public sign language dataset under the Signer-Independent and the Signer-Dependent circumstances to show the advantages of our methods.

Recently, a number of robust adaptive beamforming (RAB) methods based on Capon power spectrum estimator integrated over a specific region for covariance matrix reconstruction have been proposed. However, all of these methods ignore the residual noise existing in the Capon spectrum estimator, which results in reconstruction errors. In this paper, we propose a RAB algorithm via residual noise elimination and interference powers estimation to reconstruct covariance matrix. First, the proposed algorithm demonstrates the existence of residual noise and analyze its relationship to actual noise. Then, after eliminating the residual noise, the modified Capon power spectrum estimator is utilized to reconstruct the covariance matrix and desired signal SV. Moreover, to reduce the influence of the desired signal on interference powers estimation, we project the snapshots onto the complementary subspace of the desired signal and estimated interference powers are derived according to the theoretical formulation of the interference covariance matrix (ICM). The simulation results demonstrate that the proposed method is robust against various mismatches and can achieve superior performance.

In this paper, a novel array structure exploiting coprime arrays is proposed, which comprises the translocation of one subarray and axis rotation with a compression of another subarray to produce a larger number of consecutive lags. This composition is named CATARCS, which stands for coprime array with translocated and axis rotated compressed subarrays. The scheme offers consecutive co-array lags in remarkable number $4MN-1$ by using only $2M+N-1$ number of sensors attaining higher degrees of freedom than the state of the art, where $M$ and $N$ are coprime with congenial interelement spacings. However, a very few holes still exist in the co-array, which can be filled up through interpolation. In this paper, the iterative power factorization (IPF) algorithm has been demonstrated as a tool for incremented-rank-IPF matrix recovery which is unostentatious. No extra tuning parameter is required for this approach and several undesired issues of other techniques can be reconciled by this. The main contribution of this paper is suitable modification of array geometry arrangements to increase consecutive lags and to interpolate via IPF, which is used first time for direction of arrival (DOA) estimation in order to exhibit better performances in terms of detecting and resolving a larger number of sources by enhancing resolution. Later, DOA estimation can be achieved by performing multiple signal classification (MUSIC) algorithm. Simulation results validate its effectiveness by achieving lower cost, faster, and accurate DOA estimation even in lower angular separation of sources compared with other techniques like spatial smoothing MUSIC, co-array the least absolute shrinkage and selection operator, spline, iterative co-array array interpolation, and Toeplitz completion.

In this paper, a novel array structure exploiting coprime arrays is proposed which can be very proficient to determine the number of consecutive lags in proportion with the number of array elements. The proposed method comprises novel array structure by configuring three subarrays positioned in alignment with some prescribed values. By increasing array elements in third subarray while keeping other subarrays fixed, explicit number of consecutive lags could be obtained proportionately. The proposed method offers maximization of consecutive lags in remarkable number by calculating the fourth order difference co-array unifying interpolation. The forth order difference co-array is achieved by exploiting the second order difference co-array twice. The consideration of third subarray in addition with two coprime subarrays leads to a novel array structure which can significantly enhance degrees of freedom. An effective interpolation technique nuclear norm minimization is considered to fill the holes subsisting in the virtual co-array in order to exploit full virtual co-array length. This interpolation method uses convex framework which is trackable and very simple to implement yielding a freedom of fixing any predefined extra tuning parameter. The proposed method in this paper is named as VEFODCI which stands for virtual extension of coprime arrays by exploiting fourth order difference co-array with interpolation. The sparse Bayesian learning is used for direction of arrival (DOA) estimation exploiting the proposed novel array structure by imposing interpolation to fill the holes. The array geometry of sensor distributions proves that the proposed method is less susceptible from mutual coupling effect. The simulation results stipulate that the proposed method is performing DOA estimation accurately even in lower angular separation of sources by achieving larger number of consecutive lags than the state of the art.

This paper investigates the robustness properties of Pearson's rank-variate correlation coefficient (PRVCC) in scenarios where one channel is corrupted by impulsive noise and the other is impulsive noise-free. As shown in our previous work, these scenarios that frequently encountered in radar and/or sonar, can be well emulated by a particular bivariate contaminated Gaussian model (CGM). Under this CGM, we establish the asymptotic closed forms of the expectation and variance of PRVCC by means of the well known Delta method. To gain a deeper understanding, we also compare PRVCC with two other classical correlation coefficients, i.e., Spearman's rho (SR) and Kendall's tau (KT), in terms of the root mean squared error (RMSE). Monte Carlo simulations not only verify our theoretical findings, but also reveal the advantage of PRVCC by an example of estimating the time delay in the particular impulsive noise environment.