Search

Automatic target sound extraction (TSE) is a machine learning approach to mimic the human auditory perception capability of attending to a sound source of interest from a mixture of sources. It often uses a model conditioned on a fixed form of target sound clues, such as a sound class label, which limits the ways in which users can interact with the model to specify the target sounds. To leverage variable number of clues cross modalities available in the inference phase, including a video, a sound event class, and a text caption, we propose a unified transformer-based TSE model architecture, where a multi-clue attention module integrates all the clues across the modalities. Since there is no off-the-shelf benchmark to evaluate our proposed approach, we build a dataset based on public corpora, Audioset and AudioCaps. Experimental results for seen and unseen target-sound evaluation sets show that our proposed TSE model can effectively deal with a varying number of clues which improves the TSE performance and robustness against partially compromised clues., Accepted by ICASSP 2023

Direct speech-to-speech translation (S2ST) is an attractive research topic with many advantages compared to cascaded S2ST. However, direct S2ST suffers from the data scarcity problem because the corpora from speech of the source language to speech of the target language are very rare. To address this issue, we propose in this paper a Speech2S model, which is jointly pre-trained with unpaired speech and bilingual text data for direct speech-to-speech translation tasks. By effectively leveraging the paired text data, Speech2S is capable of modeling the cross-lingual speech conversion from source to target language. We verify the performance of the proposed Speech2S on Europarl-ST and VoxPopuli datasets. Experimental results demonstrate that Speech2S gets an improvement of about 5 BLEU scores compared to encoder-only pre-training models, and achieves a competitive or even better performance than existing state-of-the-art models1., Comment: Submitted to ICASSP 2023

We introduce a language modeling approach for text to speech synthesis (TTS). Specifically, we train a neural codec language model (called Vall-E) using discrete codes derived from an off-the-shelf neural audio codec model, and regard TTS as a conditional language modeling task rather than continuous signal regression as in previous work. During the pre-training stage, we scale up the TTS training data to 60K hours of English speech which is hundreds of times larger than existing systems. Vall-E emerges in-context learning capabilities and can be used to synthesize high-quality personalized speech with only a 3-second enrolled recording of an unseen speaker as an acoustic prompt. Experiment results show that Vall-E significantly outperforms the state-of-the-art zero-shot TTS system in terms of speech naturalness and speaker similarity. In addition, we find Vall-E could preserve the speaker's emotion and acoustic environment of the acoustic prompt in synthesis. See https://aka.ms/valle for demos of our work., Comment: Working in progress

Joint speech-language training is challenging due to the large demand for training data and GPU consumption, as well as the modality gap between speech and language. We present ComSL, a speech-language model built atop a composite architecture of public pretrained speech-only and language-only models and optimized data-efficiently for spoken language tasks. Particularly, we propose to incorporate cross-modality learning into transfer learning and conduct them simultaneously for downstream tasks in a multi-task learning manner. Our approach has demonstrated effectiveness in end-to-end speech-to-text translation tasks, achieving a new state-of-the-art average BLEU score of 31.5 on the multilingual speech to English text translation task for 21 languages, as measured on the public CoVoST2 evaluation set.

We propose a cross-lingual neural codec language model, VALL-E X, for cross-lingual speech synthesis. Specifically, we extend VALL-E and train a multi-lingual conditional codec language model to predict the acoustic token sequences of the target language speech by using both the source language speech and the target language text as prompts. VALL-E X inherits strong in-context learning capabilities and can be applied for zero-shot cross-lingual text-to-speech synthesis and zero-shot speech-to-speech translation tasks. Experimental results show that it can generate high-quality speech in the target language via just one speech utterance in the source language as a prompt while preserving the unseen speaker's voice, emotion, and acoustic environment. Moreover, VALL-E X effectively alleviates the foreign accent problems, which can be controlled by a language ID. Audio samples are available at \url{https://aka.ms/vallex}., Comment: We encourage readers to listen to the audio samples on our demo page: \url{https://aka.ms/vallex}

We propose gated language experts and curriculum training to enhance multilingual transformer transducer models without requiring language identification (LID) input from users during inference. Our method incorporates a gating mechanism and LID loss, enabling transformer experts to learn language-specific information. By combining gated transformer experts with shared transformer layers, we construct multilingual transformer blocks and utilize linear experts to effectively regularize the joint network. The curriculum training scheme leverages LID to guide the gated experts in improving their respective language performance. Experimental results on a bilingual task involving English and Spanish demonstrate significant improvements, with average relative word error reductions of 12.5% and 7.3% compared to the baseline bilingual and monolingual models, respectively. Notably, our method achieves performance comparable to the upper-bound model trained and inferred with oracle LID. Extending our approach to trilingual, quadrilingual, and pentalingual models reveals similar advantages to those observed in the bilingual models, highlighting its ease of extension to multiple languages.

The increasing prevalence of obesity worldwide is associated with an increased risk of various diseases, including multiple metabolic diseases, cardiovascular diseases, and malignant tumors. Fatty acid binding proteins (FABPs) are members of the adipokine family of multifunctional proteins that are related to fatty acid metabolism and are divided into 12 types according to their tissue origin. FABP4 is mainly secreted by adipocytes and macrophages. Under obesity, the synthesis of FABP4 increases, and the FABP4 content is higher not only in tissues but also in the blood, which promotes the occurrence and development of various cancers. Here, we comprehensively investigated obesity epidemiology and the biological mechanisms associated with the functions of FABP4 that may explain this effect. In this review, we explore the molecular mechanisms by which FABP4 promotes carcinoma development and the interaction between fat and cancer cells in obese circumstances here. This review leads us to understand how FABP4 signaling is involved in obesity-associated tumors, which could increase the potential for advancing novel therapeutic strategies and molecular targets for the systematic treatment of malignant tumors.

The advances in attention-based encoder-decoder (AED) networks have brought great progress to end-to-end (E2E) automatic speech recognition (ASR). One way to further improve the performance of AED-based E2E ASR is to introduce an extra text encoder for leveraging extensive text data and thus capture more context-aware linguistic information. However, this approach brings a mismatch problem between the speech encoder and the text encoder due to the different units used for modeling. In this paper, we propose an embedding aligner and modality switch training to better align the speech and text latent spaces. The embedding aligner is a shared linear projection between text encoder and speech encoder trained by masked language modeling (MLM) loss and connectionist temporal classification (CTC), respectively. The modality switch training randomly swaps speech and text embeddings based on the forced alignment result to learn a joint representation space. Experimental results show that our proposed approach achieves a relative 14% to 19% word error rate (WER) reduction on Librispeech ASR task. We further verify its effectiveness on spoken language understanding (SLU), i.e., an absolute 2.5% to 2.8% F1 score improvement on SNIPS slot filling task., submitted to ICASSP 2022

Based on the South China Sea deepwater high temperature and high pressure gas reservoir LS25 gas field, 3D geological model and a numerical reservoir model of LS25 gas field are established. Through numerical simulation of gas reservoir production, quantitative analysis of water invasion characteristics of 11 wells has been completed. The whole process of water invasion in space is as follows: bottom water conies in the early stage, forming water cone shape, the scale of the water body to the ridge increases, and the horizontal section completely emerges. Through comparative study, it is concluded that the combined water control technology of variable density screen pipe has the best water control effect. According to the prediction and analysis of the final production plan of variable density technology, it can prolong the water-free gas production period of the gas reservoir by 0.7 years, prolong the total production time by 1.9 years, increase the gas production by 1.9×108m3, and increase the recovery factor by 10.5% compared with that without water control measures. The research results are aimed to establish an optimal water control development model for buried hill gas reservoirs, and provide a good technical support for reasonable and scientific water control research of horizontal wells in offshore buried hill fractured gas reservoirs.

Automatic speech recognition (ASR) and speech translation (ST) can both use neural transducers as the model structure. It is thus possible to use a single transducer model to perform both tasks. In real-world applications, such joint ASR and ST models may need to be streaming and do not require source language identification (i.e. language-agnostic). In this paper, we propose LAMASSU, a streaming language-agnostic multilingual speech recognition and translation model using neural transducers. Based on the transducer model structure, we propose four methods, a unified joint and prediction network for multilingual output, a clustered multilingual encoder, target language identification for encoder, and connectionist temporal classification regularization. Experimental results show that LAMASSU not only drastically reduces the model size but also reaches the performances of monolingual ASR and bilingual ST models., Comment: INTERSPEECH 2023

How to boost speech pre-training with textual data is an unsolved problem due to the fact that speech and text are very different modalities with distinct characteristics. In this paper, we propose a cross-modal Speech and Language Model (SpeechLM) to explicitly align speech and text pre-training with a pre-defined unified discrete representation. Specifically, we introduce two alternative discrete tokenizers to bridge the speech and text modalities, including phoneme-unit and hidden-unit tokenizers, which can be trained using a small amount of paired speech-text data. Based on the trained tokenizers, we convert the unlabeled speech and text data into tokens of phoneme units or hidden units. The pre-training objective is designed to unify the speech and the text into the same discrete semantic space with a unified Transformer network. Leveraging only 10K text sentences, our SpeechLM gets a 16\% relative WER reduction over the best base model performance (from 6.8 to 5.7) on the public LibriSpeech ASR benchmark. Moreover, SpeechLM with fewer parameters even outperforms previous SOTA models on CoVoST-2 speech translation tasks. We also evaluate our SpeechLM on various spoken language processing tasks under the universal representation evaluation framework SUPERB, demonstrating significant improvements on content-related tasks. Our code and models are available at https://aka.ms/SpeechLM., We discovered some errors about the data in the pre-training experiments, which will affect all the results about SpeechLM-P Base models. We are re-conducting the related experiments and will update the paper with the new results

Traditional automatic speech recognition~(ASR) systems usually focus on individual utterances, without considering long-form speech with useful historical information, which is more practical in real scenarios. Simply attending longer transcription history for a vanilla neural transducer model shows no much gain in our preliminary experiments, since the prediction network is not a pure language model. This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor. We propose the {LongFNT-Text} architecture, which fuses the sentence-level long-form features directly with the output of the vocabulary predictor and then embeds token-level long-form features inside the vocabulary predictor, with a pre-trained contextual encoder RoBERTa to further boost the performance. Moreover, we propose the {LongFNT} architecture by extending the long-form speech to the original speech input and achieve the best performance. The effectiveness of our LongFNT approach is validated on LibriSpeech and GigaSpeech corpora with 19% and 12% relative word error rate~(WER) reduction, respectively., Comment: Submitted to ICASSP2023

This study proposed a new gelation method to prepare apatite/attapulgite/alginate composite hydrogels with high structural stability for effective adsorption of methylene blue (MB) from aqueous sol...

Endometrioid Endometrial Cancer (EEC) is the main subtype of endometrial cancer. In our study, we demonstrated that SPTBN2 was significantly overexpressed in EEC tissues. Upregulated SPTBN2 expression was positively associated with poor prognosis. In addition, we testified that SPTBN2 knockdown significantly inhibited the proliferation, migration, and invasion of EEC cells. Moreover, we found SPTBN2 could interact with CLDN4 to promote endometrial cancer metastasis via PI3K/AKT pathway. Then we further demonstrated that CLDN4 is upregulated in EEC and promotes EEC metastasis. CLDN4 overexpression could partially reversed the decrease in cell migration and invasion caused by SPTBN2 downregulation. In addition, we confirmed that SPTBN2 was a target of miR-424-5p, which plays a tumor suppressor in endometrial cancer. Rescue experiments showed that inhibition of SPTBN2 could partially reverse the effect of miR-424-5p in EEC. In conclusion, we demonstrated that by acting as a significant target of miR-424-5p, SPTBN2 could interact with CLDN4 to promote endometrial cancer metastasis via PI3K/AKT pathway in EEC. Our study revealed the prognostic and metastatic effects of SPTBN2 in EEC, suggesting that SPTBN2 could serve as a prognostic biomarker and a target for metastasis therapy.

Deepwater gas wells usually have high production rates, which result in high-speed sand movement along with the gas flow and acid components such as CO2 in the gas flow. The erosion and corrosion effect intensifies the damage to sand screens and can further lead to sand control failures, which endanger the safety of production operations. In this paper, using a differential rotation device to simulate erosion, corrosion, and erosion-corrosion, experiments observing the impacts of several factors on sand screens were carried out. The factors include CO2 partial pressure, temperature, flow velocity, sand content, and sand particle size. Their impacts on erosion, corrosion, and erosion-corrosion rate are inspected independently, and the sensitivity of factors to the erosion-corrosion rate of a sand screen was determined using the range method. Traditional erosion models involving flow rate, sand content, and sand grain size and traditional corrosion models involving CO2 partial pressure and temperature are taken as references, and an erosion-corrosion coupled model of sand screens is established based on the experimental results using the multivariate regression analysis. With critical damage thickness taken as the failure criterion of a sand screen, a prediction method for sand screen failure is formed and is applied to the case study of the S deepwater gas field. The results show that under the experimental conditions, the screen loss rate due to erosion-corrosion is significantly higher than that of erosion only or corrosion only and is even higher than the sum of both. The erosion-corrosion rate increases with the increase of CO2 partial pressure, temperature, flow velocity, sand content, and sand particle size. It has an exponential relationship with the negative reciprocal of temperature, a linear relationship with the partial pressure of CO2, and power fitting relationships with flow velocity, sand content, and sand particle size. The ranking of sensitivities to the erosion-corrosion rate of high-quality screens is as follows: sand content > flow velocity > temperature > sand particle size > CO2 partial pressure. The error between the predicted results by the proposed model and the experimental results is in the range of 0.44-9.47%. The erosion-corrosion rate of sand screens in each production well of the S gas field is in the range of 0.0111-0.0521 mm/a, while the durability of the screens is 14-68 years. The erosion-corrosion rate model of a sand screen and the prediction method for sand screen failure proposed in this paper provide theoretical support to the durability evaluation of sand screen in deepwater gas wells, which is of great significance for ensuring the safe and efficient development of offshore oil and gas resources.

Immense oil and gas reserves exist in the deepwater of the South China Sea, and the first priority for implementing successful drilling and completion operations without reservoir damage is to understand the damaging mechanisms to the reservoir formation. With the use of deepwater reservoir cores from the western South China Sea, the mineral composition, microstructure, porosity, and permeability characteristics of the reservoir formation were analyzed. With the use of core flow tests, potential fluid sensitivity was analyzed to determine how and which fluids can damage the reservoir formation during drilling and completion. The results show that the reservoir formation is composed of unconsolidated siltstone with a clay mineral content of 6–19%, and has high porosity and permeability with large pore throats. Sensitivity damage due to the increasing flow rate is strong, and an unusual salinity sensitivity exists with a critical salinity at 40,000–45,000 mg/L. Moderate alkali fluids sensitivity damage exists, with water sensitivity being weak. The main mechanisms of formation damage by drilling and completion fluids were analyzed as following: plugging of pore throats by solid invasion occurs because the sizes of the solid materials in the drilling fluids are just ideal for the formation of an external filter. Rock-fluid incompatibilities occur due to the increasing flow rate and salinity of the fluids because the formation rock is unconsolidated and the fines will easily detach and migrate, plugging the pore throats. Therefore, effective temporary plugging of the pore throats with a large diameter is important for protecting the reservoir. This study provides basic data for the design and implementation of reservoir protection measures in deepwater oil and gas exploration.

Permeability stress-sensitivity is the critical characteristic in reservoir development, especially for coalbed methane (CBM) reservoir. To investigate the permeability evolution during production in Shouyang CBM reservoir, Qinshui Basin, China, a new stress-sensitive permeability analytical model is proposed, while a time-lapse three-dimensional (4D) multi-physical coupling model is constructed. Heterogeneity and anisotropy of the reservoir flow and geomechanical properties as well as the permeability stress-sensitivity are adequately considered in the model. A coupling method is proposed to bridge data between finite difference (FD) grids and finite element (FE) grids with an interface code complied by Python. The reservoir flow data and geomechanical data are connected with each other by the interface code. Permeability stress-sensitivity experiments are carried out and compared with the numerical simulation results. According to the dynamic simulation results of 230 producing days, the stress increases with pore pressure decline, and the permeability stress-sensitivity is very strong in Shouyang 15# coalbed, especially in the initial stage of compression. The dynamic evolutions of stress and permeability are significantly affected by the anisotropy and heterogeneity of coalbed during depletion. The proposed stress-dependent permeability model is more suitable for characterizing the permeability evolution of the Shouyang CBM reservoir.

Although pre-training models have achieved great success in dialogue generation, their performance drops dramatically when the input contains an entity that does not appear in pre-training and fine-tuning datasets (unseen entity). To address this issue, existing methods leverage an external knowledge base to generate appropriate responses. In real-world scenario, the entity may not be included by the knowledge base or suffer from the precision of knowledge retrieval. To deal with this problem, instead of introducing knowledge base as the input, we force the model to learn a better semantic representation by predicting the information in the knowledge base, only based on the input context. Specifically, with the help of a knowledge base, we introduce two auxiliary training objectives: 1) Interpret Masked Word, which conjectures the meaning of the masked entity given the context; 2) Hypernym Generation, which predicts the hypernym of the entity based on the context. Experiment results on two dialogue corpus verify the effectiveness of our methods under both knowledge available and unavailable settings., EMNLP 2021

Multilingual automatic speech recognition (ASR) models have shown great promise in recent years because of the simplified model training and deployment process. Conventional methods either train a universal multilingual model without taking any language information or with a 1-hot language ID (LID) vector to guide the recognition of the target language. In practice, the user can be prompted to pre-select several languages he/she can speak. The multilingual model without LID cannot well utilize the language information set by the user while the multilingual model with LID can only handle one pre-selected language. In this paper, we propose a novel configurable multilingual model (CMM) which is trained only once but can be configured as different models based on users' choices by extracting language-specific modules together with a universal model from the trained CMM. Particularly, a single CMM can be deployed to any user scenario where the users can pre-select any combination of languages. Trained with 75K hours of transcribed anonymized Microsoft multilingual data and evaluated with 10-language test sets, the proposed CMM improves from the universal multilingual model by 26.0%, 16.9%, and 10.4% relative word error reduction when the user selects 1, 2, or 3 languages, respectively. CMM also performs significantly better on code-switching test sets.

Recently, there has been a vast interest in self-supervised learning (SSL) where the model is pre-trained on large scale unlabeled data and then fine-tuned on a small labeled dataset. The common wisdom is that SSL helps resource-limited tasks in which only a limited amount of labeled data is available. The benefit of SSL keeps diminishing when the labeled training data amount increases. To our best knowledge, at most a few thousand hours of labeled data was used in the study of SSL. In contrast, the industry usually uses tens of thousands of hours of labeled data to build high-accuracy speech recognition (ASR) systems for resource-rich languages. In this study, we take the challenge to investigate whether and how SSL can improve the ASR accuracy of a state-of-the-art production-scale Transformer-Transducer model, which was built with 65 thousand hours of anonymized labeled EN-US data.

The speech representations learned from large-scale unlabeled data have shown better generalizability than those from supervised learning and thus attract a lot of interest to be applied for various downstream tasks. In this paper, we explore the limits of speech representations learned by different self-supervised objectives and datasets for automatic speaker verification (ASV), especially with a well-recognized SOTA ASV model, ECAPA-TDNN [1], as a downstream model. The representations from all hidden layers of the pre-trained model are firstly averaged with learnable weights and then fed into the ECAPA-TDNN as input features. The experimental results on Voxceleb dataset show that the weighted average representation is significantly superior to FBank, a conventional handcrafted feature for ASV. Our best single system achieves 0.537%, 0.569%, and 1.180% equal error rate (EER) on the three official trials of VoxCeleb1, separately. Accordingly, the ensemble system with three pre-trained models can further improve the EER to 0.479%, 0.536% and 1.023%. Among the three evaluation trials, our best system outperforms the winner system [2] of the VoxCeleb Speaker Recognition Challenge 2021 (VoxSRC2021) on the VoxCeleb1-E trial., Comment: Accepted by ICASSP 2022

South China Sea is one of the offshore HPHT region, with the maximum 2.38 pressure coefficient, 480 degF bottom temperature and 50% CO2. The drilling and completion are challenging problems. This paper reviews the past 20 years' researches and practices in South China Sea HPHT area. Based on theoretical study, laboratory tests and site experiments, the project team has been working on solving problems of formation pressure prediction, wellbore integrity, testing and operation technology. The understanding and achievements are abundant. First, an accurate prediction method of multi-source abnormal pressure was built and a quantitative control drilling technique was developed, which helps relize continuous circulation in 0.01∼0.02 g/cm3 density window. Second, the "self-repair" HPHT cement slurry systems have been created and the multi-barrier wellbore integrity protection technology has been constructed to implement normal pressure in casing annulus. Third, an intrinsically safe testing system, which includes 7 major factors, was created. The safety of offshore HPHT testing can be guaranteed. Fourth, a water-based high temperature resistant drilling fluid and series acceleration tools were created, realizing the high-quality and efficient HPHT field operation. This series of technology has helped build China's first HPHT gas field and can be a reference for similar area.

This paper is concerned with the formation of bitumen during the drilling of the H oilfield in Iraq. The high viscosity and strong adhesion properties of bitumen can influence the drilling operations. Some complex problems include paste screening, and drill pipe sticking, which cause huge economic losses. Therefore, it is necessary to effectively reduce the bitumen viscosity. The contribution of a single subcomponent of bitumen to the viscosity can vary, and the combined effect of different components of bitumen on the viscosity remains unclear. Furthermore, the mechanism of viscosity reduction remains unclear. In this study, the effects of organic solvents on the viscosity of bitumen were studied, and toluene was selected as the best organic solvent. The results showed that aromatics/resins, aromatics/asphaltenes, and resin/asphaltenes can help increase the bitumen viscosity. Novel methods, including the use of nanoparticles, ethyl cellulose, and the quaternary ammonium salt of heptadecenyl hydroxyethyl imidazoline (QASHI), were proposed to decrease the viscosity. TiO2 and CuO nanoparticles were chosen, and the main factors influencing the viscosity, such as the particle type, concentration, particle size, temperature, and shear rate, were analysed. The results show that the bitumen viscosity decreases with the increase in the concentrations of ethyl cellulose and QASHI. A synergistic effect between ethyl cellulose and QASHI was found with an optimal concentrations of ethyl cellulose and QASHI (1000 and 1600 mg L−1). A synergistic effect was also observed when nanoparticles, ethyl cellulose, and QASHI were used in combination. This paper reports the micro-mechanism whereby the viscosity of bitumen is decreased.

Pre-trained models for programming language have achieved dramatic empirical improvements on a variety of code-related tasks such as code search, code completion, code summarization, etc. However, existing pre-trained models regard a code snippet as a sequence of tokens, while ignoring the inherent structure of code, which provides crucial code semantics and would enhance the code understanding process. We present GraphCodeBERT, a pre-trained model for programming language that considers the inherent structure of code. Instead of taking syntactic-level structure of code like abstract syntax tree (AST), we use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables. Such a semantic-level structure is neat and does not bring an unnecessarily deep hierarchy of AST, the property of which makes the model more efficient. We develop GraphCodeBERT based on Transformer. In addition to using the task of masked language modeling, we introduce two structure-aware pre-training tasks. One is to predict code structure edges, and the other is to align representations between source code and code structure. We implement the model in an efficient way with a graph-guided masked attention function to incorporate the code structure. We evaluate our model on four tasks, including code search, clone detection, code translation, and code refinement. Results show that code structure and newly introduced pre-training tasks can improve GraphCodeBERT and achieves state-of-the-art performance on the four downstream tasks. We further show that the model prefers structure-level attentions over token-level attentions in the task of code search., Comment: Accepted by ICLR2021

With its strong modeling capacity that comes from a multi-head and multi-layer structure, Transformer is a very powerful model for learning a sequential representation and has been successfully applied to speech separation recently. However, multi-channel speech separation sometimes does not necessarily need such a heavy structure for all time frames especially when the cross-talker challenge happens only occasionally. For example, in conversation scenarios, most regions contain only a single active speaker, where the separation task downgrades to a single speaker enhancement problem. It turns out that using a very deep network structure for dealing with signals with a low overlap ratio not only negatively affects the inference efficiency but also hurts the separation performance. To deal with this problem, we propose an early exit mechanism, which enables the Transformer model to handle different cases with adaptive depth. Experimental results indicate that not only does the early exit mechanism accelerate the inference, but it also improves the accuracy.

Evaluation metrics play a vital role in the growth of an area as it defines the standard of distinguishing between good and bad models. In the area of code synthesis, the commonly used evaluation metric is BLEU or perfect accuracy, but they are not suitable enough to evaluate codes, because BLEU is originally designed to evaluate the natural language, neglecting important syntactic and semantic features of codes, and perfect accuracy is too strict thus it underestimates different outputs with the same semantic logic. To remedy this, we introduce a new automatic evaluation metric, dubbed CodeBLEU. It absorbs the strength of BLEU in the n-gram match and further injects code syntax via abstract syntax trees (AST) and code semantics via data-flow. We conduct experiments by evaluating the correlation coefficient between CodeBLEU and quality scores assigned by the programmers on three code synthesis tasks, i.e., text-to-code, code translation, and code refinement. Experimental results show that our proposed CodeBLEU can achieve a better correlation with programmer assigned scores compared with BLEU and accuracy., Comment: 8 pages, 6 figures

During the execution of technological operations such as multiple pressure relief or foam separation during well injection, a gas column can form at the wellhead in the annular space, leading to annular pressure buildup (APB). Based on the theory of heat transfer, a model was proposed for calculating the wellbore heat transfer during production. Taking into account the expansion and compression of the fluid, changes in the wellhead volume, the thermodynamic characteristics of the liquid, and the equation of state of the gas, APB calculations were performed. Based on these parameters, the influence of the gas column height on APB was evaluated. The results of the studies show that the coefficients of expansion and compressibility in the annulus depend on density, pressure and temperature: the higher the productivity of the gas well, the higher the temperature and pressure in the annulus; the greater the height of the gas column, the lower the pressure at the wellhead. For a well with a productivity of 30· 104 m3/day and gas column height of 70 m in the annulus, the APB is reduced to atmospheric pressure. Studies of the model for calculating APB of the annular trapped gas provide a theoretical basis for developing optimal technological parameters for the operation of gas wells.

This study investigated the aerobic sludge granulation and nitrogen removal performance in a modified airlift loop reactor treating municipal wastewater under different operation conditions. Dynamic feeding and aeration control were applied to create feast/famine conditions to facilitate microbial aggregation. Experimental results demonstrated that aerobic granular sludge could be cultivated in continuous-flow reactors fed with an optimized dynamic feeding condition. Fresh granules sizing 0.4–0.6 mm were observed in the reactors after a 61-day operation, then turned to matured granules after another 33-day operation with a compact structure, a stable size of 2–4 mm, and a low SVI of ~35 mL/g. Extracellular polymeric substances (EPS) analysis results showed that both EPS contents and the ratio of protein to polysaccharides increased with the granulation process, leading to an increase of cell hydrophobicity. Granular sludge exhibited a good nitrogen removal ability with a comparable level of specific nitrification rate and denitrification rate with those measured in state-of-the-art sequential batch reactors. Microbial population analysis showed an increase in the relative abundance of functional microbes, including Zoogloea, Nitrospira, Dechloromonas, and Thauera in the cultivated granules, suggesting a potentially crucial role of these microbes in sludge granulation and nitrogen removal. The dynamic feeding strategy and the reactor configuration are considered as critical factors for aerobic granulation under continuous-flow conditions for creating feast/famine conditions and allow sludge backflow without structure damage.

The minimum critical liquid carrying flow rate of gas wells is an important parameter in the process of making gas reservoir development plans. At present, Turner equation and Li Min equation in the field are mainly used to calculate the critical liquid carrying flow rate of gas wells. However, the two equations have some limitations, which do not consider the effect of interfacial tension on the critical liquid carrying flow rate. In calculation, the gas-water interfacial tension is simplified as a constant, but interfacial tension is a function of temperature and pressure. In order to calculate the minimum liquid carrying flow rate more accurately, the existing calculation equation is modified while considering the influence of temperature and pressure on interfacial tension. The equation of minimum production is obtained by deducing the minimum flow rate of liquid carrying in gas wells. The actual gas well data is used for example calculation, and the minimum production under different well depth, bottom hole temperature and pressure is analyzed. The results show that considering the interfacial tension when calculating the fluid carrying flow rate of gas wells, the calculated fluid carrying flow rate of gas wells is more realistic.

In evaluating the reservoir physical properties and boundary characteristics of gas wells, the well test interpretation method is mainly employed currently to obtain K, KH, S and boundary characteristics of the reservoir. This method requires a test system for gas wells and a pressure gauge to record the data of flowing bottom hole pressure (FBHP) at the middle depth of the gas reservoir, with long testing time and high cost. Therefore, productivity testing and test pressure recovery data are only performed for some gas wells, and we cannot get reservoir physical property parameters by well test interpretation. It is considered that this method can better meet the needs of practical work, and has high popularization and application value.

Recently, Transformer has achieved the state-of-the-art performance on many machine translation tasks. However, without syntax knowledge explicitly considered in the encoder, incorrect context information that violates the syntax structure may be integrated into source hidden states, leading to erroneous translations. In this paper, we propose a novel method to incorporate source dependencies into the Transformer. Specifically, we adopt the source dependency tree and define two matrices to represent the dependency relations. Based on the matrices, two heads in the multi-head self-attention module are trained in a supervised manner and two extra cross entropy losses are introduced into the training objective function. Under this training objective, the model is trained to learn the source dependency relations directly. Without requiring pre-parsed input during inference, our model can generate better translations with the dependency-aware context information. Experiments on bi-directional Chinese-to-English, English-to-Japanese and English-to-German translation tasks show that our proposed method can significantly improve the Transformer baseline., Comment: 6 pages

As an important foundation of integral impeller milling manufacturing, integral impeller milling process knowledge plays a significant role in design and production. With the rapid development of aerospace industry and intelligent manufacturing technology, it is very urgent to construct knowledge base of milling process for typical integral impeller structure. Based on the analysis of typical milling process knowledge of integral impeller, the study proposes a model of knowledge base of milling process of integral impeller based on Teamcenter platform, which can effectively realize the sharing and reuse of knowledge of integral impeller milling process.

High-pressure and high-temperature (HPHT) deepwater oil and gas wells are under the influence of high-temperature formation fluid after putting into production. The thermal expansion pressure of sealed annular fluids caused by the annulus temperature change has large effect on the safety of strings among dual packers. This study presented a casing-cement sheath-formation model which take into account the effect of the sealed annulus temperature and volume changes between dual packers, established a displacement function of the sealed annular fluid, the free tubing and the cemented casing that based on the thermo-elastic mechanics, analysed the influence factors of sealed annular pressure, including the expansion coefficient and elasticity modulus of annular fluid, the wall thickness of casing and tubing stings. An example of a HPHT deepwater gas well was analysed. The results show that the sealed annular pressure has a positive linear correlation with the annulus temperature difference, and annular pressure buildup with the increase of thermal expansion coefficient and elasticity modulus of the annulus fluid, decreases with the casing wall thickness. To prevent the potential risk associated with annular pressure, high-compressibility and thermal-insulated materials could be developed to control sealed annulus thermal expansion pressure.

Language style transferring rephrases text with specific stylistic attributes while preserving the original attribute-independent content. One main challenge in learning a style transfer system is a lack of parallel data where the source sentence is in one style and the target sentence in another style. With this constraint, in this paper, we adapt unsupervised machine translation methods for the task of automatic style transfer. We first take advantage of style-preference information and word embedding similarity to produce pseudo-parallel data with a statistical machine translation (SMT) framework. Then the iterative back-translation approach is employed to jointly train two neural machine translation (NMT) based transfer systems. To control the noise generated during joint training, a style classifier is introduced to guarantee the accuracy of style transfer and penalize bad candidates in the generated pseudo data. Experiments on benchmark datasets show that our proposed method outperforms previous state-of-the-art models in terms of both accuracy of style transfer and quality of input-output correspondence.

Machine translation has made rapid advances in recent years. Millions of people are using it today in online translation systems and mobile applications in order to communicate across language barriers. The question naturally arises whether such systems can approach or achieve parity with human translations. In this paper, we first address the problem of how to define and accurately measure human parity in translation. We then describe Microsoft's machine translation system and measure the quality of its translations on the widely used WMT 2017 news translation task from Chinese to English. We find that our latest neural machine translation system has reached a new state-of-the-art, and that the translation quality is at human parity when compared to professional human translations. We also find that it significantly exceeds the quality of crowd-sourced non-professional translations.

In this study, the effects of air pressure, dissolved oxygen (DO) concentration and carbon–nitrogen (C/N) ratio on the treatment efficiency of a pilot-scale bioreactor consisted of pressurized aeration and dissolved air flotation (PA–DAF) were evaluated for domestic wastewater treatment. Results show that SS removal increased with an increased air pressure in the range of 0.10–0.50 MPa, and COD removal peaked at 86% for 0.40 MPa. At DO concentrations ranging from 0.5 to 5.5 mg/L, NH4+–N removal increased to up to 99% as DO concentration was increased, while a peak for TN removal was observed due to nitrification and denitrification. NH4+–N removal decreased while the C/N ratio was increased from 3 to 12, and a peak TN in removal was also observed. The optimal NH4+–N and TN removal efficiency achieved were 99% and 78% when the DO concentration was 2.5 mg/L and 3.5 mg/L, respectively. The NH4+–N removal efficiency was close to 100% at a C/N ratio of 3, and the maximal TN removal efficiency was 80% at a C/N ratio of 5. These results demonstrated the effectiveness of the bioreactor for domestic wastewater treatment.

The aim of this study was to determine the microbial community composition in the rumen of yaks under different feeding regimes. Microbial communities were assessed by sequencing bacterial and archaeal 16S ribosomal RNA gene fragments obtained from yaks (Bos grunniens) from Qinghai-Tibetan Plateau, China. Samples were obtained from 14 animals allocated to either pasture grazing (Graze), a grazing and supplementary feeding regime (GSF), or an indoor feeding regime (Feed). The predominant bacterial phyla across feeding regimes were Bacteroidetes (51.06%) and Firmicutes (32.73%). At genus level, 25 genera were shared across all samples. The relative abundance of Prevotella in the graze and GSF regime group were significantly higher than that in the feed regime group. Meanwhile, the relative abundance of Ruminococcus was lower in the graze group than the feed and GSF regime groups. The most abundant archaeal phylum was Euryarchaeota, which accounted for 99.67% of the sequences. Ten genera were detected across feeding regimes, seven genera were shared by all samples, and the most abundant was genus Methanobrevibacter (91.60%). The relative abundance of the most detected genera were similar across feeding regime groups. Our results suggest that the ruminal bacterial community structure differs across yak feeding regimes while the archaeal community structures are largely similar.

The protection of high porosity and high permeability reservoirs has not got enough attentions because the excellent hydrocarbon storage capacity would provide high oil and gas productivity even being damaged to some extent. Deepwater oil and gas reservoirs are mainly high porosity and permeability reservoirs, and the reservoir protection is important because severe reservoir damage will cause an enormous economic loss due to high investment and risk of deepwater operation. A new deepwater gas field in the western part of the South China Sea must be developed soon, and the reservoir protection strategy needs to be studied. The characteristics of the reservoir were studied by X-ray diffraction, scanning electron microscope, mercury intrusion porosimetry, hot rolling dispersion test and liner swelling test. Furthermore, the water and salinity sensitivities of the reservoir were studied. The results show that the reservoir has high porosity and high permeability with an average porosity larger than 30% and an average permeability higher than 500×10-3 μm. The large pore throats will be easily invaded and plugged by solids in drilling and completion fluid. Therefore, effective temporary-plugging of the pore throats with a wide size range is extremely important. The content of clay mineral is 9% to 19% with approximately 50% mixed-layer illite-smectite, presenting potential borehole instability and reservoir damage risk due to its strong hydration ability. In addition, the reservoir has strong salinity sensitivity and medium-weak water sensitivity. The reservoir protection strategy of drilling and completion fluid related to the properties of the reservoir was proposed. A multi-stage bridge temporary-plugging method was used to prevent the invasion of solids into the pore throats with a wide size range. A highly inhibitive polyamine was selected to inhibit the hydration of clays to maintain wellbore stability and reduce reservoir damage. Because of the requirement of gas hydrate inhibition and density adjustment, a high performance water-based drill-in fluid with formate brines was developed. It has excellent reservoir protection performance with the permeability recovery rate approximately 90%. Meanwhile, it performs well in inhibiting clay hydration, hydrate formation, and has good rheological as well as filtration properties over a wide temperature range from 4 °C to 75 °C. This work provides theoretical guidance of reservoir protection and a high performance drill-in fluid for the target deepwater gas filed. We hope it will also be useful for the protection of high porosity and high permeability reservoir in other deepwater oil and gas fields.

Neural machine translation has shown very promising results lately. Most NMT models follow the encoder-decoder framework. To make encoder-decoder models more flexible, attention mechanism was introduced to machine translation and also other tasks like speech recognition and image captioning. We observe that the quality of translation by attention-based encoder-decoder can be significantly damaged when the alignment is incorrect. We attribute these problems to the lack of distortion and fertility models. Aiming to resolve these problems, we propose new variations of attention-based encoder-decoder and compare them with other models on machine translation. Our proposed method achieved an improvement of 2 BLEU points over the original attention-based encoder-decoder., Comment: 11 pages, updated details

White organic light-emitting diodes based on three small molecular fluorescence materials were designed and prepared.The emitting layers consisted of a red dye,4-(dicyanomethylene)-2-tert-butyl-6-(1,1,7,7-tetra-methyljulolidin-4-yl-vinyl)-4H-pyran(DCJTB),doped in a green layer of 8-hydroxyquinoline aluminum salt(Alq3),which further formed a heterojunction with a blue layer of 2,2′-(1,3-phenylene)-bis[5-(4-tert-butylphenyl)-1,3,4-oxadiazole](OXD-7).N,N′-bis-(naphthalene-1-yl)-N,N′-bis(phenyl)-benzidine(NPB) was used as a hole transport layer.Voltage dependent emission spectra and their colorimetric characteristics were manipulated using different device structures.After adjusting the carrier injection/transporting properties,the exciton recombination region was confined and the color stability was improved against the driving voltages.An improved white light-emitting device has a wide spectrum in visible range and a good color stability,its color coordinates change slightly from CIE(x,y)=(0.364,0.314) to CIE(x,y)=(0.332,0.291) when the driven voltage changes from 9.1 V to 12.9 V.

Major factors that cause damage in drilling in unconsolidated sandstone heavy oil reservoirs include: invasion of solids in drilling fluid, incompatibility between the liquid phase of drilling fluid and crude oil, and hydration and expansion of reservoir clay minerals. Therefore, a solid-free weak gel drilling fluid system for horizontal wells to protect the formations was developed that contains seawater + 0.1%–0.2% NaOH + 0.2% Na2CO3+ 0.7% VIS + 2.0% FLO + 2.0% JLX, weighed with KCl or sodium formate. The drilling fluid system has unique rheological properties, temporally independent gel strength, and excellent lubricating and inhibition performance. It is compatible with formation fluids, it not only meets the needs of horizontal well drilling, but also effectively protects the reservoir. The technique is well performed in tens of horizontal wells in offshore oilfields, such as WC13-1, BZ34-1, NP35-2, and BZ25-1 oilfields. 摘要: 钻井过程中造成疏松砂岩稠油油藏发生损害的主要因素是：钻井液固相侵入、钻井液液相与原油不配伍及储集层黏土矿物水化膨胀。为此研制出适于疏松砂岩稠油油藏储集层保护的无固相弱凝胶水平井钻井液体系：海水+0.1%～0.2%NaOH+0.2%Na2CO3+0.7%VIS+2.0%FLO +2.0%JLX，用KCl或甲酸盐加重。该钻井液具有独特的流变性能，凝胶强度无时间依赖性，具有优异的润滑性能和抑制性，与储集层流体配伍性好，不仅能够满足水平井钻井作业需要，而且具有优异的储集层保护效果，经文昌13-1油田、渤中34-1油田、南堡35-2油田、渤中25-1油田等数十口海上油田水平井验证，效果良好。 Key words: unconsolidated sandstone, heavy oil reservoir, formation damage, horizontal well, solid-free weak gel drilling fluid

A loose coupling method is proposed to analyze electromagnetic forming with solenoid coil. Analysis of electromagnetic field is conducted by ANSYS/Emag, structural field is solved by explicit module ANSYS/Lsdyna, and data transfer from electromagnetic field o structural field is realized by APDL program Time-space distribution of magnetic pressure and discharge voltage, tube thickness and gap are obtained. Effect of discharge voltage, gap between coil and tube and tube thickness on electromagnetic forming are discussed respectively.

Language model is one of the most important modules in statistical machine translation and currently the word-based language model dominants this community. However, many translation models (e.g. phrase-based models) generate the target language sentences by rendering and compositing the phrases rather than the words. Thus, it is much more reasonable to model dependency between phrases, but few research work succeed in solving this problem. In this paper, we tackle this problem by designing a novel phrase-based language model which attempts to solve three key sub-problems: 1, how to define a phrase in language model; 2, how to determine the phrase boundary in the large-scale monolingual data in order to enlarge the training set; 3, how to alleviate the data sparsity problem due to the huge vocabulary size of phrases. By carefully handling these issues, the extensive experiments on Chinese-to-English translation show that our phrase-based language model can significantly improve the translation quality by up to +1.47 absolute BLEU score., Comment: 8 pages