Search

Pre-trained Language Models (PLMs) are integral to many modern natural language processing (NLP) systems. Although multilingual models cover a wide range of languages, they often grapple with challenges like high inference costs and a lack of diverse non-English training data. Arabic-specific PLMs are trained predominantly on modern standard Arabic, which compromises their performance on regional dialects. To tackle this, we construct an Arabic dialectal corpus comprising 3.4M sentences gathered from social media platforms. We utilize this corpus to expand the vocabulary and retrain a BERT-based model from scratch. Named AlcLaM, our model was trained using only 13 GB of text, which represents a fraction of the data used by existing models such as CAMeL, MARBERT, and ArBERT, compared to 7.8%, 10.2%, and 21.3%, respectively. Remarkably, AlcLaM demonstrates superior performance on a variety of Arabic NLP tasks despite the limited training data. AlcLaM is available at GitHub https://github.com/amurtadha/Alclam and HuggingFace https://huggingface.co/rahbi., Comment: Accepted by ArabicNLP 2024, presented in ACL 2024

Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations of transformer architecture, which pose challenges when attempting to effectively integrate supervision from a substantial number of demonstration examples. In this paper, we introduce a novel framework, called Naive Bayes-based Context Extension (NBCE), to enable existing LLMs to perform ICL with an increased number of demonstrations by significantly expanding their context size. Importantly, this expansion does not require fine-tuning or dependence on particular model architectures, all the while preserving linear efficiency. NBCE initially splits the context into equal-sized windows fitting the target LLM's maximum length. Then, it introduces a voting mechanism to select the most relevant window, regarded as the posterior context. Finally, it employs Bayes' theorem to generate the test task. Our experimental results demonstrate that NBCE substantially enhances performance, particularly as the number of demonstration examples increases, consistently outperforming alternative methods. The NBCE code will be made publicly accessible. The code NBCE is available at: https://github.com/amurtadha/NBCE-master, Comment: Accepted to main NAACL 2024

Self-injection locking scheme has the potential to narrow the linewidth of lasers in a compact setup. Here, we report a narrow linewidth laser source near 1 {\mu}m by self-injection locking scheme using a Fabry-Perot (FP) hollow resonator with a high-quality factor (Q>10^8). The measured fundamental linewidth of the laser is 41 Hz, and a coarse tuning range over 5.5 nm is achieved by changing the driving current of the laser source. Meanwhile, a fine-tuning range of 373 MHz is achieved without mode hops by changing the voltage applied to the PZT on the resonator. More importantly, benefiting from the low thermal refractive noise and low thermal expansion of the FP hollow resonator, the beat-note linewidth and the frequency Allan deviation are measured to be 510.3 Hz in and 10^-11 (1s averaging time), respectively, by using a fully stabilized frequency comb as reference. Such a high-performance laser is fully integrated with a palm-sized package (52.3 mL) for field-deployable applications.

The mechanism of detonation instability and deflagration-to-detonation transition is studied by one-dimensional numerical simulation with overall one-step chemical reaction kinetics in this paper. The detonation is ignited at the left closed end of the one-dimensional detonation tube and propagates downstream. The activation energy is increased to trigger the instability of detonation. The numerical results show that the C-J detonation is stable at lower activation energy. The stable detonation does not have the von Neumann spike and the gas Mach number at detonation front is subsonic. The von Neumann spike appears and the gas Mach number becomes supersonic as the activation energy is increased. The detonation instability appears with the von Neumann spike synchronously. At very higher activation energy, the detonation quenches abruptly and degenerates into a C-J deflagration. The detonation is extinguished abruptly by the rarefaction wave induced by the higher von Neumann spike. Then the rarefaction wave moves in front of the heat release region and weakens the leading shock wave gradually. The C-J deflagration is composed of a precursor shock wave and a flame front, and the flame front is completely decoupled from the shock wave. The gas static temperature behind the leading shock wave is too low to ignite the mixture. The rarefaction wave from the wall ceases the mixture behind the leading shock, increases its static temperature and decrease its pressure. As a result, the combustion takes place at the interface. The pressure rise caused by the combustion at the interface offsets the influence of rarefaction wave, and this mechanism makes the C-J deflagration propagate downstream with a relatively constant velocity for a long time.

Semi-supervised text classification-based paradigms (SSTC) typically employ the spirit of self-training. The key idea is to train a deep classifier on limited labeled texts and then iteratively predict the unlabeled texts as their pseudo-labels for further training. However, the performance is largely affected by the accuracy of pseudo-labels, which may not be significant in real-world scenarios. This paper presents a Rank-aware Negative Training (RNT) framework to address SSTC in learning with noisy label manner. To alleviate the noisy information, we adapt a reasoning with uncertainty-based approach to rank the unlabeled texts based on the evidential support received from the labeled texts. Moreover, we propose the use of negative training to train RNT based on the concept that ``the input instance does not belong to the complementary label''. A complementary label is randomly selected from all labels except the label on-target. Intuitively, the probability of a true label serving as a complementary label is low and thus provides less noisy information during the training, resulting in better performance on the test data. Finally, we evaluate the proposed solution on various text classification benchmark datasets. Our extensive experiments show that it consistently overcomes the state-of-the-art alternatives in most scenarios and achieves competitive performance in the others. The code of RNT is publicly available at:https://github.com/amurtadha/RNT., Comment: TACL 2023

Narrow linewidth laser(NLL) of high frequency stability and small form factor is essential to enable applications in long range sensing, quantum information and atomic clocks. Various high performance NLL have been demonstrated by Pound-Drever Hall(PDH) lock or self injection lock(SIL) of a seed laser to a vaccum-stabilized FP cavity of ultrahigh quality factor(Q). However they are often complicated lab setups due to the sophisticated stabilizing system and locking electronics. Here we report a compact NLL of 68mL volume, realized by SIL of a diode laser to a miniature FP cavity of 7.7x108 Q and 0.5mL volume, bypassing table-size vaccum, thermal and vibration isolation. We characterized the NLL with a self-delayed heterodyne system, the Lorentzian linewidth reaches 60mHz, and the integrated linewidth is ~80Hz. The frequency noise performance exceeds that of commercial NLLs and the best reported hybrid-integrated NLL realized by SIL to high Q on-chip ring resonators. Our work marks a major step toward a field-deployable NLL of superior performance utilizing ultra-high Q FP cavity.

The oblique detonation engine (ODE) has established a clear superiority for hypersonic flight because of its high thermal efficiency and compact structure. It has become the research hot spot all over the world in recent years. The aim of this study is to derive a criterion on the propulsive balance of ODE, from which we can find the key parameters governing the propulsive performance explicitly. A physical model of ODE is put forth, which consists of the inlet, the constant cross-section combustor and the divergent nozzle. The mathematical equations to calculate the thrust generated by the divergent nozzle and the pressure drag produced by the inlet are deduced. The net thrust of ODE is then obtained. The criterion shows clearly that the static temperature at the engine inlet exit is a very important parameter. The lower the inlet exit temperature is, the higher the specific impulse will be. The specific impulse of ODE with stoichiometric H2/air mixture and hydrocarbon/air mixture are calculated by using these equations. The results show that ODE can obtain positive net thrust from Ma8 to Ma15., Comment: 23 pages, 9 figures

Microresonator-based optical frequency comb (microcomb) has the potential to revolutionize the accuracy of frequency synthesizer in radar and communication applications. However, fundamental limit exists for low noise microcomb generation, especially in low size, weight, power and cost (SWaP-C) package. Here we resolve this limit, by the demonstration of an automated turnkey microcomb, operating close to its low quantum-limited phase noise, within a compact setup size of 85 mm * 90 mm * 25 mm. High quality factor fiber Fabry-Perot resonator (FFPR), with Q up to 4.0 * 10^9, is the key for both low quantum noise and pump noise limit, in the diode-pump case in a self-injection locking scheme. Low phase noise of -80 and -105 dBc/Hz at 100 Hz, -106 and -125 dBc/Hz at 1 kHz, -133 and -148 dBc/Hz at 10 kHz is achieved at 10.1 GHz and 1.7 GHz repetition frequencies, respectively. With the simultaneous automated turnkey, low-noise and direct-diode-pump capability, our microcomb is ready to be used as a low-noise frequency synthesizer with low SWaP-C and thus field deployability.

Named entity recognition (NER) task aims at identifying entities from a piece of text that belong to predefined semantic types such as person, location, organization, etc. The state-of-the-art solutions for flat entities NER commonly suffer from capturing the fine-grained semantic information in underlying texts. The existing span-based approaches overcome this limitation, but the computation time is still a concern. In this work, we propose a novel span-based NER framework, namely Global Pointer (GP), that leverages the relative positions through a multiplicative attention mechanism. The ultimate goal is to enable a global view that considers the beginning and the end positions to predict the entity. To this end, we design two modules to identify the head and the tail of a given entity to enable the inconsistency between the training and inference processes. Moreover, we introduce a novel classification loss function to address the imbalance label problem. In terms of parameters, we introduce a simple but effective approximate method to reduce the training parameters. We extensively evaluate GP on various benchmark datasets. Our extensive experiments demonstrate that GP can outperform the existing solution. Moreover, the experimental results show the efficacy of the introduced loss function compared to softmax and entropy alternatives.

In the era of deep learning, loss functions determine the range of tasks available to models and algorithms. To support the application of deep learning in multi-label classification (MLC) tasks, we propose the ZLPR (zero-bounded log-sum-exp \& pairwise rank-based) loss in this paper. Compared to other rank-based losses for MLC, ZLPR can handel problems that the number of target labels is uncertain, which, in this point of view, makes it equally capable with the other two strategies often used in MLC, namely the binary relevance (BR) and the label powerset (LP). Additionally, ZLPR takes the corelation between labels into consideration, which makes it more comprehensive than the BR methods. In terms of computational complexity, ZLPR can compete with the BR methods because its prediction is also label-independent, which makes it take less time and memory than the LP methods. Our experiments demonstrate the effectiveness of ZLPR on multiple benchmark datasets and multiple evaluation metrics. Moreover, we propose the soft version and the corresponding KL-divergency calculation method of ZLPR, which makes it possible to apply some regularization tricks such as label smoothing to enhance the generalization of models.

Aspect-based sentiment analysis (ABSA) aims to associate a text with a set of aspects and infer their respective sentimental polarities. State-of-the-art approaches are built on fine-tuning pre-trained language models, focusing on learning aspect-specific representations from the corpus. However, aspects are often expressed implicitly, making implicit mapping challenging without sufficient labeled examples, which may be scarce in real-world scenarios. This paper proposes a unified framework to address aspect categorization and aspect-based sentiment subtasks. We introduce a mechanism to construct an auxiliary-sentence for the implicit aspect using the corpus's semantic information. We then encourage BERT to learn aspect-specific representation in response to this auxiliary-sentence, not the aspect itself. We evaluate our approach on real benchmark datasets for both ABSA and Targeted-ABSA tasks. Our experiments show that it consistently achieves state-of-the-art performance in aspect categorization and aspect-based sentiment across all datasets, with considerable improvement margins. The BERT-ASC code is available at https://github.com/amurtadha/BERT-ASC., Comment: under review