Search

The relationship between language and thought remains an unresolved philosophical issue. Existing viewpoints can be broadly categorized into two schools: one asserting their independence, and another arguing that language constrains thought. In the context of large language models, this debate raises a crucial question: Does a language model's grasp of semantic meaning depend on thought processes? To explore this issue, we investigate whether reasoning techniques can facilitate semantic understanding. Specifically, we conceptualize thought as reasoning, employ chain-of-thought prompting as a reasoning technique, and examine its impact on sentiment analysis tasks. The experiments show that chain-of-thought has a minimal impact on sentiment analysis tasks. Both the standard and chain-of-thought prompts focus on aspect terms rather than sentiment in the generated content. Furthermore, counterfactual experiments reveal that the model's handling of sentiment tasks primarily depends on information from demonstrations. The experimental results support the first viewpoint.

Existing works have studied the impacts of the order of words within natural text. They usually analyze it by destroying the original order of words to create a scrambled sequence, and then comparing the models' performance between the original and scrambled sequences. The experimental results demonstrate marginal drops. Considering this findings, different hypothesis about word order is proposed, including ``the order of words is redundant with lexical semantics'', and ``models do not rely on word order''. In this paper, we revisit the aforementioned hypotheses by adding a order reconstruction perspective, and selecting datasets of different spectrum. Specifically, we first select four different datasets, and then design order reconstruction and continuing generation tasks. Empirical findings support that ChatGPT relies on word order to infer, but cannot support or negate the redundancy relations between word order lexical semantics.

Previous works have evaluated memorization by comparing model outputs with training corpora, examining how factors such as data duplication, model size, and prompt length influence memorization. However, analyzing these extensive training corpora is highly time-consuming. To address this challenge, this paper proposes an innovative approach named ROME that bypasses direct processing of the training data. Specifically, we select datasets categorized into three distinct types -- context-independent, conventional, and factual -- and redefine memorization as the ability to produce correct answers under these conditions. Our analysis then focuses on disparities between memorized and non-memorized samples by examining the logits and representations of generated texts. Experimental findings reveal that longer words are less likely to be memorized, higher confidence correlates with greater memorization, and representations of the same concepts are more similar across different contexts. Our code and data will be publicly available when the paper is accepted., Comment: Submitted to EMNLP, 2024

Word order is an important concept in natural language, and in this work, we study how word order affects the induction of world knowledge from raw text using language models. We use word analogies to probe for such knowledge. Specifically, in addition to the natural word order, we first respectively extract texts of six fixed word orders from five languages and then pretrain the language models on these texts. Finally, we analyze the experimental results of the fixed word orders on word analogies and show that i) certain fixed word orders consistently outperform or underperform others, though the specifics vary across languages, and ii) the Wov2Lex hypothesis is not hold in pre-trained language models, and the natural word order typically yields mediocre results. The source code will be made publicly available at https://github.com/lshowway/probing_by_analogy.

Few studies have worked on the effects of tonal coarticulation and prosodic positions on the low rising tone in Xiamen Dialect. This study addressed such an issue. To do so, a new method, the Tonal Contour Analysis in Tonal Triangle, was proposed to measure the subtle curvature of the tonal contour. Findings are as follows: (1) The low rising tone in Xiamen Dialect has a tendency towards the falling-rising tone, which is significantly affected by the tonal coarticulation and prosodic positions. (2) The low rising tone presents as a falling-rising tone when preceded by a tone with a high offset, and as a low rising tone when preceded by a tone that ends up low. (3) The curvature of the low rising tone is greatest in the sentence-initial position, and is positively correlated to its own duration., Comment: To be published in InterSpeech 2023

External knowledge,e.g., entities and entity descriptions, can help humans understand texts. Many works have been explored to include external knowledge in the pre-trained models. These methods, generally, design pre-training tasks and implicitly introduce knowledge by updating model weights, alternatively, use it straightforwardly together with the original text. Though effective, there are some limitations. On the one hand, it is implicit and only model weights are paid attention to, the pre-trained entity embeddings are ignored. On the other hand, entity descriptions may be lengthy, and inputting into the model together with the original text may distract the model's attention. This paper aims to explicitly include both entities and entity descriptions in the fine-tuning stage. First, the pre-trained entity embeddings are fused with the original text representation and updated by the backbone model layer by layer. Second, descriptions are represented by the knowledge module outside the backbone model, and each knowledge layer is selectively connected to one backbone layer for fusing. Third, two knowledge-related auxiliary tasks, i.e., entity/description enhancement and entity enhancement/pollution task, are designed to smooth the semantic gaps among evolved representations. We conducted experiments on four knowledge-oriented tasks and two common tasks, and the results achieved new state-of-the-art on several datasets. Besides, we conduct an ablation study to show that each module in our method is necessary. The code is available at https://github.com/lshowway/Ered.

Two-dimensional (2D) materials possess great potential for flexible devices, ascribing to their outstanding electrical, optical, and mechanical properties. However, their mechanical deformation property and fracture mechanism, which are inescapable in many applications like flexible optoelectronics, are still unclear or not thoroughly investigated due methodology limitations. In light of this, such mechanical properties and mechanisms are explored on example of gallium telluride (GaTe), a promising optoelectronic candidate with an ultrahigh photo-responsibility and a high plasticity within 2D family. Considering the driving force insufficient in atomic force microscopy (AFM)-based nanoindentation method, here the mechanical properties of both substrate-supported and suspended GaTe multilayers were systematically investigated through full-scale Berkovich-tip nanoindentation, micro-Raman spectroscopy, AFM, and scanning electron microscopy. An unusual concurrence of multiple pop-in and load-drop events in loading curve was observed. By further correlating to molecular dynamics calculations, this concurrence was unveiled originating from the interlayer sliding mediated layers-by-layers fracture mechanism within GaTe multilayers. The van der Waals force between GaTe multilayers and substrates was revealed much stronger than that between GaTe interlayers, resulting in the easy sliding and fracture of multilayers within GaTe. This work provides new insights into the deformation and fracture mechanisms of GaTe and other similar 2D multilayers in flexible applications., Comment: 21 pages, 5 figures

In this paper, we study sentiment analysis task where the outcomes are mainly contributed by a few key elements of the inputs. Motivated by the two-streams hypothesis, we propose a neural architecture, named TraceNet, to address this type of task. It not only learns discriminative representations for the target task via its encoders, but also traces key elements at the same time via its locators. In TraceNet, both encoders and locators are organized in a layer-wise manner, and a smoothness regularization is employed between adjacent encoder-locator combinations. Moreover, a sparsity constraints are enforced on locators for tracing purposes and items are proactively masked according to the item weights output by locators.A major advantage of TraceNet is that the outcomes are easier to understand, since the most responsible parts of inputs are identified. Also, under the guidance of locators, it is more robust to attacks due to its focus on key elements and the proactive masking training strategy. Experimental results show its effectiveness for sentiment classification. Moreover, we provide several case studies to demonstrate its robustness and interpretability.

Though some recent works focus on injecting sentiment knowledge into pre-trained language models, they usually design mask and reconstruction tasks in the post-training phase. In this paper, we aim to benefit from sentiment knowledge in a lighter way. To achieve this goal, we study sentence-level sentiment analysis and, correspondingly, propose two sentiment-aware auxiliary tasks named sentiment word cloze and conditional sentiment prediction. The first task learns to select the correct sentiment words within the input, given the overall sentiment polarity as prior knowledge. On the contrary, the second task predicts the overall sentiment polarity given the sentiment polarity of the word as prior knowledge. In addition, two kinds of label combination methods are investigated to unify multiple types of labels in each task. We argue that more information can promote the models to learn more profound semantic representation. We implement it in a straightforward way to verify this hypothesis. The experimental results demonstrate that our approach consistently outperforms pre-trained models and is additive to existing knowledge-enhanced post-trained models. The code and data are released at https://github.com/lshowway/KESA.

News recommendation models often fall short in capturing users' preferences due to their static approach to user-news interactions. To address this limitation, we present a novel dynamic news recommender model that seamlessly integrates continuous time information to a hierarchical attention network that effectively represents news information at the sentence, element, and sequence levels. Moreover, we introduce a dynamic negative sampling method to optimize users' implicit feedback. To validate our model's effectiveness, we conduct extensive experiments on three real-world datasets. The results demonstrate the effectiveness of our proposed approach.

Few-shot relation extraction (FSRE) is of great importance in long-tail distribution problem, especially in special domain with low-resource data. Most existing FSRE algorithms fail to accurately classify the relations merely based on the information of the sentences together with the recognized entity pairs, due to limited samples and lack of knowledge. To address this problem, in this paper, we proposed a novel entity CONCEPT-enhanced FEw-shot Relation Extraction scheme (ConceptFERE), which introduces the inherent concepts of entities to provide clues for relation prediction and boost the relations classification performance. Firstly, a concept-sentence attention module is developed to select the most appropriate concept from multiple concepts of each entity by calculating the semantic similarity between sentences and concepts. Secondly, a self-attention based fusion module is presented to bridge the gap of concept embedding and sentence embedding from different semantic spaces. Extensive experiments on the FSRE benchmark dataset FewRel have demonstrated the effectiveness and the superiority of the proposed ConceptFERE scheme as compared to the state-of-the-art baselines. Code is available at https://github.com/LittleGuoKe/ConceptFERE., Comment: Accepted at ACL2021

Paper holds the promise to replace silicon substrates in applications like internet of things or disposable electronics that require ultra-low-cost electronic components and an environmentally friendly electronic waste management. In the last years, spurred by the abovementioned properties of paper as a substrate and the exceptional electronic, mechanical and optical properties of van der Waals (vdW) materials, many research groups have worked towards the integration of vdW materials-based devices on paper. Recently, a method to deposit a continuous film of densely packed interconnects of vdW materials on paper by simply rubbing the vdW crystals against the rough surface of paper has been presented. This method utilizes the weak interlayer vdW interactions and allows cleaving of the crystals into micro platelets through the abrasion against the paper. Here, we aim to illustrate the general character and the potential of this technique by fabricating films of 39 different vdW materials (including superconductors, semi-metals, semiconductors, and insulators) on standard copier paper. We have thoroughly characterized their optical properties showing their high optical quality: one can easily resolve the absorption band edge of semiconducting vdW materials and even the excitonic features present in some vdW materials with high exciton binding energy. We also measured the electrical resistivity for several vdW materials films on paper finding exceptionally low values, which are in some cases, orders of magnitude lower than those reported for analogous films produced by inkjet printing. We finally demonstrate the fabrication of field-effect devices with vdW materials on paper using the paper substrate as an ionic gate., Comment: 4 figures in main text, 21 figures in Supp. Info

We have implemented three different optical methods to quantitatively assess the thickness of thin GaSe flakes transferred on both transparent substrates, like Gel-Film, or SiO2/Si substrates. We show how their apparent color can be an efficient way to make a quick rough estimation of the thickness of the flakes. This method is more effective for SiO2/Si substrates as the thickness dependent color change is more pronounced on these substrates than on transparent substrates. On the other hand, for transparent substrates, the transmittance of the flakes in the blue region of the visible spectrum can be used to estimate the thickness. We find that the transmittance of flakes in the blue part of the spectrum decreases at a rate of 1.2%/nm. On SiO2/Si, the thickness of the flakes can be accurately determined by fitting optical contrast spectra to a Fresnel law-based model. Finally, we also show how the quantitative analysis of transmission mode optical microscopy images can be a powerful method to quickly probe the environmental degradation of GaSe flakes exposed to aging conditions., Comment: 5 figures

Indium selenide (InSe), as a novel van der Waals layered semiconductor, has attracted a large research interest thanks to its excellent optical and electrical properties in the ultra-thin limit. Here, we discuss four different optical methods to quantitatively identify the thickness of thin InSe flakes on various substrates, such as SiO2/Si or transparent polymeric substrates. In the case of thin InSe deposited on a transparent substrate, the transmittance of the flake in the blue region of the visible spectrum can be used to estimate the thickness. For InSe supported by SiO2/Si, the thickness of the flakes can be estimated either by assessing their apparent colors or accurately analyzed using a Fresnel-law based fitting model of the optical contrast spectra. Finally, we also studied the thickness dependency of the InSe photoluminescence emission energy, which provides an additional tool to estimate the InSe thickness and it works both for InSe deposited on SiO2/Si and on a transparent polymeric substrate., Comment: 4 figures

The ultrathin nature and dangling bonds free surface of two-dimensional (2D) semiconductors allow for significant modifications of their band gap through strain engineering. Here, thin InSe photodetector devices are biaxially stretched, finding, a strong band gap tunability upon strain. The applied biaxial strain is controlled through the substrate expansion upon temperature increase and the effective strain transfer from the substrate to the thin InSe is confirmed by Raman spectroscopy. The band gap change upon biaxial strain is determined through photoluminescence measurements, finding a gauge factor of up to ~200 meV/%. We further characterize the effect of biaxial strain on the electrical properties of the InSe devices. In the dark state, a large increase of the current is observed upon applied strain which gives a piezoresistive gauge factor value of ~450-1000, ~5-12 times larger than that of other 2D materials and of state-of-the-art silicon strain gauges. Moreover, the biaxial strain tuning of the InSe band gap also translates in a strain-induced redshift of the spectral response of our InSe photodetectors with {\Delta}Ecut-off ~173 meV at a rate of ~360 meV/% of strain, indicating a strong strain tunability of the spectral bandwidth of the photodetectors., Comment: 5 main text figures, 1 table comparing the strain tunable bandgap for different 2Ds and 11 supporting information figures

Two-dimensional semiconductors are excellent candidates for next-generation electronics and optoelec-tronics thanks to their electrical properties and strong light-matter interaction. To fabricate devices with optimal electrical properties, it is crucial to have both high-quality semiconducting crystals and ideal con-tacts at metal-semiconductor interfaces. Thanks to the mechanical exfoliation of van der Waals crystals, atomically-thin high-quality single-crystals can easily be obtained in a laboratory. However, conventional metal deposition techniques can introduce chemical disorder and metal-induced mid-gap states that induce Fermi level pinning and can degrade the metal-semiconductor interfaces, resulting in poorly performing devices. In this article, we explore the electrical contact characteristics of Au-InSe and graphite-InSe van der Waals contacts, obtained by stacking mechanically exfoliated InSe flakes onto pre-patterned Au or graphite electrodes without the need of lithography or metal deposition. The high quality of the metal-semiconductor interfaces obtained by van der Waals contact allows to fabricate high-quality Schottky di-odes based on the Au-InSe Schottky barrier. Our experimental observation indicates that the contact barrier at the graphite-InSe interface is negligible due to the similar electron affinity of InSe and graphite, while the Au-InSe interfaces are dominated by a large Schottky barrier., Comment: 25 pages, 13 figures, Main text and Supporting Information

Due to the excellent electrical transport properties and optoelectronic performance, thin indium selenide (InSe) has recently attracted attention in the field of 2D semiconducting materials. However, the mechanism behind the photocurrent generation in thin InSe photodetectors remains elusive. Here, we present a set of experiments aimed at explaining the strong scattering in the photoresponsivity values reported in the literature for thin InSe photodetectors. By performing optoelectronic measurements on thin InSe-based photodetectors operated under different environmental conditions we find that the photoresponsivity, the response time and the photocurrent power dependency are strongly correlated in this material. This observation indicates that the photogating effect plays an imporant role for thin InSe flakes, and it is the dominant mechanism in the ultra-high photoresponsivity of pristine InSe devices. In addition, when exposing the pristine InSe photodetectors to the ambient environment we observe a fast and irreversible change in the photoresponse, with a decrease in the photoresponsivity accompanied by an increase of the operating speed. We attribute this photodetector performance change (upon atmospheric exposure) to the decrease in the density of the traps present in InSe, due to the passivation of selenium vacancies by atmospheric oxygen species. This passivation is accompanied by a downward shift of the InSe Fermi level and by a decrease of the Fermi level pinning, which leads to an increase of the Schottky barrier between Au and InSe. Our study reveals the important role of traps induced by defects in tailoring the properties of devices based on 2D materials and offers a controllable route to design and functionalize thin InSe photodetectors to realize devices with either ultrahigh photoresposivity or fast operation speed., Comment: 27 pages, 12 figures, Main text and Supporting Information