Search

Qian Chen,1,2 Chengyin Yu,1,2 Qingqing Wu,1,2 Rong Song,1,2 Ye Liu,1,2 Sheng Feng,3 Chen Yu,1,2 Jingying Jia1,2 1Center Laboratory, Shanghai Xuhui Central Hospital, Shanghai, People’s Republic of China; 2Shanghai Engineering Research Center of Phase I Clinical Research & Quality Consistency Evaluation for Drugs, Shanghai, People’s Republic of China; 3Jiangsu Hengrui Pharmaceuticals Co., Ltd, Lianyungang, Jiangsu, People’s Republic of ChinaCorrespondence: Jingying Jia, Center Laboratory, Shanghai Xuhui Central Hospital, No. 966, Huaihai Road (M), Shanghai, People’s Republic of China, Tel/Fax +86-21-54030254, Email jyjia@shxh-centerlab.comPurpose: Henagliflozin is an original, selective sodium-glucose cotransporter 2 (SGLT2) inhibitor. Hydrochlorothiazide (HCTZ) is a common anti-hypertensive drug. This study aimed to evaluate the potential interaction between henagliflozin and HCTZ.Methods: This was a single-arm, open-label, multi-dose, three-period study that was conducted in healthy Chinese volunteers. Twelve subjects were treated in three periods, period 1: 25 mg HCTZ for four days, period 2: 10 mg henagliflozin for four days and period 3: 25 mg HCTZ + 10 mg henagliflozin for four days. Blood samples and urine samples were collected before and up to 24 hours after drug administrations on day 4, day 10 and day 14. The plasma concentrations of henagliflozin and HCTZ were analyzed using LC-MS/MS. The urine samples were collected for pharmacodynamic glucose and electrolyte analyses. Tolerability was also evaluated.Results: The 90% CI of the ratio of geometric means (combination: monotherapy) for AUCτ,ss of henagliflozin and HCTZ was within the bioequivalence interval of 0.80– 1.25. For henagliflozin, co-administration increased Css, max by 24.32% and the 90% CI of the GMR was (108.34%, 142.65%), and the 24-hour urine volume and glucose excretion decreased by 0.43% and 19.6%, respectively. For HCTZ, co-administration decreased Css, max by 19.41% and the 9

Climate warming is a pressing global issue with substantial impacts on soil health and function. However, the influence of environmental context on the responses of soil microorganisms to warming remains largely elusive, particularly in alpine ecosystems. This study examined the responses of the soil microbiome to in situ experimental warming across three elevations (3850 m, 4100 m, and 4250 m) in the meadow of Gongga Mountain, eastern Tibetan Plateau. Our findings demonstrate that soil microbial diversity is highly resilient to warming, with significant impacts observed only at specific elevations. Furthermore, the influence of warming on the composition of the soil microbial community is also elevation-dependent, underscoring the importance of local environmental context in shaping microbial evolution in alpine soils under climate warming. Notably, we identified soil moisture at 3850 m and carbon-to-nitrogen ratio at 4250 m as indirect predictors regulating the responses of microbial diversity to warming at specific elevations. These findings underscore the paramount importance of considering pre-existing environmental conditions in predicting the response of alpine soil microbiomes to climate warming. Our study provides novel insights into the intricate interactions between climate warming, soil microbiome, and environmental context in alpine ecosystems, illuminating the complex mechanisms governing soil microbial ecology in these fragile and sensitive environments.

By accurately measuring composition and energy spectrum of cosmic ray, the origin problem of so called "keen" region (energy > 1 PeV) can be solved. However, up to the present, the results of the spectrum in the knee region obtained by several previous experiments have shown obvious differences, so they cannot give effective evidence for judging the theoretical models on the origin of the knee. Recently, the Large High Altitude Air Shower Observatory (LHAASO) has reported several major breakthroughs and important results in astro-particle physics field. Relying on its advantages of wide-sky survey, high altitude location and large area detector arrays, the research content of LHAASO experiment mainly includes ultra high-energy gamma-ray astronomy, measurement of cosmic ray spectra in the knee region, searching for dark matter and new phenomena of particle physics at higher energy. The electron and Thermal Neutron detector (EN-Detector) is a new scintillator detector which applies thermal neutron detection technology to measure cosmic ray extensive air shower (EAS). This technology is an extension of LHAASO. The EN-Detector Array (ENDA) can highly efficiently measure thermal neutrons generated by secondary hadrons so called "skeleton" of EAS. In this paper, we perform the optimization of ENDA configuration, and obtain expectations on the ENDA results, including thermal neutron distribution, trigger efficiency and capability of cosmic ray composition separation. The obtained real data results are consistent with those by the Monte Carlo simulation.

Smart contracts are computer programs running on blockchains to implement Decentralized Applications.The absence of contract specifications hinders routine tasks, such as contract understanding and testing. Inthis work, we propose a specification mining approach to infer contract specifications from past transactionhistories. Our approach derives high-level behavioral automata of function invocations, accompanied byprogram invariants statistically inferred from the transaction histories. We implemented our approach as toolSmConand evaluated it on eleven well-studied Azure benchmark smart contracts and six popular real-worldDApp smart contracts. The experiments show thatSmConmines reasonably accurate specifications that canbe used to facilitate DApp understanding and development in terms of document maintenance and test suite improvement.

Smart contracts are susceptible to various security issues, among which access control (AC) vulnerabilities are particularly critical. While existing research has proposed multiple detection tools, the automatic and appropriate repair of AC vulnerabilities in smart contracts remains a challenge. Unlike commonly supported vulnerability types by existing repair tools, such as reentrancy, which are usually fixed by template-based approaches, the main obstacle of AC lies in identifying the appropriate roles or permissions amid a long list of non-AC-related source code to generate proper patch code, a task that demands human-level intelligence. Leveraging recent advancements in large language models (LLMs), we employ the state-of-the-art GPT-4 model and enhance it with a novel approach called ACFIX. The key insight is that we can mine common AC practices for major categories of code functionality and use them to guide LLMs in fixing code with similar functionality. To this end, ACFIX involves both offline and online phases. First, during the offline phase, ACFIX mines a taxonomy of common Role-based Access Control (RBAC) practices from 344,251 on-chain contracts, categorizing 49 role-permission pairs from the top 1,000 pairs mined. Second, during the online phase, ACFIX tracks AC-related elements across the contract and uses this context information along with a Chain-of-Thought pipeline to guide LLMs in identifying the most appropriate role-permission pair for the subject contract and subsequently generating a suitable patch. This patch will then undergo a validity and effectiveness check. To evaluate ACFIX, we built the first benchmark dataset of 118 real-world AC vulnerabilities, and our evaluation revealed that ACFIX successfully repaired 94.92% of them. This represents a significant improvement compared to the baseline GPT-4, which achieved only 52.54%., Comment: This is a technical report from Nanyang Technological University

Graph Neural Networks (GNNs) have achieved impressive results in graph classification tasks, but they struggle to generalize effectively when faced with out-of-distribution (OOD) data. Several approaches have been proposed to address this problem. Among them, one solution is to diversify training distributions in vanilla classification by modifying the data environment, yet accessing the environment information is complex. Besides, another promising approach involves rationalization, extracting invariant rationales for predictions. However, extracting rationales is difficult due to limited learning signals, resulting in less accurate rationales and diminished predictions. To address these challenges, in this paper, we propose a Cooperative Classification and Rationalization (C2R) method, consisting of the classification and the rationalization module. Specifically, we first assume that multiple environments are available in the classification module. Then, we introduce diverse training distributions using an environment-conditional generative network, enabling robust graph representations. Meanwhile, the rationalization module employs a separator to identify relevant rationale subgraphs while the remaining non-rationale subgraphs are de-correlated with labels. Next, we align graph representations from the classification module with rationale subgraph representations using the knowledge distillation methods, enhancing the learning signal for rationales. Finally, we infer multiple environments by gathering non-rationale representations and incorporate them into the classification module for cooperative learning. Extensive experimental results on both benchmarks and synthetic datasets demonstrate the effectiveness of C2R. Code is available at https://github.com/yuelinan/Codes-of-C2R., Comment: Accepted to WWW 2024

The ubiquity of variable-length integers in data storage and communication necessitates efficient decoding techniques. In this paper, we present SFVInt, a simple and fast approach to decode the prevalent Little Endian Base-128 (LEB128) varints. Our approach effectively utilizes the Bit Manipulation Instruction Set 2 (BMI2) in modern Intel and AMD processors, achieving significant performance improvement while maintaining simplicity and avoiding overengineering. SFVInt, with its generic design, effectively processes both 32-bit and 64-bit unsigned integers using a unified code template, marking a significant leap forward in varint decoding efficiency. We thoroughly evaluate SFVInt's performance across various datasets and scenarios, demonstrating that it achieves up to a 2x increase in decoding speed when compared to varint decoding methods used in established frameworks like Facebook Folly and Google Protobuf., Comment: DaMoN 2024

Continual Test-Time Adaptation (CTTA) is an emerging and challenging task where a model trained in a source domain must adapt to continuously changing conditions during testing, without access to the original source data. CTTA is prone to error accumulation due to uncontrollable domain shifts, leading to blurred decision boundaries between categories. Existing CTTA methods primarily focus on suppressing domain shifts, which proves inadequate during the unsupervised test phase. In contrast, we introduce a novel approach that guides rather than suppresses these shifts. Specifically, we propose $\textbf{C}$ontrollable $\textbf{Co}$ntinual $\textbf{T}$est-$\textbf{T}$ime $\textbf{A}$daptation (C-CoTTA), which explicitly prevents any single category from encroaching on others, thereby mitigating the mutual influence between categories caused by uncontrollable shifts. Moreover, our method reduces the sensitivity of model to domain transformations, thereby minimizing the magnitude of category shifts. Extensive quantitative experiments demonstrate the effectiveness of our method, while qualitative analyses, such as t-SNE plots, confirm the theoretical validity of our approach.

Current graph clustering methods emphasize individual node and edge con nections, while ignoring higher-order organization at the level of motif. Re cently, higher-order graph clustering approaches have been designed by motif based hypergraphs. However, these approaches often suffer from hypergraph fragmentation issue seriously, which degrades the clustering performance greatly. Moreover, real-world graphs usually contain diverse motifs, with nodes participating in multiple motifs. A key challenge is how to achieve precise clustering results by integrating information from multiple motifs at the node level. In this paper, we propose a multi-order graph clustering model (MOGC) to integrate multiple higher-order structures and edge connections at node level. MOGC employs an adaptive weight learning mechanism to au tomatically adjust the contributions of different motifs for each node. This not only tackles hypergraph fragmentation issue but enhances clustering accuracy. MOGC is efficiently solved by an alternating minimization algo rithm. Experiments on seven real-world datasets illustrate the effectiveness of MOGC.

The past two decades have witnessed columnar storage revolutionizing data warehousing and analytics. However, the rapid growth of machine learning poses new challenges to this domain. This paper presents Bullion, a columnar storage system tailored for machine learning workloads. Bullion addresses the complexities of data compliance, optimizes the encoding of long sequence sparse features, efficiently manages wide-table projections, and introduces feature quantization in storage. By aligning with the evolving requirements of ML applications, Bullion extends columnar storage to various scenarios, from advertising and recommendation systems to the expanding realm of Generative AI. Preliminary experimental results and theoretical analysis demonstrate Bullion's superior performance in handling the unique demands of machine learning workloads compared to existing columnar storage solutions. Bullion significantly reduces I/O costs for deletion compliance, achieves substantial storage savings with its optimized encoding scheme for sparse features, and drastically improves metadata parsing speed for wide-table projections. These advancements position Bullion as a critical component in the future of machine learning infrastructure, enabling organizations to efficiently manage and process the massive volumes of data required for training and inference in modern AI applications.

Video temporal grounding (VTG) is a fine-grained video understanding problem that aims to ground relevant clips in untrimmed videos given natural language queries. Most existing VTG models are built upon frame-wise final-layer CLIP features, aided by additional temporal backbones (e.g., SlowFast) with sophisticated temporal reasoning mechanisms. In this work, we claim that CLIP itself already shows great potential for fine-grained spatial-temporal modeling, as each layer offers distinct yet useful information under different granularity levels. Motivated by this, we propose Reversed Recurrent Tuning ($R^2$-Tuning), a parameter- and memory-efficient transfer learning framework for video temporal grounding. Our method learns a lightweight $R^2$ Block containing only 1.5% of the total parameters to perform progressive spatial-temporal modeling. Starting from the last layer of CLIP, $R^2$ Block recurrently aggregates spatial features from earlier layers, then refines temporal correlation conditioning on the given query, resulting in a coarse-to-fine scheme. $R^2$-Tuning achieves state-of-the-art performance across three VTG tasks (i.e., moment retrieval, highlight detection, and video summarization) on six public benchmarks (i.e., QVHighlights, Charades-STA, Ego4D-NLQ, TACoS, YouTube Highlights, and TVSum) even without the additional backbone, demonstrating the significance and effectiveness of the proposed scheme. Our code is available at https://github.com/yeliudev/R2-Tuning.

With recent advances in large language models (LLMs), this paper explores the potential of leveraging state-of-the-art LLMs, such as GPT-4, to transfer existing human-written properties (e.g., those from Certora auditing reports) and automatically generate customized properties for unknown code. To this end, we embed existing properties into a vector database and retrieve a reference property for LLM-based in-context learning to generate a new prop- erty for a given code. While this basic process is relatively straight- forward, ensuring that the generated properties are (i) compilable, (ii) appropriate, and (iii) runtime-verifiable presents challenges. To address (i), we use the compilation and static analysis feedback as an external oracle to guide LLMs in iteratively revising the generated properties. For (ii), we consider multiple dimensions of similarity to rank the properties and employ a weighted algorithm to identify the top-K properties as the final result. For (iii), we design a dedicated prover to formally verify the correctness of the generated prop- erties. We have implemented these strategies into a novel system called PropertyGPT, with 623 human-written properties collected from 23 Certora projects. Our experiments show that PropertyGPT can generate comprehensive and high-quality properties, achieving an 80% recall compared to the ground truth. It successfully detected 26 CVEs/attack incidents out of 37 tested and also uncovered 12 zero-day vulnerabilities, resulting in $8,256 bug bounty rewards.

Domain decomposition provides an effective way to tackle the dilemma of physics-informed neural networks (PINN) which struggle to accurately and efficiently solve partial differential equations (PDEs) in the whole domain, but the lack of efficient tools for dealing with the interfaces between two adjacent sub-domains heavily hinders the training effects, even leads to the discontinuity of the learned solutions. In this paper, we propose a symmetry group based domain decomposition strategy to enhance the PINN for solving the forward and inverse problems of the PDEs possessing a Lie symmetry group. Specifically, for the forward problem, we first deploy the symmetry group to generate the dividing-lines having known solution information which can be adjusted flexibly and are used to divide the whole training domain into a finite number of non-overlapping sub-domains, then utilize the PINN and the symmetry-enhanced PINN methods to learn the solutions in each sub-domain and finally stitch them to the overall solution of PDEs. For the inverse problem, we first utilize the symmetry group acting on the data of the initial and boundary conditions to generate labeled data in the interior domain of PDEs and then find the undetermined parameters as well as the solution by only training the neural networks in a sub-domain. Consequently, the proposed method can predict high-accuracy solutions of PDEs which are failed by the vanilla PINN in the whole domain and the extended physics-informed neural network in the same sub-domains. Numerical results of the Korteweg-de Vries equation with a translation symmetry and the nonlinear viscous fluid equation with a scaling symmetry show that the accuracies of the learned solutions are improved largely.

Smart contract transactions associated with security attacks often exhibit distinct behavioral patterns compared with historical benign transactions before the attacking events. While many runtime monitoring and guarding mechanisms have been proposed to validate invariants and stop anomalous transactions on the fly, the empirical effectiveness of the invariants used remains largely unexplored. In this paper, we studied 23 prevalent invariants of 8 categories, which are either deployed in high-profile protocols or endorsed by leading auditing firms and security experts. Using these well-established invariants as templates, we developed a tool Trace2Inv which dynamically generates new invariants customized for a given contract based on its historical transaction data. We evaluated Trace2Inv on 42 smart contracts that fell victim to 27 distinct exploits on the Ethereum blockchain. Our findings reveal that the most effective invariant guard alone can successfully block 18 of the 27 identified exploits with minimal gas overhead. Our analysis also shows that most of the invariants remain effective even when the experienced attackers attempt to bypass them. Additionally, we studied the possibility of combining multiple invariant guards, resulting in blocking up to 23 of the 27 benchmark exploits and achieving false positive rates as low as 0.32%. Trace2Inv outperforms current state-of-the-art works on smart contract invariant mining and transaction attack detection in terms of both practicality and accuracy. Though Trace2Inv is not primarily designed for transaction attack detection, it surprisingly found two previously unreported exploit transactions, earlier than any reported exploit transactions against the same victim contracts.

Naively assuming English as a source language may hinder cross-lingual transfer for many languages by failing to consider the importance of language contact. Some languages are more well-connected than others, and target languages can benefit from transferring from closely related languages; for many languages, the set of closely related languages does not include English. In this work, we study the impact of source language for cross-lingual transfer, demonstrating the importance of selecting source languages that have high contact with the target language. We also construct a novel benchmark dataset for close contact Chinese-Japanese-Korean-Vietnamese (CJKV) languages to further encourage in-depth studies of language contact. To comprehensively capture contact between these languages, we propose to integrate Romanized transcription beyond textual scripts via Contrastive Learning objectives, leading to enhanced cross-lingual representations and effective zero-shot cross-lingual transfer., Comment: Accepted at LREC-COLING 2024

Although convolutional neural networks (CNNs) have been proposed to remove adverse weather conditions in single images using a single set of pre-trained weights, they fail to restore weather videos due to the absence of temporal information. Furthermore, existing methods for removing adverse weather conditions (e.g., rain, fog, and snow) from videos can only handle one type of adverse weather. In this work, we propose the first framework for restoring videos from all adverse weather conditions by developing a video adverse-weather-component suppression network (ViWS-Net). To achieve this, we first devise a weather-agnostic video transformer encoder with multiple transformer stages. Moreover, we design a long short-term temporal modeling mechanism for weather messenger to early fuse input adjacent video frames and learn weather-specific information. We further introduce a weather discriminator with gradient reversion, to maintain the weather-invariant common information and suppress the weather-specific information in pixel features, by adversarially predicting weather types. Finally, we develop a messenger-driven video transformer decoder to retrieve the residual weather-specific feature, which is spatiotemporally aggregated with hierarchical pixel features and refined to predict the clean target frame of input videos. Experimental results, on benchmark datasets and real-world weather videos, demonstrate that our ViWS-Net outperforms current state-of-the-art methods in terms of restoring videos degraded by any weather condition. © 2023 IEEE.

Corn is one of most important agricultural products in China. Understanding impacts of regional climate change, as well as agricultural management practices, on corn yields is critical for maintaining stable corn production. Using the DayCent model and observed climatic data in Sichuan province (a humid and hot environment) and Hebei province (a cold and dry environment) in China, corn yields in 1948-2010 were simulated. The spatial variations of simulated corn yields and the relationship between regional climate variability and warming with corn yields in these two environments were analyzed. The results demonstrated that: (1) corn yields in Zhangjiakou of Hebei and most regions of Sichuan decreased significantly after 2000 compared to other regions; (2) relative humidity and precipitation exhibit a significant negative correlation with observed crop yields in the growing season in Hebei province; (3) air temperature from 23.33 °C to 29 °C constitutes the ideal range influencing the increase of corn yields in Sichuan; (4) the planting of the large amount of silage maize in Sichuan compensated the negative impact of the rising air temperature on corn yields; (5) sensitivity tests for different fertilization levels and OMAD suggest that an increasing fertilization level significantly affects corn yields in Hebei province, a cold and dry environment, while a decreasing fertilization level has a significant negative effect in Sichuan province, a hot and humid environment. The overarching goal of these analyses is to provide the theoretical basic for maintaining stable corn production under regional climate warming and different agricultural management practices.

Ductility deterioration induced by L21-Ni2AlTi precipitates in the aged ferritic alloys was examined systematically by using a combination of scanning transmission electron microscope (STEM), mechanical tests and first-principles thermodynamic calculations. The experimental studies revealed that the strength and hardness of the aged Fe–10Cr–5Ni–1Al–1Ti ferritic alloy containing B2–NiAl and L21-Ni2AlTi precipitates were higher than that of the aged Fe–10Cr–5Ni–1Al ferritic alloy containing NiAl precipitates, whereas the elongation-to-failure decreased dramatically from 9.3% to 0.3% indicating an obvious ductility deterioration due to the formation of L21-Ni2AlTi precipitates. This was also confirmed by the observation of fracture transition mode from dimpled failure to cleavage failure. The first-principles calculations, concerning the precipitate/matrix interface, were carried out to provide a theoretical analysis for the ductile–brittle transition by means of empirical ductility criteria ratios G/B and (C12–C44)/B as well as cleavage energy. The cleavage energy results indicated an intrinsic brittleness of the L21-Ni2AlTi phase and the L21-Ni2AlTi/BCC-Fe interface. Our analysis revealed that the intrinsic brittleness of L21-Ni2AlTi phase and L21-Ni2AlTi/BCC-Fe interface plays a vital role in determining the deformation behavior of the aged Fe–10Cr–5Ni–1Al–1Ti alloy., Godkänd;2023;Nivå 0;2023-08-08 (joosat);Funder: Natural Science Foundation of China (52074032, 51974029, 52101152, 52130407); Natural Science Foundation of Hunan Province (2022JJ30564, 2022JJ40438); Guangdong Basic and Applied Basic Research Foundation (2021B1515120033); Beijing Natural Science Foundation (2232084); State Key Laboratory of Powder Metallurgy, Central South University, Changsha, ChinaLicens fulltext: CC BY-NC-ND License

Ductility deterioration induced by L21-Ni2AlTi precipitates in the aged ferritic alloys was examined systematically by using a combination of scanning transmission electron microscope (STEM), mechanical tests and first-principles thermodynamic calculations. The experimental studies revealed that the strength and hardness of the aged Fe–10Cr–5Ni–1Al–1Ti ferritic alloy containing B2–NiAl and L21-Ni2AlTi precipitates were higher than that of the aged Fe–10Cr–5Ni–1Al ferritic alloy containing NiAl precipitates, whereas the elongation-to-failure decreased dramatically from 9.3% to 0.3% indicating an obvious ductility deterioration due to the formation of L21-Ni2AlTi precipitates. This was also confirmed by the observation of fracture transition mode from dimpled failure to cleavage failure. The first-principles calculations, concerning the precipitate/matrix interface, were carried out to provide a theoretical analysis for the ductile–brittle transition by means of empirical ductility criteria ratios G/B and (C12–C44)/B as well as cleavage energy. The cleavage energy results indicated an intrinsic brittleness of the L21-Ni2AlTi phase and the L21-Ni2AlTi/BCC-Fe interface. Our analysis revealed that the intrinsic brittleness of L21-Ni2AlTi phase and L21-Ni2AlTi/BCC-Fe interface plays a vital role in determining the deformation behavior of the aged Fe–10Cr–5Ni–1Al–1Ti alloy., Godkänd;2023;Nivå 0;2023-08-08 (joosat);Funder: Natural Science Foundation of China (52074032, 51974029, 52101152, 52130407); Natural Science Foundation of Hunan Province (2022JJ30564, 2022JJ40438); Guangdong Basic and Applied Basic Research Foundation (2021B1515120033); Beijing Natural Science Foundation (2232084); State Key Laboratory of Powder Metallurgy, Central South University, Changsha, ChinaLicens fulltext: CC BY-NC-ND License

Atomic‐level manipulation of catalysts is important for both fundamental studies and practical applications. Here, the central metal atom in an atomically precise Ag25 nanocluster (NC) is replaced with a single Pd, Pt, and Au atom, respectively, and employed as a model system to study the structure–property–activity relationship at the atomic level. While the geometric structures are well‐preserved after doping, the electronic structures of Ag25 NCs are significantly altered. The combination of Ag25 and TiO2 enhances the charge separation at the interface, exhibiting a 10 times higher hydrogen production rate in photocatalytic hydrogen evolution reaction compared to bare TiO2. Further results show that heteroatoms doping has a negative impact on performance, particularly in the cases of Pd and Au doping. Ultraviolet photoelectron spectroscopy measurements and density functional theory calculations suggest that the lower activities are due to an energy mismatch between the levels of doped NCs and TiO2. These findings not only reveal the impact of heteroatoms doping on the electronic properties and photocatalytic activities of NCs, but can also guide the design of heterometallic NCs for photocatalytic applications., Heteroatoms doping alters the electronic structures of Ag25 nanoclusters (NCs), while keeping the geometric structures unchanged. In this way, the geometrical effects and the electronic effects are clearly distinguished. The photocatalytic activities of as‐prepared NCs deposited onto TiO2 are investigated. The performances follow the order of Ag25 ≥ PtAg24 > PdAg24 ≥ AuAg24. image © 2023 WILEY‐VCH GmbH, DFG http://dx.doi.org/10.13039/501100001659, China Scholarship Council http://dx.doi.org/10.13039/501100004543, Peer Reviewed

Xinyu Fan,1,2,* Jiachen Shi,1,* Ye Liu,3 Mengqiu Zhang,1 Min Lu,1,2 Ding Qu1,2 1Affiliated Hospital of Integrated Traditional Chinese and Western Medicine, Nanjing University of Chinese Medicine, Nanjing, People’s Republic of China; 2Jiangsu Province Academy of Traditional Chinese Medicine, Nanjing, People’s Republic of China; 3The Second Affiliated Hospital of Nanjing University of Chinese Medicine, Nanjing University of Chinese Medicine, Nanjing, People’s Republic of China*These authors contributed equally to this workCorrespondence: Min Lu, Jiangsu Province Academy of Traditional Chinese Medicine, 100 Shizi Street, Hongshan Road Nanjing 210028, People’s Republic of China, Email lmxlsr6666@163.com Ding Qu, Jiangsu Province Academy of Traditional Chinese Medicine, 100 Shizi Street, Hongshan Road, Nanjing, 210028, People’s Republic of China, Email quding1985@hotmail.comBackground: Berberine (BR) shows promise as a candidate for treating irritable bowel syndrome with diarrhea (IBS-D). However, the undesired physicochemical properties and poor oral absorption limit its clinical translation. A ketogenic diet (KD) can induce intestinal overexpression of cannabidiol (CB) receptors, which may offer a potential target for IBS-D-specific delivery of BR.Methods: The microemulsions loaded with BR and decorated with cannabidiol (CBD/BR-MEs) were developed through a one-step emulsion method. The pharmaceutical behaviors of the CBD/BR-MEs were measured using dynamic light scattering and high-performance liquid chromatography. The efficacy of the anti-IBS-D therapy was evaluated by assessing fecal water content, Bristol score, and AWR score. The intestinal permeability were assessed through immunofluorescent staining of CB1 and ZO-1, respectively. The signaling of CREB/BDNF/c-Fos was also studied along with immunofluorescent and immunohistochemical examination of brain sections.Results: The CBD/BR-MEs, which had a particle size of approximately 3

The oscillation phenomena associated with the control of voltage source converters (VSCs) are widely concerning, and locating the source of these oscillations is crucial to suppressing them; therefore, this paper presents a locating scheme, based on the energy structure and nonlinearity detection. On the one hand, the energy structure, which conforms with the principle of the energy-based method and dissipativity theory, is constructed to describe the transient energy flow for VSCs, and on this basis, a defined characteristic quantity is implemented to narrow the scope of oscillation source location; on the other hand, according to the self-sustained oscillation characteristics of VSCs, an index for nonlinearity detection is applied to locate the VSCs which produce the oscillation energy. The combination of the energy structure and nonlinearity detection could distinguish the contributions of different VSCs to the oscillation. The results of a case study implemented by the PSCAD/EMTDC simulation validate the proposed scheme.

Large language models~(LLMs) demonstrate significant potential to revolutionize software engineering (SE) by exhibiting outstanding performance in SE tasks such as code and document generation. However, the high reliability and risk control requirements in software engineering raise concerns about the lack of interpretability of LLMs. To address this concern, we conducted a study to evaluate the capabilities of LLMs and their limitations for code analysis in SE. We break down the abilities needed for artificial intelligence~(AI) models to address SE tasks related to code analysis into three categories: 1) syntax understanding, 2) static behavior understanding, and 3) dynamic behavior understanding. Our investigation focused on the ability of LLMs to comprehend code syntax and semantic structures, which include abstract syntax trees (AST), control flow graphs (CFG), and call graphs (CG). We employed four state-of-the-art foundational models, GPT4, GPT3.5, StarCoder and CodeLlama-13b-instruct. We assessed the performance of LLMs on cross-language tasks involving C, Java, Python, and Solidity. Our findings revealed that while LLMs have a talent for understanding code syntax, they struggle with comprehending code semantics, particularly dynamic semantics. We conclude that LLMs possess capabilities similar to an Abstract Syntax Tree (AST) parser, demonstrating initial competencies in static code analysis. Furthermore, our study highlights that LLMs are susceptible to hallucinations when interpreting code semantic structures and fabricating nonexistent facts. These results indicate the need to explore methods to verify the correctness of LLM output to ensure its dependability in SE. More importantly, our study provides an initial answer to why the codes generated by LLM are usually syntax-correct but vulnerable.

The dominant paradigm of textual question answering systems is based on end-to-end neural networks, which excels at answering natural language questions but falls short on complex ones. This stands in contrast to the broad adaptation of semantic parsing approaches over structured data sources (e.g., relational database, knowledge graphs), that convert natural language questions to logical forms and execute them with query engines. Towards combining the strengths of neural and symbolic methods, we propose a framework of question parsing and execution on textual QA. It comprises two central pillars: (1) We parse the question of varying complexity into an intermediate representation, named H-expression, which is composed of simple questions as the primitives and symbolic operations representing the relationships among them; (2) To execute the resulting H-expressions, we design a hybrid executor, which integrates the deterministic rules to translate the symbolic operations with a drop-in neural reader network to answer each decomposed simple question. Hence, the proposed framework can be viewed as a top-down question parsing followed by a bottom-up answer backtracking. The resulting H-expressions closely guide the execution process, offering higher precision besides better interpretability while still preserving the advantages of the neural readers for resolving its primitive elements. Our extensive experiments on MuSiQue, 2WikiQA, HotpotQA, and NQ show that the proposed parsing and hybrid execution framework outperforms existing approaches in supervised, few-shot, and zero-shot settings, while also effectively exposing its underlying reasoning process., Comment: EMNLP 2023 Findings

The first Multimodal Emotion Recognition Challenge (MER 2023) was successfully held at ACM Multimedia. The challenge focuses on system robustness and consists of three distinct tracks: (1) MER-MULTI, where participants are required to recognize both discrete and dimensional emotions; (2) MER-NOISE, in which noise is added to test videos for modality robustness evaluation; (3) MER-SEMI, which provides a large amount of unlabeled samples for semi-supervised learning. In this paper, we introduce the motivation behind this challenge, describe the benchmark dataset, and provide some statistics about participants. To continue using this dataset after MER 2023, please sign a new End User License Agreement and send it to our official email address merchallenge.contact@gmail.com. We believe this high-quality dataset can become a new benchmark in multimodal emotion recognition, especially for the Chinese research community.

Location recommendation plays a vital role in improving users' travel experience. The timestamp of the POI to be predicted is of great significance, since a user will go to different places at different times. However, most existing methods either do not use this kind of temporal information, or just implicitly fuse it with other contextual information. In this paper, we revisit the problem of location recommendation and point out that explicitly modeling temporal information is a great help when the model needs to predict not only the next location but also further locations. In addition, state-of-the-art methods do not make effective use of geographic information and suffer from the hard boundary problem when encoding geographic information by gridding. To this end, a Temporal Prompt-based and Geography-aware (TPG) framework is proposed. The temporal prompt is firstly designed to incorporate temporal information of any further check-in. A shifted window mechanism is then devised to augment geographic data for addressing the hard boundary problem. Via extensive comparisons with existing methods and ablation studies on five real-world datasets, we demonstrate the effectiveness and superiority of the proposed method under various settings. Most importantly, our proposed model has the superior ability of interval prediction. In particular, the model can predict the location that a user wants to go to at a certain time while the most recent check-in behavioral data is masked, or it can predict specific future check-in (not just the next one) at a given timestamp.

The popularity of concurrent transmissions (CT) has soared after recent studies have shown their feasibility on the four physical layers specified by BLE 5, hence providing an alternative to the use of IEEE 802.15.4 for the design of reliable and efficient low-power wireless protocols. However, to date, the extent to which physical layer properties affect the performance of CT has not yet been investigated in detail. This paper fills this gap and provides an extensive study on the impact of the physical layer on CT-based solutions using IEEE 802.15.4 and BLE 5. We first highlight through simulation how the impact of errors induced by relative carrier frequency offsets on the performance of CT highly depends on the choice of the underlying physical layer. We then confirm these observations experimentally on real hardware and with varying environmental conditions through an analysis of the bit error distribution across received packets, unveiling possible techniques to effectively handle these errors. We further study the performance of CT-based data collection and dissemination protocols in the presence of RF interference on a large-scale testbed, deriving insights on how the employed physical layer affects their dependability., Comment: arXiv admin note: substantial text overlap with arXiv:2005.13816

Predicting the next location is a highly valuable and common need in many location-based services such as destination prediction and route planning. The goal of next location recommendation is to predict the next point-of-interest a user might go to based on the user's historical trajectory. Most existing models learn mobility patterns merely from users' historical check-in sequences while overlooking the significance of user preference modeling. In this work, a novel Point-of-Interest Transformer (POIFormer) with contrastive user preference modeling is developed for end-to-end next location recommendation. This model consists of three major modules: history encoder, query generator, and preference decoder. History encoder is designed to model mobility patterns from historical check-in sequences, while query generator explicitly learns user preferences to generate user-specific intention queries. Finally, preference decoder combines the intention queries and historical information to predict the user's next location. Extensive comparisons with representative schemes and ablation studies on four real-world datasets demonstrate the effectiveness and superiority of the proposed scheme under various settings.

Just noticeable difference (JND) refers to the maximum visual change that human eyes cannot perceive, and it has a wide range of applications in multimedia systems. However, most existing JND approaches only focus on a single modality, and rarely consider the complementary effects of multimodal information. In this article, we investigate the JND modeling from an end-to-end homologous multimodal perspective, namely hmJND-Net. Specifically, we explore three important visually sensitive modalities, including saliency, depth, and segmentation. To better utilize homologous multimodal information, we establish an effective fusion method via summation enhancement and subtractive offset, and align homologous multimodal features based on a self-attention driven encoder-decoder paradigm. Extensive experimental results on eight different benchmark datasets validate the superiority of our hmJND-Net over eight representative methods.

Xiao-Hui Song,1,* Bo Liu,2,3,* Fang Lei,2,4 Ye-Mao Liu,2,5,6 Xingyuan Zhang,2,4 Ze Chen,7 Peng Zhang,2,4 Xiao-Jing Zhang,2,4 Zhi-Gang She,1,2 Jingjing Cai,2,8 Jin-Hua Wang,9 Hongliang Li1– 3,6 1Department of Cardiology, Renmin Hospital of Wuhan University, Wuhan, People’s Republic of China; 2Institute of Model Animal, Wuhan University, Wuhan, People’s Republic of China; 3Medical Science Research Center, Zhongnan Hospital of Wuhan University, Wuhan, People’s Republic of China; 4School of Basic Medical Science, Wuhan University, Wuhan, People’s Republic of China; 5Department of Cardiology, Huanggang Central Hospital of Yangtze University, Huanggang, People’s Republic of China; 6Huanggang Institute of Translational Medicine, Huanggang Central Hospital of Yangtze University, Huanggang, People’s Republic of China; 7Department of Cardiology, Zhongnan Hospital of Wuhan University, Wuhan, People’s Republic of China; 8Department of Cardiology, The Third Xiangya Hospital, Central South University, Changsha, People’s Republic of China; 9Department of Neurology, Huanggang Central Hospital of Yangtze University, Hubei, People’s Republic of China*These authors contributed equally to this workCorrespondence: Hongliang Li, Department of Cardiology, Renmin Hospital of Wuhan University, Luojia Mount Wuchang, Wuhan, 430072, People’s Republic of China, Tel/Fax +86-27-68759302, Email lihl@whu.edu.cn Jin-Hua Wang, Department of Neurology, Huanggang Central Hospital of Yangtze University, Hubei, 438000, People’s Republic of China, Email wangjinhua1985@qq.comPurpose: Emerging evidence suggested that metabolic dysfunction-associated fatty liver disease (MAFLD) was significantly associated with atherosclerotic diseases. Atherosclerosis in the peripheral arteries is the most common cause of peripheral arterial disease (PAD), which has not been substantially controlled in the past. We aimed to investigate the association between MAFLD and PAD in

Smart contracts are computer programs running on blockchains to automate the transaction execution between users. The absence of contract specifications poses a real challenge to the correctness verification of smart contracts. Program invariants are properties that are always preserved throughout the execution, which characterize an important aspect of the program behaviors. In this paper, we propose a novel invariant generation framework, INVCON+, for Solidity smart contracts. INVCON+ extends the existing invariant detector, InvCon, to automatically produce verified contract invariants based on both dynamic inference and static verification. Unlike INVCON+, InvCon only produces likely invariants, which have a high probability to hold, yet are still not verified against the contract code. Particularly, INVCON+ is able to infer more expressive invariants that capture richer semantic relations of contract code. We evaluate INVCON+ on 361 ERC20 and 10 ERC721 real-world contracts, as well as common ERC20 vulnerability benchmarks. The experimental results indicate that INVCON+ efficiently produces high-quality invariant specifications, which can be used to secure smart contracts from common vulnerabilities.

Task-Oriented Parsing (TOP) enables conversational assistants to interpret user commands expressed in natural language, transforming them into structured outputs that combine elements of both natural language and intent/slot tags. Recently, Large Language Models (LLMs) have achieved impressive performance in synthesizing computer programs based on a natural language prompt, mitigating the gap between natural language and structured programs. Our paper focuses on harnessing the capabilities of LLMs for semantic parsing tasks, addressing the following three key research questions: 1) How can LLMs be effectively utilized for semantic parsing tasks? 2) What defines an effective prompt? and 3) How can LLM overcome the length constraint and streamline prompt design by including all examples as prompts? We introduce k Nearest Neighbor In-Context Learning(kNN-ICL), which simplifies prompt engineering by allowing it to be built on top of any design strategy while providing access to all demo examples. Extensive experiments show that: 1)Simple ICL without kNN search can achieve a comparable performance with strong supervised models on the TOP tasks, and 2) kNN-ICL significantly improves the comprehension of complex requests by seamlessly integrating ICL with a nearest-neighbor approach. Notably, this enhancement is achieved without the need for additional data or specialized prompts.

Decentralized Finance (DeFi) is a prominent application of smart contracts, representing a novel financial paradigm in contrast to centralized finance. While DeFi applications are rapidly emerging on mainstream blockchain platforms, their quality varies greatly, presenting numerous challenges, particularly in terms of their governance mechanisms. In this paper, we present a comprehensive study of governance issues in DeFi applications. Drawing upon insights from industry reports and academic research articles, we develop a taxonomy to categorize these governance issues. We collect and build a dataset of 4,446 audit reports from 17 Web3 security companies, categorizing their governance issues according to our constructed taxonomy. We conducted a thorough analysis of governance issues and identified vulnerabilities in governance design and implementation, e.g., voting sybil attack and proposal front-running. Our findings highlight a significant observation: the disparity between smart contract code and DeFi whitepapers plays a central role in these governance issues. As an initial step to address the challenges of code-whitepaper consistency checks for DeFi applications, we built a machine-learning-based prototype, and validated its performance on eight widely used DeFi projects, achieving a 56.14% F1 score and a 80% recall. Our study culminates in providing several key practical implications for various DeFi stakeholders, including developers, users, researchers, and regulators, aiming to deepen the understanding of DeFi governance issues and contribute to the robust growth of DeFi systems.

Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when solely relying on their internal knowledge, especially when answering questions that require less commonly known information. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge. Nonetheless, recent approaches have primarily emphasized retrieval from unstructured text corpora, owing to its seamless integration into prompts. When using structured data such as knowledge graphs, most methods simplify it into natural text, neglecting the underlying structures. Moreover, a significant gap in the current landscape is the absence of a realistic benchmark for evaluating the effectiveness of grounding LLMs on heterogeneous knowledge sources (e.g., knowledge base and text). To fill this gap, we have curated a comprehensive dataset that poses two unique challenges: (1) Two-hop multi-source questions that require retrieving information from both open-domain structured and unstructured knowledge sources; retrieving information from structured knowledge sources is a critical component in correctly answering the questions. (2) The generation of symbolic queries (e.g., SPARQL for Wikidata) is a key requirement, which adds another layer of challenge. Our dataset is created using a combination of automatic generation through predefined reasoning chains and human annotation. We also introduce a novel approach that leverages multiple retrieval tools, including text passage retrieval and symbolic language-assisted retrieval. Our model outperforms previous approaches by a significant margin, demonstrating its effectiveness in addressing the above-mentioned reasoning challenges.

While Chain-of-Thought prompting is popular in reasoning tasks, its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. Motivated by multi-step reasoning of LLMs, we propose Coarse-to-Fine Chain-of-Thought (CoF-CoT) approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire and leverage essential concepts to solve tasks from different granularities. Moreover, we propose leveraging semantic-based Abstract Meaning Representation (AMR) structured knowledge as an intermediate step to capture the nuances and diverse structures of utterances, and to understand connections between their varying levels of granularity. Our proposed approach is demonstrated effective in assisting the LLMs adapt to the multi-grained NLU tasks under both zero-shot and few-shot multi-domain settings., Comment: Accepted at EMNLP 2023 (Main Conference)

Extracting structured information from videos is critical for numerous downstream applications in the industry. In this paper, we define a significant task of extracting hierarchical key information from visual texts on videos. To fulfill this task, we decouple it into four subtasks and introduce two implementation solutions called PipVKIE and UniVKIE. PipVKIE sequentially completes the four subtasks in continuous stages, while UniVKIE is improved by unifying all the subtasks into one backbone. Both PipVKIE and UniVKIE leverage multimodal information from vision, text, and coordinates for feature representation. Extensive experiments on one well-defined dataset demonstrate that our solutions can achieve remarkable performance and efficient inference speed.

Recently, large language models (LLMs), especially those that are pretrained on code, have demonstrated strong capabilities in generating programs from natural language inputs in a few-shot or even zero-shot manner. Despite promising results, there is a notable lack of a comprehensive evaluation of these models language-to-code generation capabilities. Existing studies often focus on specific tasks, model architectures, or learning paradigms, leading to a fragmented understanding of the overall landscape. In this work, we present L2CEval, a systematic evaluation of the language-to-code generation capabilities of LLMs on 7 tasks across the domain spectrum of semantic parsing, math reasoning and Python programming, analyzing the factors that potentially affect their performance, such as model size, pretraining data, instruction tuning, and different prompting methods. In addition to assessing model performance, we measure confidence calibration for the models and conduct human evaluations of the output programs. This enables us to identify and analyze the typical failure modes across various tasks and models. L2CEval offers a comprehensive understanding of the capabilities and limitations of LLMs in language-to-code generation. We also release the evaluation framework and all model outputs, hoping to lay the groundwork for further future research in this domain., Comment: Project Website: https://l2c-eval.github.io

Although convolutional neural networks (CNNs) have been proposed to remove adverse weather conditions in single images using a single set of pre-trained weights, they fail to restore weather videos due to the absence of temporal information. Furthermore, existing methods for removing adverse weather conditions (e.g., rain, fog, and snow) from videos can only handle one type of adverse weather. In this work, we propose the first framework for restoring videos from all adverse weather conditions by developing a video adverse-weather-component suppression network (ViWS-Net). To achieve this, we first devise a weather-agnostic video transformer encoder with multiple transformer stages. Moreover, we design a long short-term temporal modeling mechanism for weather messenger to early fuse input adjacent video frames and learn weather-specific information. We further introduce a weather discriminator with gradient reversion, to maintain the weather-invariant common information and suppress the weather-specific information in pixel features, by adversarially predicting weather types. Finally, we develop a messenger-driven video transformer decoder to retrieve the residual weather-specific feature, which is spatiotemporally aggregated with hierarchical pixel features and refined to predict the clean target frame of input videos. Experimental results, on benchmark datasets and real-world weather videos, demonstrate that our ViWS-Net outperforms current state-of-the-art methods in terms of restoring videos degraded by any weather condition.

Question answering on tabular data (a.k.a TableQA), which aims at generating answers to questions grounded on a provided table, has gained significant attention recently. Prior work primarily produces concise factual responses through information extraction from individual or limited table cells, lacking the ability to reason across diverse table cells. Yet, the realm of free-form TableQA, which demands intricate strategies for selecting relevant table cells and the sophisticated integration and inference of discrete data fragments, remains mostly unexplored. To this end, this paper proposes a generalized three-stage approach: Table-to- Graph conversion and cell localizing, external knowledge retrieval, and the fusion of table and text (called TAG-QA), to address the challenge of inferring long free-form answers in generative TableQA. In particular, TAG-QA (1) locates relevant table cells using a graph neural network to gather intersecting cells between relevant rows and columns, (2) leverages external knowledge from Wikipedia, and (3) generates answers by integrating both tabular data and natural linguistic information. Experiments showcase the superior capabilities of TAG-QA in generating sentences that are both faithful and coherent, particularly when compared to several state-of-the-art baselines. Notably, TAG-QA surpasses the robust pipeline-based baseline TAPAS by 17% and 14% in terms of BLEU-4 and PARENT F-score, respectively. Furthermore, TAG-QA outperforms the end-to-end model T5 by 16% and 12% on BLEU-4 and PARENT F-score, respectively., Comment: Accepted by AACL-IJCNLP 2023

Large Language Models (LLMs) have gained prominence in the field of Legal Intelligence, offering potential applications in assisting legal professionals and laymen. However, the centralized training of these Legal LLMs raises data privacy concerns, as legal data is distributed among various institutions containing sensitive individual information. This paper addresses this challenge by exploring the integration of Legal LLMs with Federated Learning (FL) methodologies. By employing FL, Legal LLMs can be fine-tuned locally on devices or clients, and their parameters are aggregated and distributed on a central server, ensuring data privacy without directly sharing raw data. However, computation and communication overheads hinder the full fine-tuning of LLMs under the FL setting. Moreover, the distribution shift of legal data reduces the effectiveness of FL methods. To this end, in this paper, we propose the first Federated Legal Large Language Model (FedJudge) framework, which fine-tunes Legal LLMs efficiently and effectively. Specifically, FedJudge utilizes parameter-efficient fine-tuning methods to update only a few additional parameters during the FL training. Besides, we explore the continual learning methods to preserve the global model's important parameters when training local clients to mitigate the problem of data shifts. Extensive experimental results on three real-world datasets clearly validate the effectiveness of FedJudge. Code is released at https://github.com/yuelinan/FedJudge., Comment: Accepted to DASFAA 2024

As we embark on a new era of LLMs, it becomes increasingly crucial to understand their capabilities, limitations, and differences. Toward making further progress in this direction, we strive to build a deeper understanding of the gaps between massive LLMs (e.g., ChatGPT) and smaller yet effective open-source LLMs and their distilled counterparts. To this end, we specifically focus on long-form question answering (LFQA) because it has several practical and impactful applications (e.g., troubleshooting, customer service, etc.) yet is still understudied and challenging for LLMs. We propose a question-generation method from abstractive summaries and show that generating follow-up questions from summaries of long documents can create a challenging setting for LLMs to reason and infer from long contexts. Our experimental results confirm that: (1) our proposed method of generating questions from abstractive summaries pose a challenging setup for LLMs and shows performance gaps between LLMs like ChatGPT and open-source LLMs (Alpaca, Llama) (2) open-source LLMs exhibit decreased reliance on context for generated questions from the original document, but their generation capabilities drop significantly on generated questions from summaries -- especially for longer contexts (>1024 tokens)

Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context. To address this, we have trained XGen, a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We have also finetuned the XGen models on public-domain instructional data, creating their instruction-tuned counterparts (XGen-Inst). We open-source our models for both research advancements and commercial applications. Our evaluation on standard benchmarks shows that XGen models achieve comparable or better results when compared with state-of-the-art open-source LLMs. Our targeted evaluation on long sequence modeling tasks shows the benefits of our 8K-sequence models over 2K-sequence open-source LLMs.

The multimedia community has shown a significant interest in perceiving and representing the physical world with multimodal pretrained neural network models, and among them, the visual-language pertaining (VLP) is, currently, the most captivating topic. However, there have been few endeavors dedicated to the exploration of 1) whether essential linguistic knowledge (e.g., semantics and syntax) can be extracted during VLP, and 2) how such linguistic knowledge impact or enhance the multimodal alignment. In response, here we aim to elucidate the impact of comprehensive linguistic knowledge, including semantic expression and syntactic structure, on multimodal alignment. Specifically, we design and release the SNARE, the first large-scale multimodal alignment probing benchmark, to detect the vital linguistic components, e.g., lexical, semantic, and syntax knowledge, containing four tasks: Semantic structure, Negation logic, Attribute ownership, and Relationship composition. Based on our proposed probing benchmarks, our holistic analyses of five advanced VLP models illustrate that the VLP model: i) shows insensitivity towards complex syntax structures and relies on content words for sentence comprehension; ii) demonstrates limited comprehension of combinations between sentences and negations; iii) faces challenges in determining the presence of actions or spatial relationships within visual information and struggles with verifying the correctness of triple combinations. We make our benchmark and code available at \url{https://github.com/WangFei-2019/SNARE/}., Comment: [TL;DR] we design and release the SNARE, the first large-scale multimodal alignment probing benchmark for current vision-language pretrained models

Studies on semi-supervised medical image segmentation (SSMIS) have seen fast progress recently. Due to the limited labelled data, SSMIS methods mainly focus on effectively leveraging unlabeled data to enhance the segmentation performance. However, despite their promising performance, current state-of-the-art methods often prioritize integrating complex techniques and loss terms rather than addressing the core challenges of semi-supervised scenarios directly. We argue that the key to SSMIS lies in generating substantial and appropriate prediction disagreement on unlabeled data. To this end, we emphasize the crutiality of data perturbation and model stabilization in semi-supervised segmentation, and propose a simple yet effective approach to boost SSMIS performance significantly, dubbed DPMS. Specifically, we first revisit SSMIS from three distinct perspectives: the data, the model, and the loss, and conduct a comprehensive study of corresponding strategies to examine their effectiveness. Based on these examinations, we then propose DPMS, which adopts a plain teacher-student framework with a standard supervised loss and unsupervised consistency loss. To produce appropriate prediction disagreements, DPMS perturbs the unlabeled data via strong augmentations to enlarge prediction disagreements considerably. On the other hand, using EMA teacher when strong augmentation is applied does not necessarily improve performance. DPMS further utilizes a forwarding-twice and momentum updating strategies for normalization statistics to stabilize the training on unlabeled data effectively. Despite its simplicity, DPMS can obtain new state-of-the-art performance on the public 2D ACDC and 3D LA datasets across various semi-supervised settings, e.g. obtaining a remarkable 22.62% improvement against previous SOTA on ACDC with 5% labels., Comment: Code and logs are available at https://github.com/ZhenZHAO/DPMS

The integration of retrieved passages and large language models (LLMs), such as ChatGPTs, has significantly contributed to improving open-domain question answering. However, there is still a lack of exploration regarding the optimal approach for incorporating retrieved passages into the answer generation process. This paper aims to fill this gap by investigating different methods of combining retrieved passages with LLMs to enhance answer generation. We begin by examining the limitations of a commonly-used concatenation approach. Surprisingly, this approach often results in generating "unknown" outputs, even when the correct document is among the top-k retrieved passages. To address this issue, we explore four alternative strategies for integrating the retrieved passages with the LLMs. These strategies include two single-round methods that utilize chain-of-thought reasoning and two multi-round strategies that incorporate feedback loops. Through comprehensive analyses and experiments, we provide insightful observations on how to effectively leverage retrieved passages to enhance the answer generation capability of LLMs.

Recent advanced methods in Natural Language Understanding for Task-oriented Dialogue (TOD) Systems (e.g., intent detection and slot filling) require a large amount of annotated data to achieve competitive performance. In reality, token-level annotations (slot labels) are time-consuming and difficult to acquire. In this work, we study the Slot Induction (SI) task whose objective is to induce slot boundaries without explicit knowledge of token-level slot annotations. We propose leveraging Unsupervised Pre-trained Language Model (PLM) Probing and Contrastive Learning mechanism to exploit (1) unsupervised semantic knowledge extracted from PLM, and (2) additional sentence-level intent label signals available from TOD. Our approach is shown to be effective in SI task and capable of bridging the gaps with token-level supervised models on two NLU benchmark datasets. When generalized to emerging intents, our SI objectives also provide enhanced slot label representations, leading to improved performance on the Slot Filling tasks., Comment: Accepted at SIGDIAL 2023

In this work, we study dialogue scenarios that start from chit-chat but eventually switch to task-related services, and investigate how a unified dialogue model, which can engage in both chit-chat and task-oriented dialogues, takes the initiative during the dialogue mode transition from chit-chat to task-oriented in a coherent and cooperative manner. We firstly build a {transition info extractor} (TIE) that keeps track of the preceding chit-chat interaction and detects the potential user intention to switch to a task-oriented service. Meanwhile, in the unified model, a {transition sentence generator} (TSG) is extended through efficient Adapter tuning and transition prompt learning. When the TIE successfully finds task-related information from the preceding chit-chat, such as a transition domain, then the TSG is activated automatically in the unified model to initiate this transition by generating a transition sentence under the guidance of transition information extracted by TIE. The experimental results show promising performance regarding the proactive transitions. We achieve an additional large improvement on TIE model by utilizing Conditional Random Fields (CRF). The TSG can flexibly generate transition sentences while maintaining the unified capabilities of normal chit-chat and task-oriented response generation., Comment: accepted by INLG 2023

Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness. To tackle these issues, we introduce DialogStudio: the largest and most diverse collection of dialogue datasets, unified under a consistent format while preserving their original information. Our collection encompasses data from open-domain dialogues, task-oriented dialogues, natural language understanding, conversational recommendation, dialogue summarization, and knowledge-grounded dialogues, making it an incredibly rich and diverse resource for dialogue research and model training. To further enhance the utility of DialogStudio, we identify the licenses for each dataset, design external knowledge and domain-aware prompts for selected dialogues to facilitate instruction-aware fine-tuning. Furthermore, we develop conversational AI models using the dataset collection, and our experiments in both zero-shot and few-shot learning scenarios demonstrate the superiority of DialogStudio. To improve transparency and support dataset and task-based research, as well as language model pre-training, all datasets, licenses, codes, and models associated with DialogStudio are made publicly accessible\footnote{\url{https://github.com/salesforce/DialogStudio}}., Comment: 17 pages, accepted by EACL 2024 Findings as a long paper. All datasets, licenses, codes, and models are available at at https://github.com/salesforce/DialogStudio

Choice modeling is at the core of understanding how changes to the competitive landscape affect consumer choices and reshape market equilibria. In this paper, we propose a fundamental characterization of choice functions that encompasses a wide variety of extant choice models. We demonstrate how non-parametric estimators like neural nets can easily approximate such functionals and overcome the curse of dimensionality that is inherent in the non-parametric estimation of choice functions. We demonstrate through extensive simulations that our proposed functionals can flexibly capture underlying consumer behavior in a completely data-driven fashion and outperform traditional parametric models. As demand settings often exhibit endogenous features, we extend our framework to incorporate estimation under endogenous features. Further, we also describe a formal inference procedure to construct valid confidence intervals on objects of interest like price elasticity. Finally, to assess the practical applicability of our estimator, we utilize a real-world dataset from S. Berry, Levinsohn, and Pakes (1995). Our empirical analysis confirms that the estimator generates realistic and comparable own- and cross-price elasticities that are consistent with the observations reported in the existing literature.