Search

How are LLM-based agents used in the future? While many of the existing work on agents has focused on improving the performance of a specific family of objective and challenging tasks, in this work, we take a different perspective by thinking about full delegation: agents take over humans' routine decision-making processes and are trusted by humans to find solutions that fit people's personalized needs and are adaptive to ever-changing context. In order to achieve such a goal, the behavior of the agents, i.e., agentic behaviors, should be evaluated not only on their achievements (i.e., outcome evaluation), but also how they achieved that (i.e., procedure evaluation). For this, we propose APEC Agent Constitution, a list of criteria that an agent should follow for good agentic behaviors, including Accuracy, Proactivity, Efficiency and Credibility. To verify whether APEC aligns with human preferences, we develop APEC-Travel, a travel planning agent that proactively extracts hidden personalized needs via multi-round dialog with travelers. APEC-Travel is constructed purely from synthetic data generated by Llama3.1-405B-Instruct with a diverse set of travelers' persona to simulate rich distribution of dialogs. Iteratively fine-tuned to follow APEC Agent Constitution, APEC-Travel surpasses baselines by 20.7% on rule-based metrics and 9.1% on LLM-as-a-Judge scores across the constitution axes.

Travel planning is a challenging and time-consuming task that aims to find an itinerary which satisfies multiple, interdependent constraints regarding flights, accommodations, attractions, and other travel arrangements. In this paper, we propose To the Globe (TTG), a real-time demo system that takes natural language requests from users, translates it to symbolic form via a fine-tuned Large Language Model, and produces optimal travel itineraries with Mixed Integer Linear Programming solvers. The overall system takes ~5 seconds to reply to the user request with guaranteed itineraries. To train TTG, we develop a synthetic data pipeline that generates user requests, flight and hotel information in symbolic form without human annotations, based on the statistics of real-world datasets, and fine-tune an LLM to translate NL user requests to their symbolic form, which is sent to the symbolic solver to compute optimal itineraries. Our NL-symbolic translation achieves ~91% exact match in a backtranslation metric (i.e., whether the estimated symbolic form of generated natural language matches the groundtruth), and its returned itineraries have a ratio of 0.979 compared to the optimal cost of the ground truth user request. When evaluated by users, TTG achieves consistently high Net Promoter Scores (NPS) of 35-40% on generated itinerary.

Randomized neural network (RNN) methods have been proposed for solving various partial differential equations (PDEs), demonstrating high accuracy and efficiency. However, initializing the fixed parameters remains a challenging issue. Additionally, RNNs often struggle to solve PDEs with sharp or discontinuous solutions. In this paper, we propose a novel approach called Adaptive Growing Randomized Neural Network (AG-RNN) to address these challenges. First, we establish a parameter initialization strategy based on frequency information to construct the initial RNN. After obtaining a numerical solution from this initial network, we use the residual as an error indicator. Based on the error indicator, we introduce growth strategies that expand the neural network, making it wider and deeper to improve the accuracy of the numerical solution. A key feature of AG-RNN is its adaptive strategy for determining the weights and biases of newly added neurons, enabling the network to expand in both width and depth without requiring additional training. Instead, all weights and biases are generated constructively, significantly enhancing the network's approximation capabilities compared to conventional randomized neural network methods. In addition, a domain splitting strategy is introduced to handle the case of discontinuous solutions. Extensive numerical experiments are conducted to demonstrate the efficiency and accuracy of this innovative approach.

This paper investigates the isentropic compressible Navier-Stokes equations on k-connected domains under Navier-slip boundary conditions. We study the multi-solvability of the stationary systems on general domains, which is closely related with the Cauchy-Riemann systems and critical points of harmonic functions on the domain. Then based on the structure of Green's functions, the commutator estimates are obtained on the circular domains and extended to general domains with the help of conformal mappings. Moreover, we will utilize these assertions to discuss the global well-posedness and large time behaviours of the non-stationary systems on general domains with large initial values containing vacuum.

We develop a Macroscopic Auxiliary Asymptotic-Preserving Neural Network (MA-APNN) method to solve the time-dependent linear radiative transfer equations (LRTEs), which have a multi-scale nature and high dimensionality. To achieve this, we utilize the Physics-Informed Neural Networks (PINNs) framework and design a new adaptive exponentially weighted Asymptotic-Preserving (AP) loss function, which incorporates the macroscopic auxiliary equation that is derived from the original transfer equation directly and explicitly contains the information of the diffusion limit equation. Thus, as the scale parameter tends to zero, the loss function gradually transitions from the transport state to the diffusion limit state. In addition, the initial data, boundary conditions, and conservation laws serve as the regularization terms for the loss. We present several numerical examples to demonstrate the effectiveness of MA-APNNs., Comment: 24 pages, 29 figures

This paper studies the isothermal stationary compressible Navier-Stokes equations on global space and cylinder domains. There are two critical exponents in such settings: The heat ratio 1 is an end point of the classical theory on weak solutions; the axisymmetric solutions in the global domain involve the critical index of Sobolev's inequality. Some new observations based on the cancellation structure of the equations are collected to get over these obstacles due to critical exponents.

Chain-of-thought (CoT) prompting, which offers step-by-step problem-solving rationales, has impressively unlocked the reasoning potential of large language models (LLMs). Yet, the standard CoT is less effective in problems demanding multiple reasoning steps. This limitation arises from the complex reasoning process in multi-step problems: later stages often depend on the results of several steps earlier, not just the results of the immediately preceding step. Such complexities suggest the reasoning process is naturally represented as a graph. The almost linear and straightforward structure of CoT prompting, however, struggles to capture this complex reasoning graph. To address this challenge, we propose Residual Connection Prompting (RESPROMPT), a new prompting strategy that advances multi-step reasoning in LLMs. Our key idea is to reconstruct the reasoning graph within prompts. We achieve this by integrating necessary connections-links present in the reasoning graph but missing in the linear CoT flow-into the prompts. Termed "residual connections", these links are pivotal in morphing the linear CoT structure into a graph representation, effectively capturing the complex reasoning graphs inherent in multi-step problems. We evaluate RESPROMPT on six benchmarks across three diverse domains: math, sequential, and commonsense reasoning. For the open-sourced LLaMA family of models, RESPROMPT yields a significant average reasoning accuracy improvement of 12.5% on LLaMA-65B and 6.8% on LLaMA2-70B. Breakdown analysis further highlights RESPROMPT particularly excels in complex multi-step reasoning: for questions demanding at least five reasoning steps, RESPROMPT outperforms the best CoT based benchmarks by a remarkable average improvement of 21.1% on LLaMA-65B and 14.3% on LLaMA2-70B. Through extensive ablation studies and analyses, we pinpoint how to most effectively build residual connections., Comment: 29 pages

A spatial second-order scheme for the nonlinear radiative transfer equations is introduced in this paper. The discretization scheme is based on the filtered spherical harmonics ($FP_N$) method for the angular variable and the unified gas kinetic scheme (UGKS) framework for the spatial and temporal variables respectively. In order to keep the scheme positive and second-order accuracy, firstly, we use the implicit Monte Carlo linearization method [6] in the construction of the UGKS numerical boundary fluxes. Then, by carefully analyzing the constructed second-order fluxes involved in the macro-micro decomposition, which is induced by the $FP_N$ angular discretization, we establish the sufficient conditions that guarantee the positivity of the radiative energy density and material temperature. Finally, we employ linear scaling limiters for the angular variable in the $P_N$ reconstruction and for the spatial variable in the piecewise linear slopes reconstruction respectively, which are shown to be realizable and reasonable to enforce the sufficient conditions holding. Thus, the desired scheme, called the $PPFP_N$-based UGKS, is obtained. Furthermore, in the regime $\epsilon\ll 1$ and the regime $\epsilon=O(1)$, a simplified spatial second-order scheme, called the $PPFP_N$-based SUGKS, is presented, which possesses all the properties of the non-simplified one. Inheriting the merit of UGKS, the proposed schemes are asymptotic preserving. By employing the $FP_N$ method for the angular variable, the proposed schemes are almost free of ray effects. To our best knowledge, this is the first time that spatial second-order, positive, asymptotic preserving and almost free of ray effects schemes are constructed for the nonlinear radiative transfer equations without operator splitting. Various numerical experiments are included to validate the properties of the proposed schemes.

This paper investigates the relationship between graph convolution and Mixup techniques. Graph convolution in a graph neural network involves aggregating features from neighboring samples to learn representative features for a specific node or sample. On the other hand, Mixup is a data augmentation technique that generates new examples by averaging features and one-hot labels from multiple samples. One commonality between these techniques is their utilization of information from multiple samples to derive feature representation. This study aims to explore whether a connection exists between these two approaches. Our investigation reveals that, under two mild conditions, graph convolution can be viewed as a specialized form of Mixup that is applied during both the training and testing phases. The two conditions are: 1) \textit{Homophily Relabel} - assigning the target node's label to all its neighbors, and 2) \textit{Test-Time Mixup} - Mixup the feature during the test time. We establish this equivalence mathematically by demonstrating that graph convolution networks (GCN) and simplified graph convolution (SGC) can be expressed as a form of Mixup. We also empirically verify the equivalence by training an MLP using the two conditions to achieve comparable performance., Comment: Accepted by TMLR

Guo--Tice formally established in 2011 that the Rayleigh--Taylor instability inevitably occurs within stratified compressible viscous fluids in a slab domain $\mathbb{R}^2\times (h_-,h_+)$, irrespecive of the presence of interfacial surface tension, where the instability solutions are non-periodic with respect to both horizontal spacial variables $x_1$ and $x_2$, by applying a so-called ''normal mode'' method and a modified variational method to the linearized (motion) equations. It is a long-standing open problem, however, whether Guo--Tice's conclusion can be rigorously verified by the (original) nonlinear equations. This challenge arises due to the failure of constructing a growing mode solution, which is non-periodic with respect to both horizontal spacial variables, to the linearized equations defined on a slab domain. In the present work, we circumvent the difficulty related to growing mode solutions by developing an alternative approximate scheme. In essence, our approach hinges on constructing the horizontally periodic growing mode solution of the linearized equations to approximate the {\it nonlinear} Rayleigh--Taylor instability solutions, which do not exhibit horizontal periodicity. Thanks to this new approximate scheme, we can apply Guo--Hallstrom--Spirn's bootstrap instability method to the nonlinear equations in Lagrangian coordinates, and thus prove Guo--Tice's conclusion. In particular, our approximate method could also be applied to other instability solutions characterized by non-periodic motion in a slab domain, such as the Parker instability and thermal instability., Comment: arXiv admin note: text overlap with arXiv:0911.4703 by other authors

Text-based recommendation holds a wide range of practical applications due to its versatility, as textual descriptions can represent nearly any type of item. However, directly employing the original item descriptions may not yield optimal recommendation performance due to the lack of comprehensive information to align with user preferences. Recent advances in large language models (LLMs) have showcased their remarkable ability to harness commonsense knowledge and reasoning. In this study, we introduce a novel approach, coined LLM-Rec, which incorporates four distinct prompting strategies of text enrichment for improving personalized text-based recommendations. Our empirical experiments reveal that using LLM-augmented text significantly enhances recommendation quality. Even basic MLP (Multi-Layer Perceptron) models achieve comparable or even better results than complex content-based methods. Notably, the success of LLM-Rec lies in its prompting strategies, which effectively tap into the language model's comprehension of both general and specific item characteristics. This highlights the importance of employing diverse prompts and input augmentation techniques to boost the recommendation effectiveness of LLMs.

Multi-agent dynamical systems refer to scenarios where multiple units interact with each other and evolve collectively over time. To make informed decisions in multi-agent dynamical systems, such as determining the optimal vaccine distribution plan, it is essential for decision-makers to estimate the continuous-time counterfactual outcomes. However, existing studies of causal inference over time rely on the assumption that units are mutually independent, which is not valid for multi-agent dynamical systems. In this paper, we aim to bridge this gap and study how to estimate counterfactual outcomes in multi-agent dynamical systems. Causal inference in a multi-agent dynamical system has unique challenges: 1) Confounders are time-varying and are present in both individual unit covariates and those of other units; 2) Units are affected by not only their own but also others' treatments; 3) The treatments are naturally dynamic, such as receiving vaccines and boosters in a seasonal manner. We model a multi-agent dynamical system as a graph and propose CounterFactual GraphODE (CF-GODE), a causal model that estimates continuous-time counterfactual outcomes in the presence of inter-dependencies between units. To facilitate continuous-time estimation, we propose Treatment-Induced GraphODE, a novel ordinary differential equation based on GNN, which incorporates dynamical treatments as additional inputs to predict potential outcomes over time. To remove confounding bias, we propose two domain adversarial learning based objectives that learn balanced continuous representation trajectories, which are not predictive of treatments and interference. We further provide theoretical justification to prove their effectiveness. Experiments on two semi-synthetic datasets confirm that CF-GODE outperforms baselines on counterfactual estimation. We also provide extensive analyses to understand how our model works., Comment: 13 pages, 8 figures

This paper concerns the sharp interface limit of solutions to the inhomogeneous incompressible Navier-Stokes/Allen-Cahn coupled system in a bounded domain $\Omega \subset \mathbb{R}^n,\ n =2,3$. Based on a relative energy method, we prove that the solutions to the Navier-Stokes/Allen-Cahn system converge to the corresponding solutions to a sharp interface model provided that the thickness of the diffuse interfacial zone goes to zero. It is noted that the relative energy method can avoid both the spectral estimates of the linearized Allen-Cahn operator and the construction of approximate solutions by the matched asymptotic expansion method in the study of the sharp interface limit process. And some suitable functionals are designed and estimated by elaborated energy methods accordingly.

We propose a model-data asymptotic-preserving neural network(MD-APNN) method to solve the nonlinear gray radiative transfer equations(GRTEs). The system is challenging to be simulated with both the traditional numerical schemes and the vanilla physics-informed neural networks(PINNs) due to the multiscale characteristics. Under the framework of PINNs, we employ a micro-macro decomposition technique to construct a new asymptotic-preserving(AP) loss function, which includes the residual of the governing equations in the micro-macro coupled form, the initial and boundary conditions with additional diffusion limit information, the conservation laws, and a few labeled data. A convergence analysis is performed for the proposed method, and a number of numerical examples are presented to illustrate the efficiency of MD-APNNs, and particularly, the importance of the AP property in the neural networks for the diffusion dominating problems. The numerical results indicate that MD-APNNs lead to a better performance than APNNs or pure data-driven networks in the simulation of the nonlinear non-stationary GRTEs.

We prove the existence and stability of smooth solutions to the steady Navier-Stokes equations near plane Poiseuille-Couette flow. Consequently, we also provide the zero viscosity limit of the 2D steady Navier-Stokes equations to the steady Euler equations. First, in the absence of any external force, we prove that there exist smooth solutions to the steady Navier-Stokes equations which are stable under infinitesimal perturbations of plane Poiseuille-Couette flow. In particular, if the basic flow is the Couette flow, then we can prove that the flow is stable for any finite perturbation small enough. Moreover, we also show that, if we put a proper external force to control the flow, then we can also obtain a large class of smooth solution of the steady Navier-Stokes equations which is stable for infinitesimal perturbation of the external force. Finally, based on the same linear estimates, we establish the zero viscosity limit of all the solutions obtained above to the solutions of the Euler equations.

This paper deals with the problem of learning the probabilities of causation of subpopulations given finite population data. The tight bounds of three basic probabilities of causation, the probability of necessity and sufficiency (PNS), the probability of sufficiency (PS), and the probability of necessity (PN), were derived by Tian and Pearl. However, obtaining the bounds for each subpopulation requires experimental and observational distributions of each subpopulation, which is usually impractical to estimate given finite population data. We propose a machine learning model that helps to learn the bounds of the probabilities of causation for subpopulations given finite population data. We further show by a simulated study that the machine learning model is able to learn the bounds of PNS for 32768 subpopulations with only knowing roughly 500 of them from the finite population data.

The unit selection problem is to identify a group of individuals who are most likely to exhibit a desired mode of behavior, for example, selecting individuals who would respond one way if incentivized and a different way if not. The unit selection problem consists of evaluation and search subproblems. Li and Pearl defined the "benefit function" to evaluate the average payoff of selecting a certain individual with given characteristics. The search subproblem is then to design an algorithm to identify the characteristics that maximize the above benefit function. The hardness of the search subproblem arises due to the large number of characteristics available for each individual and the sparsity of the data available in each cell of characteristics. In this paper, we present a machine learning framework that uses the bounds of the benefit function that are estimable from the finite population data to learn the bounds of the benefit function for each cell of characteristics. Therefore, we could easily obtain the characteristics that maximize the benefit function.

In knowledge graph completion (KGC), predicting triples involving emerging entities and/or relations, which are unseen when the KG embeddings are learned, has become a critical challenge. Subgraph reasoning with message passing is a promising and popular solution. Some recent methods have achieved good performance, but they (i) usually can only predict triples involving unseen entities alone, failing to address more realistic fully inductive situations with both unseen entities and unseen relations, and (ii) often conduct message passing over the entities with the relation patterns not fully utilized. In this study, we propose a new method named RMPI which uses a novel Relational Message Passing network for fully Inductive KGC. It passes messages directly between relations to make full use of the relation patterns for subgraph reasoning with new techniques on graph transformation, graph pruning, relation-aware neighborhood attention, addressing empty subgraphs, etc., and can utilize the relation semantics defined in the ontological schema of KG. Extensive evaluation on multiple benchmarks has shown the effectiveness of techniques involved in RMPI and its better performance compared with the existing methods that support fully inductive KGC. RMPI is also comparable to the state-of-the-art partially inductive KGC methods with very promising results achieved. Our codes and data are available at https://github.com/zjukg/RMPI., Comment: Accepted by ICDE 2023

A scanning tunneling microscope is used to study the fluorescence of a model charged molecule (quinacridone) adsorbed on a sodium chloride (NaCl)-covered metallic sample. Fluorescence from the neutral and positively charged species is reported and imaged using hyper-resolved fluorescence microscopy. A many-body excitation model is established based on a detailed analysis of voltage, current and spatial dependencies of the fluorescence and electron transport features. This model reveals that quinacridone adopts a large palette of charge states, transient or not, depending on the voltage used and the nature of the underlying substrate. This model has a universal character and explains the electronic and fluorescence properties of many other molecules adsorbed on thin insulators.

Excitonic emission from atomically precise graphene nanoribbons (GNRs) synthesised on a metal surface is probed with atomic-scale spatial resolution using a scanning tunneling microscopy (STM) approach. A STM-based strategy to transfer the GNRs to a partially insulating surface is used to prevent light emission quenching of the ribbons by the metal substrate. Sub-nanometer resolved STM-induced fluorescence spectra reveal emission from localized dark excitons build upon the topological end states of the GNRs. A low frequency vibronic emission comb whose characteristics change with the GNR length is attributed to longitudinal acoustic modes confined to a finite box. Overall, our study provides a novel path to investigate the interplay between excitons, vibrons and topology in atomically precise graphene nanostructures.

Atomically precise graphene nanoribbons (GNRs) have emerged as promising candidates for nanoelectronic applications due to their widely tunable energy band gaps resulting from lateral quantum confinement and edge effects. Here we report on the electronic transport characterization of an edge-modified GNR suspended between the tip of a scanning tunneling microscope (STM) and a Au(111) substrate. Differential conductance measurements on this metal-GNR-metal junction reveal loss-less transport properties (inverse decay length $\beta < 0.001 /\overset{\circ}{\mathrm{A}}$) with high conductance ($\sim 0.1$ G$_0$) at low voltages (50 meV) over long distances ($z > 10$ nm). The transport behavior is sensitive to the coupling between ribbon and electrodes, an effect that is rationalized using tight-binding and density functional theory simulations. From extensive modelling we infer that the length-independent transport is a manifestation of band transport through topological valence states, which originate from the zigzag segments on the GNR edges.

We are concerned with the sharp interface limit for an incompressible Navier-Stokes and Allen-Cahn coupled system in this paper. When the thickness of the diffuse interfacial zone, which is parameterized by $\varepsilon$, goes to zero, we prove that a solution of the incompressible Navier-Stokes and Allen-Cahn coupled system converges to a solution of a sharp interface model in the $L^\infty(L^2)\cap L^2(H^1)$ sense on a uniform time interval independent of the small parameter $\varepsilon$. The proof consists of two parts: one is the construction of a suitable approximate solution and another is the estimate of the error functions in Sobolev spaces. Besides the careful energy estimates, a spectral estimate of the linearized operator for the incompressible Navier-Stokes and Allen-Cahn coupled system around the approximate solution is essentially used to derive the uniform estimates of the error functions. The convergence of the velocity is well expected due to the fact that the layer of the velocity across the diffuse interfacial zone is relatively weak.