Search

Mixture of Experts (MoE) models are highly effective in scaling model capacity while preserving computational efficiency, with the gating network, or router, playing a central role by directing inputs to the appropriate experts. In this paper, we establish a novel connection between MoE frameworks and attention mechanisms, demonstrating how quadratic gating can serve as a more expressive and efficient alternative. Motivated by this insight, we explore the implementation of quadratic gating within MoE models, identifying a connection between the self-attention mechanism and the quadratic gating. We conduct a comprehensive theoretical analysis of the quadratic softmax gating MoE framework, showing improved sample efficiency in expert and parameter estimation. Our analysis provides key insights into optimal designs for quadratic gating and expert functions, further elucidating the principles behind widely used attention mechanisms. Through extensive evaluations, we demonstrate that the quadratic gating MoE outperforms the traditional linear gating MoE. Moreover, our theoretical insights have guided the development of a novel attention mechanism, which we validated through extensive experiments. The results demonstrate its favorable performance over conventional models across various tasks., Comment: Pedram Akbarian, Huy Nguyen, and Xing Han made equal contributions to this work

With the growing prominence of the Mixture of Experts (MoE) architecture in developing large-scale foundation models, we investigate the Hierarchical Mixture of Experts (HMoE), a specialized variant of MoE that excels in handling complex inputs and improving performance on targeted tasks. Our investigation highlights the advantages of using varied gating functions, moving beyond softmax gating within HMoE frameworks. We theoretically demonstrate that applying tailored gating functions to each expert group allows HMoE to achieve robust results, even when optimal gating functions are applied only at select hierarchical levels. Empirical validation across diverse scenarios supports these theoretical claims. This includes large-scale multimodal tasks, image classification, and latent domain discovery and prediction tasks, where our modified HMoE models show great performance improvements., Comment: 58 pages

With the growing demand for offline PDF chatbots in automotive industrial production environments, optimizing the deployment of large language models (LLMs) in local, low-performance settings has become increasingly important. This study focuses on enhancing Retrieval-Augmented Generation (RAG) techniques for processing complex automotive industry documents using locally deployed Ollama models. Based on the Langchain framework, we propose a multi-dimensional optimization approach for Ollama's local RAG implementation. Our method addresses key challenges in automotive document processing, including multi-column layouts and technical specifications. We introduce improvements in PDF processing, retrieval mechanisms, and context compression, tailored to the unique characteristics of automotive industry documents. Additionally, we design custom classes supporting embedding pipelines and an agent supporting self-RAG based on LangGraph best practices. To evaluate our approach, we constructed a proprietary dataset comprising typical automotive industry documents, including technical reports and corporate regulations. We compared our optimized RAG model and self-RAG agent against a naive RAG baseline across three datasets: our automotive industry dataset, QReCC, and CoQA. Results demonstrate significant improvements in context precision, context recall, answer relevancy, and faithfulness, with particularly notable performance on the automotive industry dataset. Our optimization scheme provides an effective solution for deploying local RAG systems in the automotive sector, addressing the specific needs of PDF chatbots in industrial production environments. This research has important implications for advancing information processing and intelligent production in the automotive industry.

Binary verification plays a pivotal role in software security, yet building a verification service that is both open and trustworthy poses a formidable challenge. In this paper, we introduce a novel binary verification service, AGORA, scrupulously designed to overcome the challenge. At the heart of this approach lies a strategic insight: certain tasks can be delegated to untrusted entities, while the corresponding validators are securely housed within the trusted computing base (TCB). AGORA can validate untrusted assertions generated for versatile policies. Through a novel blockchain-based bounty task manager, it also utilizes crowdsourcing to remove trust in theorem provers. These synergistic techniques successfully ameliorate the TCB size burden associated with two procedures: binary analysis and theorem proving. The design of AGORA allows untrusted parties to participate in these complex processes. Moreover, based on running the optimized TCB within trusted execution environments and recording the verification process on a blockchain, the public can audit the correctness of verification results. By implementing verification workflows for software-based fault isolation policy and side-channel mitigation, our evaluation demonstrates the efficacy of AGORA.

This paper investigates the neutrino transition magnetic moment in the $U(1)_X$SSM. $U(1)_X$SSM is the $U(1)$ extension of Minimal Supersymmetric Standard Model (MSSM) and its local gauge group is extended to $SU(3)_C\times SU(2)_L \times U(1)_Y\times U(1)_X$. To obtain this model, three singlet new Higgs superfields and right-handed neutrinos are added to the MSSM, which can explain the results of neutrino oscillation experiments. The neutrino transition magnetic moment is induced by electroweak radiative corrections. By applying effective Lagrangian method and on-shell scheme, we study the associated Feynman diagrams and the transition magnetic moment of neutrinos in the model. We fit experimental data for neutrino mass variances and mixing angle. Based on the range of data selection, the influences of different sensitive parameters on the results are analysed. The numerical analysis shows that many parameters have an effect on the neutrino transition moment, such as $g_X$, $M_2$, $\lambda_H$ and $g_{YX}$. For our numerical results, the order of magnitude of $\mu_{ij}^M/\mu_B$ is around $10^{-20}$ $\sim$ $10^{-19}$.

Achieving fairness across diverse clients in Federated Learning (FL) remains a significant challenge due to the heterogeneity of the data and the inaccessibility of sensitive attributes from clients' private datasets. This study addresses this issue by introducing \texttt{EquiFL}, a novel approach designed to enhance both local and global fairness in federated learning environments. \texttt{EquiFL} incorporates a fairness term into the local optimization objective, effectively balancing local performance and fairness. The proposed coordination mechanism also prevents bias from propagating across clients during the collaboration phase. Through extensive experiments across multiple benchmarks, we demonstrate that \texttt{EquiFL} not only strikes a better balance between accuracy and fairness locally at each client but also achieves global fairness. The results also indicate that \texttt{EquiFL} ensures uniform performance distribution among clients, thus contributing to performance fairness. Furthermore, we showcase the benefits of \texttt{EquiFL} in a real-world distributed dataset from a healthcare application, specifically in predicting the effects of treatments on patients across various hospital locations.

In the $U(1)_X$SSM, we delve into the flavor violation of $h \rightarrow bs$, where $h$ is identified with the SM-like Higgs boson discovered at the LHC. As the U(1) extension of the minimal supersymmetric standard model (MSSM), the U(1)XSSM has new super fields such as right-handed neutrinos and three Higgs singlets. We conduct a thorough analysis of the underlying mechanisms and parameter dependencies of $h \rightarrow bs$ in the $U(1)_X$SSM. In the $U(1)_X$SSM, we discover that the Br$(h \rightarrow bs)$ for the Higgs decay to $bs$ could significantly differ from the expectation in the standard model (SM), depending on the values of the new parameters introduced in the model. Our research not only contributes to a deeper understanding of Higgs physics within the $U(1)_X$SSM, but also provides valuable guidance for new physics (NP)., Comment: 24 pages,8 figures

It is well known that the muon magnetic dipole moment (MDM) has close relation with the new physics (NP) in the development of the Standard Model (SM). Combined with the Fermilab National Accelerator Laboratory (FNAL) and the Brookhaven National Laboratory (BNL) E821 result, the departure from the SM prediction is about 5.0 $\sigma$. We study the electroweak corrections from several type two-loop SUSY diagrams and the virtual SUSY particles include chargino, neutralino, scalar lepton and scalar neutrino. Based on the latest experimental constraints, we study the muon MDM under the next to the minimal supersymmetric extension of the SM with local B-L gauge symmetry (N-B-LSSM). The abundant numerical results verify that $\tan{\beta},~T_e,~M^2_L,~M^2_e,~M_{BB'}$ play an important role in muon MDM. $M^2_e,~\tan{\beta}$ and $T_e$ are sensitive parameters to muon MDM. From the data obtained in all the figures of the numerical results, most of the values of $a_{\mu}^{NBL}$ are in 2$\sigma$ interval, which can compensate the departure between the experiment data and the SM prediction.

Originating from semantic bugs, Entity-Inconsistency Bugs (EIBs) involve misuse of syntactically valid yet incorrect program entities, such as variable identifiers and function names, which often have security implications. Unlike straightforward syntactic vulnerabilities, EIBs are subtle and can remain undetected for years. Traditional detection methods, such as static analysis and dynamic testing, often fall short due to the versatile and context-dependent nature of EIBs. However, with advancements in Large Language Models (LLMs) like GPT-4, we believe LLM-powered automatic EIB detection becomes increasingly feasible through these models' semantics understanding abilities. This research first undertakes a systematic measurement of LLMs' capabilities in detecting EIBs, revealing that GPT-4, while promising, shows limited recall and precision that hinder its practical application. The primary problem lies in the model's tendency to focus on irrelevant code snippets devoid of EIBs. To address this, we introduce a novel, cascaded EIB detection system named WitheredLeaf, which leverages smaller, code-specific language models to filter out most negative cases and mitigate the problem, thereby significantly enhancing the overall precision and recall. We evaluated WitheredLeaf on 154 Python and C GitHub repositories, each with over 1,000 stars, identifying 123 new flaws, 45% of which can be exploited to disrupt the program's normal operations. Out of 69 submitted fixes, 27 have been successfully merged.

In real-world graph data, distribution shifts can manifest in various ways, such as the emergence of new categories and changes in the relative proportions of existing categories. It is often important to detect nodes of novel categories under such distribution shifts for safety or insight discovery purposes. We introduce a new approach, Recall-Constrained Optimization with Selective Link Prediction (RECO-SLIP), to detect nodes belonging to novel categories in attributed graphs under subpopulation shifts. By integrating a recall-constrained learning framework with a sample-efficient link prediction mechanism, RECO-SLIP addresses the dual challenges of resilience against subpopulation shifts and the effective exploitation of graph structure. Our extensive empirical evaluation across multiple graph datasets demonstrates the superior performance of RECO-SLIP over existing methods. The experimental code is available at https://github.com/hsinghuan/novel-node-category-detection., Comment: Accepted to ECML-PKDD 2024

Accurate traffic forecasting is a fundamental problem in intelligent transportation systems and learning long-range traffic representations with key information through spatiotemporal graph neural networks (STGNNs) is a basic assumption of current traffic flow prediction models. However, due to structural limitations, existing STGNNs can only utilize short-range traffic flow data; therefore, the models cannot adequately learn the complex trends and periodic features in traffic flow. Besides, it is challenging to extract the key temporal information from the long historical traffic series and obtain a compact representation. To solve the above problems, we propose a novel LSTTN (Long-Short Term Transformer-based Network) framework comprehensively considering the long- and short-term features in historical traffic flow. First, we employ a masked subseries Transformer to infer the content of masked subseries from a small portion of unmasked subseries and their temporal context in a pretraining manner, forcing the model to efficiently learn compressed and contextual subseries temporal representations from long historical series. Then, based on the learned representations, long-term trend is extracted by using stacked 1D dilated convolution layers, and periodic features are extracted by dynamic graph convolution layers. For the difficulties in making time-step level prediction, LSTTN adopts a short-term trend extractor to learn fine-grained short-term temporal features. Finally, LSTTN fuses the long-term trend, periodic features and short-term features to obtain the prediction results. Experiments on four real-world datasets show that in 60-minute-ahead long-term forecasting, the LSTTN model achieves a minimum improvement of 5.63\% and a maximum improvement of 16.78\% over baseline models. The source code is available at https://github.com/GeoX-Lab/LSTTN., Comment: 15 pages, 10 figures, 6 tables

As machine learning models in critical fields increasingly grapple with multimodal data, they face the dual challenges of handling a wide array of modalities, often incomplete due to missing elements, and the temporal irregularity and sparsity of collected samples. Successfully leveraging this complex data, while overcoming the scarcity of high-quality training samples, is key to improving these models' predictive performance. We introduce ``FuseMoE'', a mixture-of-experts framework incorporated with an innovative gating function. Designed to integrate a diverse number of modalities, FuseMoE is effective in managing scenarios with missing modalities and irregularly sampled data trajectories. Theoretically, our unique gating function contributes to enhanced convergence rates, leading to better performance in multiple downstream tasks. The practical utility of FuseMoE in the real world is validated by a diverse set of challenging prediction tasks., Comment: 47 pages, 9 tables, 12 figures

In the present study, we investigate the decays of the top quark $t\rightarrow c\gamma$, $t\rightarrow cg$, $t\rightarrow cZ$ and $t\rightarrow ch$. They are extremely rare processes in the Standard Model (SM). As the $U(1)$ extension of the minimal supersymmetric standard model (MSSM), the $U(1)_X$SSM features new superfields such as the right-handed neutrinos and three Higgs singlets. We analyze the effects of different sensitive parameters on the results and make reasonable theorecial predictions, which provides a useful reference for future experimental development. Considering the constraint of the updated experimental data, the numerical results show that the branching ratios of all four processes $t\rightarrow c\gamma,~cg,~cZ,~ch$ can reach the same order of magnitude as their experimental upper limits. Among them, $\tan\beta$ has the most obvious effect on each process and is the main parameter. $g_X$, $g_{YX}$, $\mu$, $M_2$, $\lambda_H$, $M_{U23}^2$ and $M_{Q23}^2$ are important parameters for the processes, and have effects on the numerical results., Comment: pages 25, figures 27

$U(1)_X$SSM is a non-universal Abelian extension of the Minimal Supersymmetric Standard Model (MSSM) and its local gauge group is extended to $SU(3)_C\times SU(2)_L \times U(1)_Y\times U(1)_X$. Based on the latest data of neutral meson mixing and experimental limitations, we investigate the process of $B^{0}-\bar{B^{0}}$ mixing in $U(1)_X$SSM. Using the effective Hamiltonian method, the Wilson coefficients and mass difference $\triangle m_{B}$ are derived. The abundant numerical results verify that $~v_S,~M^2_D,~\lambda_C,~{\mu},~M_2,~\tan{\beta},~g_{YX},~M_1$ and $~\lambda_H$ are sensitive parameters to the process of $B^{0}-\bar{B^{0}}$ mixing. With further measurement in the experiment, the parameter space of the $U(1)_X$SSM will be further constrained during the mixing process of $B^{0}-\bar{B^{0}}$.

We study lepton flavor violating (LFV) decays $B^0\rightarrow{{l_i}^{\pm}{l_j}^{\mp}}$ ($B^0\rightarrow e{\mu}$, $B^0\rightarrow e{\tau}$ and $B^0\rightarrow {\mu}{\tau}$) in the $U(1)_X$SSM (SSM is acronym for the supersymmetric standard model), which is the $U(1)_X$ extension of the minimal supersymmetric standard model (MSSM). The local gauge group of $U(1)_X$SSM is $SU(3)_C\times SU(2)_L \times U(1)_Y \times U(1)_X$. These processes ($B^0\rightarrow e{\mu}$, $B^0\rightarrow e{\tau}$ and $B^0\rightarrow {\mu}{\tau}$) are strictly forbidden in the standard model (SM), but these LFV decays are a signal of new physics (NP). We use the Mass Insertion Approximation (MIA) to find sensitive parameters that directly influence the result of the branching ratio of LFV decay $B_d\rightarrow{{l_i}^{\pm}{l_j}^{\mp}}$. Combined with the latest experimental results, we analyze the relationship between different sensitive parameters and the branching ratios of the three processes. According to the numerical analysis, these elements are very restricted from the non observation of CLFV.

Gravitational wave high-energy Electromagnetic Counterpart All-sky Monitor (GECAM) is a pair of microsatellites (i.e. GECAM-A and GECAM-B) dedicated to monitoring gamma-ray transients including gravitational waves high-energy electromagnetic counterparts, Gamma-ray Bursts, Soft Gamma-ray Repeaters, Solar Flares and Terrestrial Gamma-ray Flashes. Since launch in December 2020, GECAM-B has detected hundreds of astronomical and terrestrial events. For these bursts, localization is the key for burst identification and classification as well as follow-up observations in multi-wavelength. Here, we propose a Bayesian localization method with Poisson data with Gaussian background profile likelihood to localize GECAM bursts based on the burst counts distribution in detectors with different orientations. We demonstrate that this method can work well for all kinds of bursts, especially for extremely short ones. In addition, we propose a new method to estimate the systematic error of localization based on a confidence level test, which can overcome some problems of the existing method in literature. We validate this method by Monte Carlo simulations, and then apply it to a burst sample with accurate location and find that the mean value of the systematic error of GECAM-B localization is $\sim 2.5^{\circ}$. By considering this systematic error, we can obtain a reliable localization probability map for GECAM bursts. Our methods can be applied to other gamma-ray monitors., Comment: The paper has been accepted by Astrophysical Journal Supplement Series

Recent advances in Transformer architectures have empowered their empirical success in a variety of tasks across different domains. However, existing works mainly focus on predictive accuracy and computational cost, without considering other practical issues, such as robustness to contaminated samples. Recent work by Nguyen et al., (2022) has shown that the self-attention mechanism, which is the center of the Transformer architecture, can be viewed as a non-parametric estimator based on kernel density estimation (KDE). This motivates us to leverage a set of robust kernel density estimation methods for alleviating the issue of data contamination. Specifically, we introduce a series of self-attention mechanisms that can be incorporated into different Transformer architectures and discuss the special properties of each method. We then perform extensive empirical studies on language modeling and image classification tasks. Our methods demonstrate robust performance in multiple scenarios while maintaining competitive results on clean datasets., Comment: Accepted by NeurIPS 2023 as a poster; 23 pages, 5 figures, 11 tables

Conformal prediction is a simple and powerful tool that can quantify uncertainty without any distributional assumptions. Many existing methods only address the average coverage guarantee, which is not ideal compared to the stronger conditional coverage guarantee. Existing methods of approximating conditional coverage require additional models or time effort, which makes them not easy to scale. In this paper, we propose a modified non-conformity score by leveraging the local approximation of the conditional distribution using kernel density estimation. The modified score inherits the spirit of split conformal methods, which is simple and efficient and can scale to high dimensional settings. We also proposed a unified framework that brings together our method and several state-of-the-art. We perform extensive empirical evaluations: results measured by both average and conditional coverage confirm the advantage of our method., Comment: 21 pages, 7 figures, 8 tables

We propose a novel approach to the problem of clustering hierarchically aggregated time-series data, which has remained an understudied problem though it has several commercial applications. We first group time series at each aggregated level, while simultaneously leveraging local and global information. The proposed method can cluster hierarchical time series (HTS) with different lengths and structures. For common two-level hierarchies, we employ a combined objective for local and global clustering over spaces of discrete probability measures, using Wasserstein distance coupled with Soft-DTW divergence. For multi-level hierarchies, we present a bottom-up procedure that progressively leverages lower-level information for higher-level clustering. Our final goal is to improve both the accuracy and speed of forecasts for a larger number of HTS needed for a real-world application. To attain this goal, each time series is first assigned the forecast for its cluster representative, which can be considered as a "shrinkage prior" for the set of time series it represents. Then this base forecast can be quickly fine-tuned to adjust to the specifics of that time series. We empirically show that our method substantially improves performance in terms of both speed and accuracy for large-scale forecasting tasks involving much HTS., Comment: 17 pages, 3 figures, 4 tables

With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep neural network model in a FL setting proves to be a non-trivial task because of the complexities associated with the neural networks, such as varied architectures across clients, permutation invariance of the neurons, and presence of non-linear transformations in each layer. This work introduces a novel Federated Heterogeneous Neural Networks (FedHeNN) framework that allows each client to build a personalised model without enforcing a common architecture across clients. This allows each client to optimize with respect to local data and compute constraints, while still benefiting from the learnings of other (potentially more powerful) clients. The key idea of FedHeNN is to use the instance-level representations obtained from peer clients to guide the simultaneous training on each client. The extensive experimental results demonstrate that the FedHeNN framework is capable of learning better performing models on clients in both the settings of homogeneous and heterogeneous architectures across clients.

We introduce a framework to dynamically combine heterogeneous models called \texttt{DYCHEM}, which forecasts a set of time series that are related through an aggregation hierarchy. Different types of forecasting models can be employed as individual ``experts'' so that each model is tailored to the nature of the corresponding time series. \texttt{DYCHEM} learns hierarchical structures during the training stage to help generalize better across all the time series being modeled and also mitigates coherency issues that arise due to constraints imposed by the hierarchy. To improve the reliability of forecasts, we construct quantile estimations based on the point forecasts obtained from combined heterogeneous models. The resulting quantile forecasts are coherent and independent of the choice of forecasting models. We conduct a comprehensive evaluation of both point and quantile forecasts for hierarchical time series (HTS), including public data and user records from a large financial software company. In general, our method is robust, adaptive to datasets with different properties, and highly configurable and efficient for large-scale forecasting pipelines., Comment: ICDM 2022 Workshops, 10 pages, 6 figures

Realtime trigger and localization of bursts are the key functions of GECAM, which is an all-sky gamma-ray monitor launched in Dec 10, 2020. We developed a multifunctional trigger and localization software operating on the CPU of the GECAM electronic box (EBOX). This onboard software has the following features: high trigger efficiency for real celestial bursts with a suppression of false triggers caused by charged particle bursts and background fluctuation, dedicated localization algorithm optimized for short and long bursts respetively, short time latency of the trigger information which is downlinked throught the BeiDou satellite navigation System (BDS). This paper presents the detailed design and deveopment of this trigger and localization software system of GECAM, including the main functions, general design, workflow and algorithms, as well as the verification and demonstration of this software, including the on-ground trigger tests with simulated gamma-ray bursts made by a dedicated X-ray tube and the in-flight performance to real gamma-ray bursts and magnetar bursts., Comment: Draft, comments welcome

Text-style transfer aims to convert text given in one domain into another by paraphrasing the sentence or substituting the keywords without altering the content. By necessity, state-of-the-art methods have evolved to accommodate nonparallel training data, as it is frequently the case there are multiple data sources of unequal size, with a mixture of labeled and unlabeled sentences. Moreover, the inherent style defined within each source might be distinct. A generic bidirectional (e.g., formal $\Leftrightarrow$ informal) style transfer regardless of different groups may not generalize well to different applications. In this work, we developed a task adaptive meta-learning framework that can simultaneously perform a multi-pair text-style transfer using a single model. The proposed method can adaptively balance the difference of meta-knowledge across multiple tasks. Results show that our method leads to better quantitative performance as well as coherent style variations. Common challenges of unbalanced data and mismatched domains are handled well by this method., Comment: Meta Learning and Its Applications to Natural Language Processing, ACL 2021 Workshop

Many real-life applications involve simultaneously forecasting multiple time series that are hierarchically related via aggregation or disaggregation operations. For instance, commercial organizations often want to forecast inventories simultaneously at store, city, and state levels for resource planning purposes. In such applications, it is important that the forecasts, in addition to being reasonably accurate, are also consistent w.r.t one another. Although forecasting such hierarchical time series has been pursued by economists and data scientists, the current state-of-the-art models use strong assumptions, e.g., all forecasts being unbiased estimates, noise distribution being Gaussian. Besides, state-of-the-art models have not harnessed the power of modern nonlinear models, especially ones based on deep learning. In this paper, we propose using a flexible nonlinear model that optimizes quantile regression loss coupled with suitable regularization terms to maintain the consistency of forecasts across hierarchies. The theoretical framework introduced herein can be applied to any forecasting model with an underlying differentiable loss function. A proof of optimality of our proposed method is also provided. Simulation studies over a range of datasets highlight the efficacy of our approach.

In the heavy fermion materials, the characteristic energy scales of many exotic strongly correlated phenomena (Kondo effect, magnetic order, superconductivity, etc.) are at milli-electron-volt order, implying that the heavy fermion materials are surface sensitive. Here, we investigate the electronic structures for Si- and Ce-terminated surfaces of CeRh$_2$Si$_2$ by first-principles methods. Our research reveals three notable impacts of surface effects on electronic structures, which are consistent with recent angle-resolved photoemission spectroscopy (ARPES) experiments. Firstly, the relaxation of surface crystal structures changes the relative position of Fermi level, adjusts the dispersion of bands and enhances the Kondo resonance. Secondly, the decrease of the hybridization between the Ce-4$f$ and conduction electrons in the surface layer leads to a weaker Kondo resonance peak and the shift of spin-orbit bands. Thirdly, the variation of crystal electric field around surface Ce atoms affects the splitting of Kondo resonance peaks, and also pushes down the lower-Hubbard bands of surface 4$f$ electrons. Moreover, we find the characteristic of bulk's lower-Hubbard bands, which was overlooked in previous works. Our investigation suggests that these surface effects are potentially important and highlighted in the future researches on properties of strongly correlated materials., Comment: 9 pages (text), 5 figures (text), 1 table (text), 6 pages (SI), 6 figures (SI), and 1 table (SI)

Learning monotonic models with respect to a subset of the inputs is a desirable feature to effectively address the fairness, interpretability, and generalization issues in practice. Existing methods for learning monotonic neural networks either require specifically designed model structures to ensure monotonicity, which can be too restrictive/complicated, or enforce monotonicity by adjusting the learning process, which cannot provably guarantee the learned model is monotonic on selected features. In this work, we propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem.This provides a new general approach for learning monotonic neural networks with arbitrary model structures. Our method allows us to train neural networks with heuristic monotonicity regularizations, and we can gradually increase the regularization magnitude until the learned network is certified monotonic. Compared to prior works, our approach does not require human-designed constraints on the weight space and also yields more accurate approximation. Empirical studies on various datasets demonstrate the efficiency of our approach over the state-of-the-art methods, such as Deep Lattice Networks.

How can we find a subset of training samples that are most responsible for a specific prediction made by a complex black-box machine learning model? More generally, how can we explain the model's decisions to end-users in a transparent way? We propose a new model-agnostic algorithm to identify a minimal set of training samples that are indispensable for a given model's decision at a particular test point, i.e., the model's decision would have changed upon the removal of this subset from the training dataset. Our algorithm identifies such a set of "indispensable" samples iteratively by solving a constrained optimization problem. Further, we speed up the algorithm through efficient approximations and provide theoretical justification for its performance. To demonstrate the applicability and effectiveness of our approach, we apply it to a variety of tasks including data poisoning detection, training set debugging and understanding loan decisions. The results show that our algorithm is an effective and easy-to-comprehend tool that helps to better understand local model behavior, and therefore facilitates the adoption of machine learning in domains where such understanding is a requisite., Comment: International Joint Conference on Neural Networks (IJCNN), 2021

The effective interaction of downfolded low-energy models for electrons in solids can be obtained by integrating out the high energy bands away from the target band near the Fermi level. Here, we apply the constrained random-phase approximation (cRPA) and constrained functional renormalization group (cfRG), which can go beyond cRPA by including all one-loop diagrams, to calculate and compare the effective interactions of the three-band Emery model, which is often used to investigate cuprate high-temperature superconductors. At half band filling, we find that the effective interaction increases as the charge transfer energy ($\Delta_{dp}$) increases and similar behavior is obtained as a function of the interatomic 2$p$-3$d$ interaction ($U_{dp}$). However, the effective interaction is more sensitive to $\Delta_{dp}$ than $ U_{dp}$. For most of the parameter sets, the effective static interaction is overscreened in cRPA compared to cfRG. The low-energy models at half-filling are solved within dynamical mean-field theory (DMFT). The results show that despite the different static interactions, the systems with cRPA and cfRG interaction exhibit a Mott transition at similar values of $\Delta_{dp}$. We also investigate the effective interaction as a function of doping. The cfRG effective interaction decreases as the electron number increases and displays a trend opposite to that of cRPA. Antiscreening is observed for the hole-doped case. For all the cases studied, the near-cancellation of the direct particle-hole channel is observed. This indicates that at least for the downfolding of the onsite interaction terms, methods beyond cRPA may be required., Comment: 12 pages, 8 figures

Breaking the time-reversal symmetry of light is of great importance for fundamental physics and has attracted increasing interest in the study of non-reciprocal photonic devices. Here, we experimentally demonstrate a chiral cavity QED system with multiple atoms strongly coupled to a Fabry-Perot cavity. By polarizing the internal quantum state of the atoms, the time-reversal symmetry of the atom-cavity interaction is broken. The strongly coupled atom-cavity system can be described by non-reciprocal quasiparticles, i.e., the cavity polariton. When it works in the linear regime, the inherent nonreciprocity makes the system work as a single-photon-level optical isolator. Benefiting from the collective enhancement of multiple atoms, an isolation ratio exceeding 30~dB on the single-quanta level (~0.1 photon on average) is achieved. The validity of the non-reciprocal device under zero magnetic field and the reconfigurability of the isolation direction are also experimentally demonstrated. Moreover, when the cavity polariton works in the nonlinear regime, the quantum interference between polaritons with weak anharmonicity induces non-reciprocal nonclassical statistics of cavity transmission from coherent probe light., Comment: 10 pages, 6 figures

Optical nonreciprocity is important in photonic information processing to route the optical signal or prevent the reverse flow of noise. By adopting the strong nonlinearity associated with a few atoms in a strongly coupled cavity QED system and an asymmetric cavity configuration, we experimentally demonstrate the nonreciprocal transmission between two counterpropagating light fields with extremely low power. This nonreciprocity can even occur on a few-photon level due to the high optical nonlinearity of the system. The working power can be flexibly tuned by changing the effective number of atoms strongly coupled to the cavity. The idea and result can be applied to optical chips as optical diodes by using fiber-based cavity QED systems. Our work opens up new perspectives for realizing optical nonreciprocity on a few-photon level based on the nonlinearities of atoms strongly coupled to an optical cavity., Comment: 6 pages, 4 figures

Symbolic analysis of security exploits in smart contracts has demonstrated to be valuable for analyzing predefined vulnerability properties. While some symbolic tools perform complex analysis steps, they require a predetermined invocation depth to search vulnerable execution paths, and the search time increases with depth. The number of contracts on blockchains like Ethereum has increased 176 fold since December 2015. If these symbolic tools fail to analyze the increasingly large number of contracts in time, entire classes of exploits could cause irrevocable damage. In this paper, we aim to have safer smart contracts against emerging threats. We propose the approach of sequential learning of smart contract weaknesses using machine learning---long-short term memory (LSTM)---that allows us to be able to detect new attack trends relatively quickly, leading to safer smart contracts. Our experimental studies on 620,000 smart contracts prove that our model can easily scale to analyze a massive amount of contracts; that is, the LSTM maintains near constant analysis time as contracts increase in complexity. In addition, our approach achieves $99\%$ test accuracy and correctly analyzes contracts that were false positive (FP) errors made by a symbolic tool., Comment: 10 pages

The Mott insulator is the quintessential strongly correlated electronic state. We obtain complete insight into the physics of the two-dimensional Mott insulator by extending the slave-fermion (holon-doublon) description to finite temperatures. We first benchmark its predictions against state-of-the-art quantum Monte Carlo simulations, demonstrating quantitative agreement. Qualitatively, the short-ranged spin fluctuations both induce holon-doublon bound states and renormalize the charge sector to form the Hubbard bands. The Mott gap is understood as the charge gap renormalized downwards by these spin fluctuations. As temperature increases, the Mott gap closes before the charge gap, causing a pseudogap regime to appear naturally during the melting of the Mott insulator., Comment: 12 pages, 10 figures

We investigate the critical behavior and the duality property of the ferromagnetic $q$-state clock model on the square lattice based on the tensor-network formalism. From the entanglement spectra of local tensors defined in the original and dual lattices, we obtain the exact self-dual points for the model with $q \leq 5 $ and approximate self-dual points for $q \geq 6$. We calculate accurately the lower and upper critical temperatures for the six-state clock model from the fixed-point tensors determined using the higher-order tensor renormalization group method and compare with other numerical results., Comment: 5 pages, 4 figures

The ill-posed analytic continuation problem for Green's functions or self-energies can be done using the Pad\'e rational polynomial approximation. However, to extract accurate results from this approximation, high precision input data of the Matsubara Green's function are needed. The calculation of the Matsubara Green's function generally involves a Matsubara frequency summation which cannot be evaluated analytically. Numerical summation is requisite but it converges slowly with the increase of the Matsubara frequency. Here we show that this slow convergence problem can be significantly improved by utilizing the Pad\'e decomposition approach to replace the Matsubara frequency summation by a Pad\'e frequency summation, and high precision input data can be obtained to successfully perform the Pad\'e analytic continuation., Comment: 4 pages, 3 figures

We propose an eigen-operator scheme to study the lattice model of interacting spinless fermions at half filling and show that this model possesses a hidden form of reflection positivity in its Majorana fermion representation. Based on this observation, we prove rigourously that the ground state of this model is either unique or doubly degenerate if the lattice size $N$ is even, and is always doubly degenerate if $N$ is odd. This proof holds in all dimensions with arbitrary lattice structures.

The $^{1}S_{0}$ nucleonic superfluids are investigated within the relativistic mean-field model and Bardeen-Cooper-Schrieffer theory in hyperonic neutron stars. The $^{1}S_{0}$ pairing gaps of neutrons and protons are calculated based on the Reid soft-core interaction as the nucleon-nucleon interaction. We have studied particularly the influence of hyperons degrees of freedom on the $^{1}S_{0}$ nucleonic pairing gap in neutron star matter. It is found that the appearance of hyperons has little impact on baryonic density range and size for the $^{1}S_{0}$ neutronic pairing gap, the $^{1}S_{0}$ protonic pairing gap also decreases slightly in this region $\rho_B=0.0-0.393$ fm$^{-3}$. However, if baryonic density becomes greater than 0.393 fm${^{-3}}$, the $^{1}S_{0}$ protonic pairing gap obviously increases. In addition, the protonic superfluid range is obviously enlarged due to the presence of hyperons. In our results, the hyperons change the $^{1}S_{0}$ protonic pairing gap which must change the cooling properties of neutron stars., Comment: 8 pages, 4 figures