Search

Efficient sampling from un-normalized target distributions is pivotal in scientific computing and machine learning. While neural samplers have demonstrated potential with a special emphasis on sampling efficiency, existing neural implicit samplers still have issues such as poor mode covering behavior, unstable training dynamics, and sub-optimal performances. To tackle these issues, in this paper, we introduce Denoising Fisher Training (DFT), a novel training approach for neural implicit samplers with theoretical guarantees. We frame the training problem as an objective of minimizing the Fisher divergence by deriving a tractable yet equivalent loss function, which marks a unique theoretical contribution to assessing the intractable Fisher divergences. DFT is empirically validated across diverse sampling benchmarks, including two-dimensional synthetic distribution, Bayesian logistic regression, and high-dimensional energy-based models (EBMs). Notably, in experiments with high-dimensional EBMs, our best one-step DFT neural sampler achieves results on par with MCMC methods with up to 200 sampling steps, leading to a substantially greater efficiency over 100 times higher. This result not only demonstrates the superior performance of DFT in handling complex high-dimensional sampling but also sheds light on efficient sampling methodologies across broader applications.

Polarization measurements provide insight into the magnetic field, a critical aspect of the dynamics and emission properties around the compact object. In this paper, we present the polarized magnetic field of the Crab outer torus and the Vela arc utilizing Imaging X-ray Polarimetry Explorer observation data. The polarization angle (PA) measured for the Crab outer torus exhibits two monotonic evolutions along the azimuth angle, which are consistent with the normal line of the elliptical ring. There is a slight increase in PA along the azimuth angle for both the inner arc and the outer arc of the Vela nebula. The polarized magnetic vector along the outer torus of the Crab nebula shows the polarized magnetic field aligns with Crab outer torus structure. The PA variation along the Crab outer torus suggests a bulk flow speed of 0.8c. Meanwhile, the Vela nebula polarized magnetic field does not exactly align with the Vela arc structure. We noted that the Crab nebula possesses a polarized toroidal magnetic field, where as the Vela nebula exhibits an incomplete toroidal magnetic field., Comment: 8 pages, 5 figures

Sequential models like recurrent neural networks and transformers have become standard for probabilistic multivariate time series forecasting across various domains. Despite their strengths, they struggle with capturing high-dimensional distributions and cross-feature dependencies. Recent work explores generative approaches using diffusion or flow-based models, extending to time series imputation and forecasting. However, scalability remains a challenge. This work proposes a novel method combining recurrent neural networks' efficiency with diffusion models' probabilistic modeling, based on stochastic interpolants and conditional generation with control features, offering insights for future developments in this dynamic field.

We introduce DiffSteISR, a pioneering framework for reconstructing real-world stereo images. DiffSteISR utilizes the powerful prior knowledge embedded in pre-trained text-to-image model to efficiently recover the lost texture details in low-resolution stereo images. Specifically, DiffSteISR implements a time-aware stereo cross attention with temperature adapter (TASCATA) to guide the diffusion process, ensuring that the generated left and right views exhibit high texture consistency thereby reducing disparity error between the super-resolved images and the ground truth (GT) images. Additionally, a stereo omni attention control network (SOA ControlNet) is proposed to enhance the consistency of super-resolved images with GT images in the pixel, perceptual, and distribution space. Finally, DiffSteISR incorporates a stereo semantic extractor (SSE) to capture unique viewpoint soft semantic information and shared hard tag semantic information, thereby effectively improving the semantic accuracy and consistency of the generated left and right images. Extensive experimental results demonstrate that DiffSteISR accurately reconstructs natural and precise textures from low-resolution stereo images while maintaining a high consistency of semantic and texture between the left and right views.

In this paper, we propose a new experiment method to check contribution of chiral magnetic effect (CME). With experimental data of DIS involving transversely polarized proton, we have calculated the 3-D charge density inside the polarized proton, which is found to have a significant violation to spherical symmetry. Then we have calculated the property of electromagnetic field (E-M field) generated by a single transversely polarized proton ($p^{\uparrow}$). Based on them, the E-M field generated in small collision system $p^{\uparrow}+A$ are studied. We find that the orientation of this E-M field has a significant dependence on the polarization direction of the proton, and the correlator ($\Delta\gamma$ ) has also significant dependence on the angle between reaction plane and polarization direction. As background contribution are canceled comparing two collision geometry schemes, only contribution of CME is remained., Comment: 5 pages, 8 figures

Pictor A is a $\gamma$-ray emitting radio galaxy and has a bright hotspot called WHS, located $\sim$4 arcmin away from the nucleus. In this letter, we present an analysis of its 16-year Fermi-LAT data and report the first Imaging X-ray Polarimetry Explorer (IXPE) observation for this source. Our analysis of the Fermi-LAT observations reveals evidence of two components in the average $\gamma$-ray spectrum of Pictor A, exhibiting a statistically significant hardening from $\Gamma^1_{\gamma}=3.25\pm0.15$ to $\Gamma^2_{\gamma}=1.81\pm0.07$ at a break energy of $2.46\pm0.09$ GeV. The evident variability of $\gamma$-rays is observed in Pictor A. Interestingly, the variability is dominated by the component below the break energy, and the component above the break energy shows no variability. Furthermore, we find that a power-law function can adequately fit the spectrum during high-flux states, whereas a broken power-law is still required to explain the spectrum during low-flux state. We suggest that the low-energy component originates from the nucleus, while the high-energy component primarily stems from WHS. The broadband spectral energy distributions of both nucleus and WHS can be well represented by a simple leptonic model, with both $\gamma$-ray components attributed to the synchrotron-self-Compton (SSC) process. The analysis of IXPE data on the nucleus yields an upper limit to the polarization degree $\Pi_{\rm X}<$8.9\% in the 2--8 keV band, agreeing with its X-ray emission originating from SSC. However, $\Pi_{\rm X}=23.5\%\pm5.6\%$ is observed at a confidence level of $>99\%$ in the 5--7 keV band, and the possible physical origin of this narrow-energy-band polarization signal is discussed., Comment: 14 Pages, 4 Figures, 3 Tables, submitted, comments are welcome

Boyle and Turok's $CPT$-symmetric universe model posits that the universe was symmetric at the Big Bang, addressing numerous problems in both cosmology and the Standard Model of particle physics. We extend this model by incorporating the symmetric conditions at the end of the Universe, which impose constraints on the allowed perturbation modes. These constrained modes conflict with the integer wave vectors required by the global spatial geometry in a closed universe. To resolve this conflict, only specific values of curvature are permissible, and in particular the curvature density is constrained to be $\Omega_K \in \{-0.014, -0.009, -0.003, \ldots\}$, consistent with Planck observations., Comment: 5 pages, 5 figures

We present Crab X-ray polarization measurements using IXPE data with a total exposure of 300ks, three times more than the initial 2022 discovery paper. Polarization is detected in three times more pulsar phase bins, revealing an S-shaped $+40^\circ$ polarization angle sweep in the main pulse and ${>}1\sigma$ departures from the OPTIMA optical polarization in both pulses, suggesting different radiation mechanisms or sites for the polarized emission at the two wavebands. Our polarization map of the inner nebula reveals a toroidal magnetic field, as seen in prior IXPE analyses. Along the southern jet, the magnetic field orientation relative to the jet axis changes from perpendicular to parallel and the polarization degree decreases by ${\sim}6\%$. These observations may be explained by kink instabilities along the jet or a collision with a dense, jet-deflecting medium at the tip. Using spectropolarimetric analysis, we find asymmetric polarization in the four quadrants of the inner nebula, as expected for a toroidal field geometry, and a spatial correlation between polarization degree and photon index., Comment: 14 pages, 7 figures, 3 tables. Accepted by The Astrophysical Journal

This work introduces a novel and efficient Bayesian federated learning algorithm, namely, the Federated Averaging stochastic Hamiltonian Monte Carlo (FA-HMC), for parameter estimation and uncertainty quantification. We establish rigorous convergence guarantees of FA-HMC on non-iid distributed data sets, under the strong convexity and Hessian smoothness assumptions. Our analysis investigates the effects of parameter space dimension, noise on gradients and momentum, and the frequency of communication (between the central node and local nodes) on the convergence and communication costs of FA-HMC. Beyond that, we establish the tightness of our analysis by showing that the convergence rate cannot be improved even for continuous FA-HMC process. Moreover, extensive empirical studies demonstrate that FA-HMC outperforms the existing Federated Averaging-Langevin Monte Carlo (FA-LD) algorithm.

Given the remarkable achievements of existing learning-based malware detection in both academia and industry, this paper presents MalGuise, a practical black-box adversarial attack framework that evaluates the security risks of existing learning-based Windows malware detection systems under the black-box setting. MalGuise first employs a novel semantics-preserving transformation of call-based redividing to concurrently manipulate both nodes and edges of malware's control-flow graph, making it less noticeable. By employing a Monte-Carlo-tree-search-based optimization, MalGuise then searches for an optimized sequence of call-based redividing transformations to apply to the input Windows malware for evasions. Finally, it reconstructs the adversarial malware file based on the optimized transformation sequence while adhering to Windows executable format constraints, thereby maintaining the same semantics as the original. MalGuise is systematically evaluated against three state-of-the-art learning-based Windows malware detection systems under the black-box setting. Evaluation results demonstrate that MalGuise achieves a remarkably high attack success rate, mostly exceeding 95%, with over 91% of the generated adversarial malware files maintaining the same semantics. Furthermore, MalGuise achieves up to a 74.97% attack success rate against five anti-virus products, highlighting potential tangible security concerns to real-world users., Comment: This paper has been accepted by 33rd USENIX Security Symposium 2024

Despite advances in the paradigm of pre-training then fine-tuning in low-level vision tasks, significant challenges persist particularly regarding the increased size of pre-trained models such as memory usage and training time. Another concern often encountered is the unsatisfying results yielded when directly applying pre-trained single-image models to multi-image domain. In this paper, we propose a efficient method for transferring a pre-trained single-image super-resolution (SISR) transformer network to the domain of stereo image super-resolution (SteISR) through a parameter-efficient fine-tuning (PEFT) method. Specifically, we introduce the concept of stereo adapters and spatial adapters which are incorporated into the pre-trained SISR transformer network. Subsequently, the pre-trained SISR model is frozen, enabling us to fine-tune the adapters using stereo datasets along. By adopting this training method, we enhance the ability of the SISR model to accurately infer stereo images by 0.79dB on the Flickr1024 dataset. This method allows us to train only 4.8% of the original model parameters, achieving state-of-the-art performance on four commonly used SteISR benchmarks. Compared to the more complicated full fine-tuning approach, our method reduces training time and memory consumption by 57% and 15%, respectively.

Following the work of Capstick and Isgur [\href{https://doi.org/10.1103/PhysRevD.34.2809}{Phys.~Rev.~D~34,~2809~(1986)}], we systematically study the mass spectrum of the heavy baryons in the relativized quark potential model with chromodynamics. Besides the original Godfrey-Isgur (GI) model, we also adopt a modified GI model which replaces the linear confinement by a screened one. The two models give similar results in our work. All heavy baryons observed so far can be explained as three-quark states. In particular, we identify the $\Omega_{c}(3000)$/$\Omega_{b}(6316)$, $\Omega_{c}(3050)$/$\Omega_{b}(6330)$, $\Omega_{c}(3065)$/$\Omega_{b}(6340)$ and $\Omega_{c}(3090)$/$\Omega_{b}(6350)$ states as the $p_{\lambda}$ excitations with quantum numbers $1/2^{-}$, $3/2^{-}$, $3/2^{-}$ and $5/2^{-}$. The $\Omega_{c}(3120)$ is a $3/2^{-}$ state with the $p_{\rho}$ excitation, whose bottom partner is predicted to be $\Omega_{b}(6446/6457,3/2^{-})$. The higher state $\Omega_{c}(3188)$ is the $2s_{\lambda}$ excitation with quantum numbers $1/2^{+}$, and $\Omega_{c}(3327)$ is a $d_{\lambda}$ excitation with quantum numbers $3/2^{+}$ or $5/2^{+}$. In addition, the $\Lambda_{c}(2940)$ with quantum numbers $J^{P}=3/2^{-}$ could be explained as the $p_{\rho}$ excitation., Comment: 22 pages, 10 figures, revised version accepted by PRD

Replica exchange stochastic gradient Langevin dynamics (reSGLD) is an effective sampler for non-convex learning in large-scale datasets. However, the simulation may encounter stagnation issues when the high-temperature chain delves too deeply into the distribution tails. To tackle this issue, we propose reflected reSGLD (r2SGLD): an algorithm tailored for constrained non-convex exploration by utilizing reflection steps within a bounded domain. Theoretically, we observe that reducing the diameter of the domain enhances mixing rates, exhibiting a $\textit{quadratic}$ behavior. Empirically, we test its performance through extensive experiments, including identifying dynamical systems with physical constraints, simulations of constrained multi-modal distributions, and image classification tasks. The theoretical and empirical findings highlight the crucial role of constrained exploration in improving the simulation efficiency., Comment: 28 pages, 13 figures

Schr\"odinger bridge (SB) has emerged as the go-to method for optimizing transportation plans in diffusion models. However, SB requires estimating the intractable forward score functions, inevitably resulting in the costly implicit training loss based on simulated trajectories. To improve the scalability while preserving efficient transportation plans, we leverage variational inference to linearize the forward score functions (variational scores) of SB and restore simulation-free properties in training backward scores. We propose the variational Schr\"odinger diffusion model (VSDM), where the forward process is a multivariate diffusion and the variational scores are adaptively optimized for efficient transport. Theoretically, we use stochastic approximation to prove the convergence of the variational scores and show the convergence of the adaptively generated samples based on the optimal variational scores. Empirically, we test the algorithm in simulated examples and observe that VSDM is efficient in generations of anisotropic shapes and yields straighter sample trajectories compared to the single-variate diffusion. We also verify the scalability of the algorithm in real-world data and achieve competitive unconditional generation performance in CIFAR10 and conditional generation in time series modeling. Notably, VSDM no longer depends on warm-up initializations and has become tuning-friendly in training large-scale experiments., Comment: ICML 2024

This paper introduces a novel benchmark as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to upscale compressed images from 540p to 4K resolution (4x factor) in real-time on commercial GPUs. For this, we use a diverse test set containing a variety of 4K images ranging from digital art to gaming and photography. The images are compressed using the modern AVIF codec, instead of JPEG. All the proposed methods improve PSNR fidelity over Lanczos interpolation, and process images under 10ms. Out of the 160 participants, 25 teams submitted their code and models. The solutions present novel designs tailored for memory-efficiency and runtime on edge devices. This survey describes the best solutions for real-time SR of compressed high-resolution images., Comment: CVPR 2024, AI for Streaming (AIS) Workshop

This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results. The aim of this challenge is to discover an effective network design or solution capable of generating brighter, clearer, and visually appealing results when dealing with a variety of conditions, including ultra-high resolution (4K and beyond), non-uniform illumination, backlighting, extreme darkness, and night scenes. A notable total of 428 participants registered for the challenge, with 22 teams ultimately making valid submissions. This paper meticulously evaluates the state-of-the-art advancements in enhancing low-light images, reflecting the significant progress and creativity in this field., Comment: NTIRE 2024 Challenge Report

Multiple-choice questions (MCQs) are widely used in the evaluation of large language models (LLMs) due to their simplicity and efficiency. However, there are concerns about whether MCQs can truly measure LLM's capabilities, particularly in knowledge-intensive scenarios where long-form generation (LFG) answers are required. The misalignment between the task and the evaluation method demands a thoughtful analysis of MCQ's efficacy, which we undertake in this paper by evaluating nine LLMs on four question-answering (QA) datasets in two languages: Chinese and English. We identify a significant issue: LLMs exhibit an order sensitivity in bilingual MCQs, favoring answers located at specific positions, i.e., the first position. We further quantify the gap between MCQs and long-form generation questions (LFGQs) by comparing their direct outputs, token logits, and embeddings. Our results reveal a relatively low correlation between answers from MCQs and LFGQs for identical questions. Additionally, we propose two methods to quantify the consistency and confidence of LLMs' output, which can be generalized to other QA evaluation benchmarks. Notably, our analysis challenges the idea that the higher the consistency, the greater the accuracy. We also find MCQs to be less reliable than LFGQs in terms of expected calibration error. Finally, the misalignment between MCQs and LFGQs is not only reflected in the evaluation performance but also in the embedding space. Our code and models can be accessed at https://github.com/Meetyou-AI-Lab/Can-MC-Evaluate-LLMs., Comment: LREC-COLING 2024

With the comprehensive research conducted on various face analysis tasks, there is a growing interest among researchers to develop a unified approach to face perception. Existing methods mainly discuss unified representation and training, which lack task extensibility and application efficiency. To tackle this issue, we focus on the unified model structure, exploring a face generalist model. As an intuitive design, Naive Faceptor enables tasks with the same output shape and granularity to share the structural design of the standardized output head, achieving improved task extensibility. Furthermore, Faceptor is proposed to adopt a well-designed single-encoder dual-decoder architecture, allowing task-specific queries to represent new-coming semantics. This design enhances the unification of model structure while improving application efficiency in terms of storage overhead. Additionally, we introduce Layer-Attention into Faceptor, enabling the model to adaptively select features from optimal layers to perform the desired tasks. Through joint training on 13 face perception datasets, Faceptor achieves exceptional performance in facial landmark localization, face parsing, age estimation, expression recognition, binary attribute classification, and face recognition, achieving or surpassing specialized methods in most tasks. Our training framework can also be applied to auxiliary supervised learning, significantly improving performance in data-sparse tasks such as age estimation and expression recognition. The code and models will be made publicly available at https://github.com/lxq1000/Faceptor.

BlackJAX is a library implementing sampling and variational inference algorithms commonly used in Bayesian computation. It is designed for ease of use, speed, and modularity by taking a functional approach to the algorithms' implementation. BlackJAX is written in Python, using JAX to compile and run NumpPy-like samplers and variational methods on CPUs, GPUs, and TPUs. The library integrates well with probabilistic programming languages by working directly with the (un-normalized) target log density function. BlackJAX is intended as a collection of low-level, composable implementations of basic statistical 'atoms' that can be combined to perform well-defined Bayesian inference, but also provides high-level routines for ease of use. It is designed for users who need cutting-edge methods, researchers who want to create complex sampling methods, and people who want to learn how these work., Comment: Companion paper for the library https://github.com/blackjax-devs/blackjax Update: minor changes and updated the list of authors to include technical contributors

A recent study on the interpretability of real-valued convolutional neural networks (CNNs) {Stankovic_Mandic_2023CNN} has revealed a direct and physically meaningful link with the task of finding features in data through matched filters. However, applying this paradigm to illuminate the interpretability of complex-valued CNNs meets a formidable obstacle: the extension of matched filtering to a general class of noncircular complex-valued data, referred to here as the widely linear matched filter (WLMF), has been only implicit in the literature. To this end, to establish the interpretability of the operation of complex-valued CNNs, we introduce a general WLMF paradigm, provide its solution and undertake analysis of its performance. For rigor, our WLMF solution is derived without imposing any assumption on the probability density of noise. The theoretical advantages of the WLMF over its standard strictly linear counterpart (SLMF) are provided in terms of their output signal-to-noise-ratios (SNRs), with WLMF consistently exhibiting enhanced SNR. Moreover, the lower bound on the SNR gain of WLMF is derived, together with condition to attain this bound. This serves to revisit the convolution-activation-pooling chain in complex-valued CNNs through the lens of matched filtering, which reveals the potential of WLMFs to provide physical interpretability and enhance explainability of general complex-valued CNNs. Simulations demonstrate the agreement between the theoretical and numerical results.

Approximate Thompson sampling with Langevin Monte Carlo broadens its reach from Gaussian posterior sampling to encompass more general smooth posteriors. However, it still encounters scalability issues in high-dimensional problems when demanding high accuracy. To address this, we propose an approximate Thompson sampling strategy, utilizing underdamped Langevin Monte Carlo, where the latter is the go-to workhorse for simulations of high-dimensional posteriors. Based on the standard smoothness and log-concavity conditions, we study the accelerated posterior concentration and sampling using a specific potential function. This design improves the sample complexity for realizing logarithmic regrets from $\mathcal{\tilde O}(d)$ to $\mathcal{\tilde O}(\sqrt{d})$. The scalability and robustness of our algorithm are also empirically validated through synthetic experiments in high-dimensional bandit problems., Comment: 52 pages, 2 figures

Diffusion models have become the go-to method for large-scale generative models in real-world applications. These applications often involve data distributions confined within bounded domains, typically requiring ad-hoc thresholding techniques for boundary enforcement. Reflected diffusion models (Lou23) aim to enhance generalizability by generating the data distribution through a backward process governed by reflected Brownian motion. However, reflected diffusion models may not easily adapt to diverse domains without the derivation of proper diffeomorphic mappings and do not guarantee optimal transport properties. To overcome these limitations, we introduce the Reflected Schrodinger Bridge algorithm: an entropy-regularized optimal transport approach tailored for generating data within diverse bounded domains. We derive elegant reflected forward-backward stochastic differential equations with Neumann and Robin boundary conditions, extend divergence-based likelihood training to bounded domains, and explore natural connections to entropic optimal transport for the study of approximate linear convergence - a valuable insight for practical training. Our algorithm yields robust generative modeling in diverse domains, and its scalability is demonstrated in real-world constrained generative modeling through standard image benchmarks.

Accurately predicting line loss rates is vital for effective line loss management in distribution networks, especially over short-term multi-horizons ranging from one hour to one week. In this study, we propose Attention-GCN-LSTM, a novel method that combines Graph Convolutional Networks (GCN), Long Short-Term Memory (LSTM), and a three-level attention mechanism to address this challenge. By capturing spatial and temporal dependencies, our model enables accurate forecasting of line loss rates across multiple horizons. Through comprehensive evaluation using real-world data from 10KV feeders, our Attention-GCN-LSTM model consistently outperforms existing algorithms, exhibiting superior performance in terms of prediction accuracy and multi-horizon forecasting. This model holds significant promise for enhancing line loss management in distribution networks.

Real-world stereo image super-resolution has a significant influence on enhancing the performance of computer vision systems. Although existing methods for single-image super-resolution can be applied to improve stereo images, these methods often introduce notable modifications to the inherent disparity, resulting in a loss in the consistency of disparity between the original and the enhanced stereo images. To overcome this limitation, this paper proposes a novel approach that integrates a implicit stereo information discriminator and a hybrid degradation model. This combination ensures effective enhancement while preserving disparity consistency. The proposed method bridges the gap between the complex degradations in real-world stereo domain and the simpler degradations in real-world single-image super-resolution domain. Our results demonstrate impressive performance on synthetic and real datasets, enhancing visual perception while maintaining disparity consistency. The complete code is available at the following \href{https://github.com/fzuzyb/SCGLANet}{link}.

In this paper, we present a full spatially resolved polarization map for the Vela Pulsar Wind Nebula (PWN) observed by IXPE. By employing effective background discrimination techniques, our results show a remarkably high degree of local polarization in the outskirt region, exceeding 60% (55%) with a probability of 95% (99%), which approaches the upper limit predicted by the synchrotron emission mechanism. The high degree of polarization suggests that the turbulent magnetic energy is at most 33% of the ordered one. In addition, the X-ray polarization map exhibits a toroidal magnetic field pattern that is consistent with the field revealed by radio observations across the entire nebula. This consistency reveals that the observed X-ray and radio emissions are radiated by electrons from the same magnetic field. Different from the Crab PWN, the consistency observed in the Vela PWN may be attributed to the interaction between the reverse shock of supernova blast wave and the PWN, which leads to a displacement between the synchrotron-cooled nebula and the fresh nebula close to the pulsar. These findings deepen our understanding of the structure and evolution of the Vela PWN, and the magnetohydrodynamic interaction in PWNe., Comment: Accepted by ApJL

The Imaging X-ray Polarimetry Explorer (IXPE) measured with high significance the X-ray polarization of the brightest Z-source Scorpius X-1, resulting in the nominal 2-8 keV energy band in a polarization degree of 1.0(0.2)% and a polarization angle of 8(6){\deg} at 90% of confidence level. This observation was strictly simultaneous with observations performed by NICER, NuSTAR, and Insight-HXMT, which allowed for a precise characterization of its broad-band spectrum from soft to hard X-rays. The source has been observed mainly in its soft state, with short periods of flaring. We also observed low-frequency quasi-periodic oscillations. From a spectro-polarimetric analysis, we associate a polarization to the accretion disk at <3.2% at 90% of confidence level, compatible with expectations for an electron-scattering dominated optically thick atmosphere at the Sco X-1 inclination of 44{\deg}; for the higher-energy Comptonized component, we obtain a polarization of 1.3(0.4)%, in agreement with expectations for a slab of Thomson optical depth of ~7 and an electron temperature of ~3 keV. A polarization rotation with respect to previous observations by OSO-8 and PolarLight, and also with respect to the radio-jet position angle, is observed. This result may indicate a variation of the polarization with the source state that can be related to relativistic precession or to a change in the corona geometry with the accretion flow.

The rise of artificial intelligence (AI) hinges on the efficient training of modern deep neural networks (DNNs) for non-convex optimization and uncertainty quantification, which boils down to a non-convex Bayesian learning problem. A standard tool to handle the problem is Langevin Monte Carlo, which proposes to approximate the posterior distribution with theoretical guarantees. In this thesis, we start with the replica exchange Langevin Monte Carlo (also known as parallel tempering), which proposes appropriate swaps between exploration and exploitation to achieve accelerations. However, the na\"ive extension of swaps to big data problems leads to a large bias, and bias-corrected swaps are required. Such a mechanism leads to few effective swaps and insignificant accelerations. To alleviate this issue, we first propose a control variates method to reduce the variance of noisy energy estimators and show a potential to accelerate the exponential convergence. We also present the population-chain replica exchange based on non-reversibility and obtain an optimal round-trip rate for deep learning. In the second part of the thesis, we study scalable dynamic importance sampling algorithms based on stochastic approximation. Traditional dynamic importance sampling algorithms have achieved success, however, the lack of scalability has greatly limited their extensions to big data. To handle this scalability issue, we resolve the vanishing gradient problem and propose two dynamic importance sampling algorithms. Theoretically, we establish the stability condition for the underlying ordinary differential equation (ODE) system and guarantee the asymptotic convergence of the latent variable to the desired fixed point. Interestingly, such a result still holds given non-convex energy landscapes., Comment: Ph.D. Thesis

The Schr\"odinger bridge problem (SBP) is gaining increasing attention in generative modeling and showing promising potential even in comparison with the score-based generative models (SGMs). SBP can be interpreted as an entropy-regularized optimal transport problem, which conducts projections onto every other marginal alternatingly. However, in practice, only approximated projections are accessible and their convergence is not well understood. To fill this gap, we present a first convergence analysis of the Schr\"odinger bridge algorithm based on approximated projections. As for its practical applications, we apply SBP to probabilistic time series imputation by generating missing values conditioned on observed data. We show that optimizing the transport cost improves the performance and the proposed algorithm achieves the state-of-the-art result in healthcare and environmental data while exhibiting the advantage of exploring both temporal and feature patterns in probabilistic time series imputation., Comment: Accepted by ICML 2023

In this paper, we prove that the Cauchy problem for a generalized Camassa-Holm equation with higher-order nonlinearity is ill-posed in the critical Besov space $B^1_{\infty,1}(\R)$. It is shown in (J. Differ. Equ., 327:127-144,2022) that the Camassa-Holm equation is ill-posed in $B^1_{\infty,1}(\R)$, here we turn our attention to a higher-order nonlinear generalization of Camassa-Holm equation proposed by Hakkaev and Kirchev (Commun Partial Differ Equ 30:761-781,2005). With newly constructed initial data, we get the norm inflation in the critical space $B^1_{\infty,1}(\R)$ which leads to ill-posedness., Comment: 13 pages

Parallel tempering (PT), also known as replica exchange, is the go-to workhorse for simulations of multi-modal distributions. The key to the success of PT is to adopt efficient swap schemes. The popular deterministic even-odd (DEO) scheme exploits the non-reversibility property and has successfully reduced the communication cost from $O(P^2)$ to $O(P)$ given sufficiently many $P$ chains. However, such an innovation largely disappears in big data due to the limited chains and few bias-corrected swaps. To handle this issue, we generalize the DEO scheme to promote non-reversibility and propose a few solutions to tackle the underlying bias caused by the geometric stopping time. Notably, in big data scenarios, we obtain an appealing communication cost $O(P\log P)$ based on the optimal window size. In addition, we also adopt stochastic gradient descent (SGD) with large and constant learning rates as exploration kernels. Such a user-friendly nature enables us to conduct approximation tasks for complex posteriors without much tuning costs., Comment: Accepted by AAAI 2023

In the presence of energetic particles (EPs), the Long-Lived Mode (LLM) frequency multiplication with n = 1,2,3 or higher is often observed on HL-2A, where n is the toroidal mode number. Hybrid kinetic-MHD model simulations of the energetic particle (EP) driven kink/fishbone modes on a static HL-2A-like tokamak using NIMROD code find that, when the background plasma pressure is relatively high, and the EP pressure and the beam energy are relatively low, the mode frequency increases almost linearly with EP pressure, and the frequency is proportional to n ("frequency multiplication"), even in absence of any equilibrium plasma rotation. In addition, the frequency multiplication persists as the safety factor at magnetic axis q0 varies. In absence of EPs, the growth rate of the 1/1 mode is the largest; however, as the EP pressure increases, the growth rate of 2/2 modes or 3/3 modes becomes dominant, suggesting that the higher-n modes are more vulnerable to EPs. These results may shed light on the understanding about the toroidal mode number dependence of kink/fishbone modes in the advanced scenarios of tokamaks with weak or reversed central magnetic shear., Comment: 28 pages, 13 figures

Multi-collinearity is a wide-spread phenomenon in modern statistical applications and when ignored, can negatively impact model selection and statistical inference. Classic tools and measures that were developed for "$n>p$" data are not applicable nor interpretable in the high-dimensional regime. Here we propose 1) new individualized measures that can be used to visualize patterns of multi-collinearity, and subsequently 2) global measures to assess the overall burden of multi-collinearity without limiting the observed data dimensions. We applied these measures to genomic applications to investigate patterns of multi-collinearity in genetic variations across individuals with diverse ancestral backgrounds. The measures were able to visually distinguish genomic regions of excessive multi-collinearity and contrast the level of multi-collinearity between different continental populations.

We propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, an embarrassingly parallel multiple-chain contour stochastic gradient Langevin dynamics (CSGLD) sampler with efficient interactions. We show that ICSGLD can be theoretically more efficient than a single-chain CSGLD with an equivalent computational budget. We also present a novel random-field function, which facilitates the estimation of self-adapting parameters in big data and obtains free mode explorations. Empirically, we compare the proposed algorithm with popular benchmark methods for posterior sampling. The numerical results show a great potential of ICSGLD for large-scale uncertainty estimation tasks., Comment: ICLR 2022

Malware has been one of the most damaging threats to computers that span across multiple operating systems and various file formats. To defend against ever-increasing and ever-evolving malware, tremendous efforts have been made to propose a variety of malware detection that attempt to effectively and efficiently detect malware so as to mitigate possible damages as early as possible. Recent studies have shown that, on the one hand, existing ML and DL techniques enable superior solutions in detecting newly emerging and previously unseen malware. However, on the other hand, ML and DL models are inherently vulnerable to adversarial attacks in the form of adversarial examples. In this paper, we focus on malware with the file format of portable executable (PE) in the family of Windows operating systems, namely Windows PE malware, as a representative case to study the adversarial attack methods in such adversarial settings. To be specific, we start by first outlining the general learning framework of Windows PE malware detection based on ML/DL and subsequently highlighting three unique challenges of performing adversarial attacks in the context of Windows PE malware. Then, we conduct a comprehensive and systematic review to categorize the state-of-the-art adversarial attacks against PE malware detection, as well as corresponding defenses to increase the robustness of Windows PE malware detection. Finally, we conclude the paper by first presenting other related attacks against Windows PE malware detection beyond the adversarial attacks and then shedding light on future research directions and opportunities. In addition, a curated resource list of adversarial attacks and defenses for Windows PE malware detection is also available at https://github.com/ryderling/adversarial-attacks-and-defenses-for-windows-pe-malware-detection., Comment: Accepted by ELSEVIER Computers & Security (COSE)

We propose a federated averaging Langevin algorithm (FA-LD) for uncertainty quantification and mean predictions with distributed clients. In particular, we generalize beyond normal posterior distributions and consider a general class of models. We develop theoretical guarantees for FA-LD for strongly log-concave distributions with non-i.i.d data and study how the injected noise and the stochastic-gradient noise, the heterogeneity of data, and the varying learning rates affect the convergence. Such an analysis sheds light on the optimal choice of local updates to minimize communication costs. Important to our approach is that the communication efficiency does not deteriorate with the injected noise in the Langevin algorithms. In addition, we examine in our FA-LD algorithm both independent and correlated noise used over different clients. We observe there is a trade-off between the pairs among communication, accuracy, and data privacy. As local devices may become inactive in federated networks, we also show convergence results based on different averaging schemes where only partial device updates are available. In such a case, we discover an additional bias that does not decay to zero., Comment: A polished proof without the federated formulation of Langevin diffusion to avoid confusion

In the framework of an extended chromomagnetic model, we systematically study the mass spectrum of the $S$-wave $qQ\bar{Q}\bar{Q}$ tetraquarks. Their mass spectra are mainly determined by the color interaction. For the $qc\bar{c}\bar{c}$, $qb\bar{c}\bar{c}$ and $qb\bar{b}\bar{b}$ tetraquarks, the color interaction favors the color-sextet $\ket{(qQ)^{6_{c}}(\bar{Q}\bar{Q})^{\bar{6}_{c}}}$ configuration over the color-triplet $\ket{(qQ)^{\bar{3}_{c}}(\bar{Q}\bar{Q})^{3_{c}}}$ one. But for the $qc\bar{b}\bar{b}$ tetraquarks, the color-triplet configuration is favored. We find no stable states which lie below the thresholds of two pseudoscalar mesons. The lowest axial-vector states with the $qQ\bar{b}\bar{b}$ flavor configuration may be narrow. They lie just above the thresholds of two pseudoscalar mesons, but cannot decay into these channels because of the conservation of the angular momentum and parity., Comment: 25 pages, 6 figures. revised version accepted by PRD

Recent experiments show that $\Delta\gamma$, an observable designed for detecting the chiral magnetic effect (CME), in small collision system $p+A$ is similar with that in heavy ion collision $A+A$. This brings a challenge to the existence of CME because it is believed that there is no azimuthal correlation between the orientation of the magnetic field ($\Phi_B$) and the participant plane ($\Phi_2$) in small collision system. In this work, we introduce three charge density models to describe the inner charge distributions of proton and neutron, and calculate the electric and magnetic fields produced in small $p+A$ collisions at both RHIC and LHC energies. Our results show that the contribution of the single projectile proton to the magnetic field is the main source after average over all participants. The azimuthal correlation between $\Phi_B$ and $\Phi_2$ is small but not vanished. And due to the huge fluctuation of fields strength, the magnetic-field contribution to $\Delta\gamma$ could be large., Comment: 14 pages, 13 figures

Using an extended chromomagnetic model, we perform a systematic study of the masses of the doubly heavy tetraquarks. We find that the ground states of the doubly heavy tetraquarks are dominated by color-triplet $\ket{(qq)^{\bar{3}_{c}}(\bar{Q}\bar{Q})^{3_{c}}}$ configuration, which is opposite to that of the fully heavy tetraquarks. The combined results suggest that the color-triplet configuration becomes more important when the mass difference between the quarks and antiquarks increases. We find three stable states which lie below the thresholds of two pseudoscalar mesons. They are the $IJ^{P}=01^{+}$ $nn\bar{b}\bar{b}$ tetraquark, the $IJ^{P}=00^{+}$ $nn\bar{c}\bar{b}$ tetraquark and the $J^{P}=1^{+}$ $ns\bar{b}\bar{b}$ tetraquark., Comment: 30 pages, 6 figures; to be published in CPC

Three-dimensional (3D) artificial metacrystals host rich topological phases, such as Weyl points, nodal rings and 3D photonic topological insulators. These topological states enable a wide range of applications, including 3D robust waveguide, one-way fiber and negative refraction of surface wave. However, these carefully designed metacrystals are usually very complex, hindering their extension to nanoscale photonic systems. Here, we theoretically proposed and experimentally realized an ideal nodal ring in visible region using a simple 1D photonic crystal. The pi Berry phase around the ring is manifested by a 2pi reflection phase's winding and the resultant drumhead surface states. By breaking the inversion symmetry, the nodal ring can be gapped and the pi-Berry phase would diffuse into a toroidal shaped Berry flux, resulting in photonic ridge states (the 3D extension of quantum valley Hall states). Our results provide a simple and feasible platform for exploring 3D topological physics and their potential applications in nanophotonics., Comment: 15 pages, 5 figures

Stochastic sparse linear bandits offer a practical model for high-dimensional online decision-making problems and have a rich information-regret structure. In this work we explore the use of information-directed sampling (IDS), which naturally balances the information-regret trade-off. We develop a class of information-theoretic Bayesian regret bounds that nearly match existing lower bounds on a variety of problem instances, demonstrating the adaptivity of IDS. To efficiently implement sparse IDS, we propose an empirical Bayesian approach for sparse posterior sampling using a spike-and-slab Gaussian-Laplace prior. Numerical results demonstrate significant regret reductions by sparse IDS relative to several baselines.

With updated experimental data and improved theoretical calculations, several significant deviations are being observed between the Standard Model predictions and the experimental measurements of the branching ratios of $\bar{B}_{(s)}^0\to D_{(s)}^{(*)+} L^-$ decays, where $L$ is a light meson from the set $\{\pi,\rho,K^{(\ast)}\}$. Especially for the two channels $\bar{B}^0\to D^{+}K^-$ and $\bar{B}_{s}^0\to D_{s}^{+}\pi^-$, both of which are free of the weak annihilation contribution, the deviations observed can even reach 4-5$\sigma$. Here we exploit possible new-physics effects in these class-I non-leptonic $B$-meson decays within the framework of QCD factorization. Firstly, we perform a model-independent analysis of the effects from twenty linearly independent four-quark operators that can contribute, either directly or through operator mixing, to the quark-level $b\to c\bar{u} d(s)$ transitions. It is found that, under the combined constraints from the current experimental data, the deviations observed could be well explained at the $1\sigma$ level by the new-physics four-quark operators with $\gamma^{\mu}(1-\gamma_5)\otimes\gamma_{\mu} (1-\gamma_5)$ structure, and also at the $2\sigma$ level by the operators with $(1+\gamma_5)\otimes(1-\gamma_5)$ and $(1+\gamma_5)\otimes(1+\gamma_5)$ structures. However, the new-physics four-quark operators with other Dirac structures fail to provide a consistent interpretation, even at the $2\sigma$ level. Then, as two specific examples of model-dependent considerations, we discuss the case where the new-physics four-quark operators are generated by either a colorless charged gauge boson or a colorless charged scalar, with their masses fixed both at the $1$~TeV. Constraints on the effective coefficients describing the couplings of these mediators to the relevant quarks are obtained by fitting to the current experimental data., Comment: 49 pages, 21 figures, and 7 tables; v2: typos corrected, parts of the plots updated; v3: estimation of annihilation contributions and more discussions added, final version published in JHEP

Although the widely linear least mean square error (WLMMSE) receiver has been an appealing option for multiple-input-multiple-output (MIMO) wireless systems, a statistical understanding on its pose-detection signal-to-interference-plus-noise ratio (SINR) in detail is still missing. To this end, we consider a WLMMSE MIMO transmission system with rectilinear or quasi-rectilinear (QR) signals over the uncorrelated Rayleigh fading channel and investigate the statistical properties of its SINR for an arbitrary antenna configuration with $N_t$ transmit antennas and $N_r$ receive ones. We first derive an analytic probability density function (PDF) of the SINR in terms of the confluent hypergeometric function of the second kind, for WLMMSE MIMO systems with an arbitrary $N_r$ and $N_t=2, 3$. For a more general case in practice, i.e., $N_t>3$, we resort to the moment generating function to obtain an approximate but closed form PDF under some mild conditions, which, as expected, is more Gaussian-like as $2N_r-N_t$ increases. The so-derived PDFs are able to provide key insights into the WLMMSE MIMO receiver in terms of the outage probability, the symbol error rate, and the diversity gain, all presented in closed form. In particular, its diversity gain and the gain improvement over the conventional LMMSE one are explicitly quantified as $N_r-(N_t-1)/2$ and $(N_t-1)/2$, respectively. Finally, Monte Carlo simulations support the analysis.

We propose an adaptively weighted stochastic gradient Langevin dynamics algorithm (SGLD), so-called contour stochastic gradient Langevin dynamics (CSGLD), for Bayesian learning in big data statistics. The proposed algorithm is essentially a \emph{scalable dynamic importance sampler}, which automatically \emph{flattens} the target distribution such that the simulation for a multi-modal distribution can be greatly facilitated. Theoretically, we prove a stability condition and establish the asymptotic convergence of the self-adapting parameter to a {\it unique fixed-point}, regardless of the non-convexity of the original energy function; we also present an error analysis for the weighted averaging estimators. Empirically, the CSGLD algorithm is tested on multiple benchmark datasets including CIFAR10 and CIFAR100. The numerical results indicate its superiority to avoid the local trap problem in training deep neural networks., Comment: Accepted by NeurIPS 2020

In this work, we systematically study the mass spectrum of the fully heavy tetraquark in an extended chromomagnetic model, which includes both color and chromomagnetic interactions. Numerical results indicate that the energy level is mainly determined by the color interaction, which favors the color-sextet $\ket{(QQ)^{6_{c}}(\bar{Q}\bar{Q})^{\bar{6}_{c}}}$ configuration over the color-triplet $\ket{(QQ)^{\bar{3}_{c}}(\bar{Q}\bar{Q})^{3_{c}}}$ one. The chromomagnetic interaction mixes the two color configurations and gives small splitting. The ground state is always dominated by the color-sextet configuration. We find no stable state below the lowest heavy quarkonium pair thresholds. Most states may be wide since they have at least one $S$-wave decay channel into two $S$-wave mesons. One possible narrow state is the $1^{+}$ $bb\bar{b}\bar{c}$ state with a mass $15719.1~\text{MeV}$. It is just above the $\eta_{b}\bar{B}_{c}$ threshold. But this channel is forbidden because of the conservation of the angular momentum and parity., Comment: 17 pages, 4 figures

Bayesian approaches have been successfully integrated into training deep neural networks. One popular family is stochastic gradient Markov chain Monte Carlo methods (SG-MCMC), which have gained increasing interest due to their scalability to handle large datasets and the ability to avoid overfitting. Although standard SG-MCMC methods have shown great performance in a variety of problems, they may be inefficient when the random variables in the target posterior densities have scale differences or are highly correlated. In this work, we present an adaptive Hessian approximated stochastic gradient MCMC method to incorporate local geometric information while sampling from the posterior. The idea is to apply stochastic approximation to sequentially update a preconditioning matrix at each iteration. The preconditioner possesses second-order information and can guide the random walk of a sampler efficiently. Instead of computing and saving the full Hessian of the log posterior, we use limited memory of the sample and their stochastic gradients to approximate the inverse Hessian-vector multiplication in the updating formula. Moreover, by smoothly optimizing the preconditioning matrix, our proposed algorithm can asymptotically converge to the target distribution with a controllable bias under mild conditions. To reduce the training and testing computational burden, we adopt a magnitude-based weight pruning method to enforce the sparsity of the network. Our method is user-friendly and is scalable to standard SG-MCMC updating rules by implementing an additional preconditioner. The sparse approximation of inverse Hessian alleviates storage and computational complexities for large dimensional models. The bias introduced by stochastic approximation is controllable and can be analyzed theoretically. Numerical experiments are performed on several problems.

Replica exchange stochastic gradient Langevin dynamics (reSGLD) has shown promise in accelerating the convergence in non-convex learning; however, an excessively large correction for avoiding biases from noisy energy estimators has limited the potential of the acceleration. To address this issue, we study the variance reduction for noisy energy estimators, which promotes much more effective swaps. Theoretically, we provide a non-asymptotic analysis on the exponential acceleration for the underlying continuous-time Markov jump process; moreover, we consider a generalized Girsanov theorem which includes the change of Poisson measure to overcome the crude discretization based on the Gr\"{o}wall's inequality and yields a much tighter error in the 2-Wasserstein ($\mathcal{W}_2$) distance. Numerically, we conduct extensive experiments and obtain the state-of-the-art results in optimization and uncertainty estimates for synthetic experiments and image data., Comment: Accepted by ICLR 2021

Replica exchange Monte Carlo (reMC), also known as parallel tempering, is an important technique for accelerating the convergence of the conventional Markov Chain Monte Carlo (MCMC) algorithms. However, such a method requires the evaluation of the energy function based on the full dataset and is not scalable to big data. The na\"ive implementation of reMC in mini-batch settings introduces large biases, which cannot be directly extended to the stochastic gradient MCMC (SGMCMC), the standard sampling method for simulating from deep neural networks (DNNs). In this paper, we propose an adaptive replica exchange SGMCMC (reSGMCMC) to automatically correct the bias and study the corresponding properties. The analysis implies an acceleration-accuracy trade-off in the numerical discretization of a Markov jump process in a stochastic environment. Empirically, we test the algorithm through extensive experiments on various setups and obtain the state-of-the-art results on CIFAR10, CIFAR100, and SVHN in both supervised learning and semi-supervised learning tasks., Comment: Accepted by ICML 2020

In this work, we propose a Bayesian type sparse deep learning algorithm. The algorithm utilizes a set of spike-and-slab priors for the parameters in the deep neural network. The hierarchical Bayesian mixture will be trained using an adaptive empirical method. That is, one will alternatively sample from the posterior using preconditioned stochastic gradient Langevin Dynamics (PSGLD), and optimize the latent variables via stochastic approximation. The sparsity of the network is achieved while optimizing the hyperparameters with adaptive searching and penalizing. A popular SG-MCMC approach is Stochastic gradient Langevin dynamics (SGLD). However, considering the complex geometry in the model parameter space in non-convex learning, updating parameters using a universal step size in each component as in SGLD may cause slow mixing. To address this issue, we apply a computationally manageable preconditioner in the updating rule, which provides a step-size parameter to adapt to local geometric properties. Moreover, by smoothly optimizing the hyperparameter in the preconditioning matrix, our proposed algorithm ensures a decreasing bias, which is introduced by ignoring the correction term in preconditioned SGLD. According to the existing theoretical framework, we show that the proposed algorithm can asymptotically converge to the correct distribution with a controllable bias under mild conditions. Numerical tests are performed on both synthetic regression problems and learning the solutions of elliptic PDE, which demonstrate the accuracy and efficiency of present work.

This paper reviews the NTIRE 2020 challenge on real world super-resolution. It focuses on the participating methods and final results. The challenge addresses the real world setting, where paired true high and low-resolution images are unavailable. For training, only one set of source input images is therefore provided along with a set of unpaired high-quality target images. In Track 1: Image Processing artifacts, the aim is to super-resolve images with synthetically generated image processing artifacts. This allows for quantitative benchmarking of the approaches \wrt a ground-truth image. In Track 2: Smartphone Images, real low-quality smart phone images have to be super-resolved. In both tracks, the ultimate goal is to achieve the best perceptual quality, evaluated using a human study. This is the second challenge on the subject, following AIM 2019, targeting to advance the state-of-the-art in super-resolution. To measure the performance we use the benchmark protocol from AIM 2019. In total 22 teams competed in the final testing phase, demonstrating new and innovative solutions to the problem.

In this paper, we investigate the dependence on initial data of solutions to higher dimensional Camassa-Holm equations. We show that the data-to-solution map is not uniformly continuous dependence in Besov spaces $B^s_{p,r}(\mathbb{R}^d),s>\max\{1+\frac d2,\frac32\}$.