Search

Manipulation tasks often require a high degree of dexterity, typically necessitating grippers with multiple degrees of freedom (DoF). While a robotic hand equipped with multiple fingers can execute precise and intricate manipulation tasks, the inherent redundancy stemming from its extensive DoF often adds unnecessary complexity. In this paper, we introduce the design of a tactile sensor-equipped gripper with two fingers and five DoF. We present a novel design integrating a GelSight tactile sensor, enhancing sensing capabilities and enabling finer control during specific manipulation tasks. To evaluate the gripper's performance, we conduct experiments involving two challenging tasks: 1) retrieving, singularizing, and classification of various objects embedded in granular media, and 2) executing scooping manipulations of credit cards in confined environments to achieve precise insertion. Our results demonstrate the efficiency of the proposed approach, with a high success rate for singulation and classification tasks, particularly for spherical objects at high as 94.3%, and a 100% success rate for scooping and inserting credit cards., Comment: 6 pages, 6 figures, accepted to the 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024). Video is available at: https://youtu.be/6c1AyeaGjbk

The increasing development of large language models (LLMs) in code generation has drawn significant attention among researchers. To enhance LLM-based code generation ability, current efforts are predominantly directed towards collecting high-quality datasets and leveraging diverse training technologies. However, there is a notable lack of comprehensive studies examining the limitations and boundaries of these existing methods. To bridge this gap, we conducted an extensive empirical study evaluating the performance of three leading closed-source LLMs and four popular open-source LLMs on three commonly used benchmarks. Our investigation, which evaluated the length, cyclomatic complexity and API number of the generated code, revealed that these LLMs face challenges in generating successful code for more complex problems, and tend to produce code that is shorter yet more complicated as compared to canonical solutions. Additionally, we developed a taxonomy of bugs for incorrect codes that includes three categories and 12 sub-categories, and analyze the root cause for common bug types. Furthermore, to better understand the performance of LLMs in real-world projects, we manually created a real-world benchmark comprising 140 code generation tasks. Our analysis highlights distinct differences in bug distributions between actual scenarios and existing benchmarks. Finally, we propose a novel training-free iterative method that introduces self-critique, enabling LLMs to critique and correct their generated code based on bug types and compiler feedback. Experimental results demonstrate that our approach can significantly mitigate bugs and increase the passing rate by 29.2% after two iterations, indicating substantial potential for LLMs to handle more complex problems., Comment: 17 pages, 7 figures

With the development of the open source community, the code is often copied, spread, and evolved in multiple software systems, which brings uncertainty and risk to the software system (e.g., bug propagation and copyright infringement). Therefore, it is important to conduct code clone detection to discover similar code pairs. Many approaches have been proposed to detect code clones where token-based tools can scale to big code. However, due to the lack of program details, they cannot handle more complicated code clones, i.e., semantic code clones. In this paper, we introduce CC2Vec, a novel code encoding method designed to swiftly identify simple code clones while also enhancing the capability for semantic code clone detection. To retain the program details between tokens, CC2Vec divides them into different categories (i.e., typed tokens) according to the syntactic types and then applies two self-attention mechanism layers to encode them. To resist changes in the code structure of semantic code clones, CC2Vec performs contrastive learning to reduce the differences introduced by different code implementations. We evaluate CC2Vec on two widely used datasets (i.e., BigCloneBench and Google Code Jam) and the results report that our method can effectively detect simple code clones. In addition, CC2Vec not only attains comparable performance to widely used semantic code clone detection systems such as ASTNN, SCDetector, and FCCA by simply fine-tuning, but also significantly surpasses these methods in both detection efficiency., Comment: 21 pages, 7 figures

Large-scale coronal plasma evolutions can be adequately described by magnetohydrodynamics (MHD) equations. However, full multi-dimensional MHD simulations require substantial computational resources. Given the low plasma $\beta$ in the solar corona, in many coronal studies, it suffices to approximate the magnetic field to remain topologically fixed and effectively conduct one-dimensional (1D) hydrodynamic (HD) simulations instead. This approach is often employed in studies of coronal loops and their liability to form condensations related to thermal instability. While 1D HD simulations along given and fixed field line shapes are convenient and fast, they are difficult to directly compare with multi-dimensional phenomena. Therefore, it is more convenient to solve volume-filling, multi-dimensional versions of the MHD equations where we freeze the magnetic field, transforming it into frozen-field HD (ffHD) equations for simulation. We have incorporated this ffHD module into our open-source MPI-AMRVAC code and tested it using a two-dimensional (2D) evaporation--condensation model to study prominence formation due to radiative losses. The 2D ffHD results are compared with those from actual 2D MHD and pseudo-2D HD simulations, analyzing the differences and their causes. Pseudo-2D studies account for the known {flux tube} expansion effects. Overall, the performance of 2D ffHD is close to that of 2D MHD and pseudo-2D HD. The 2D tests conducted in this paper will be extended in follow-up studies to 3D simulations based on analytical or observational approaches., Comment: Accepted for publication in ApJ. 25 pages, 8 figures

Bayesian flow networks (BFNs) iteratively refine the parameters, instead of the samples in diffusion models (DMs), of distributions at various noise levels through Bayesian inference. Owing to its differentiable nature, BFNs are promising in modeling both continuous and discrete data, while simultaneously maintaining fast sampling capabilities. This paper aims to understand and enhance BFNs by connecting them with DMs through stochastic differential equations (SDEs). We identify the linear SDEs corresponding to the noise-addition processes in BFNs, demonstrate that BFN's regression losses are aligned with denoise score matching, and validate the sampler in BFN as a first-order solver for the respective reverse-time SDE. Based on these findings and existing recipes of fast sampling in DMs, we propose specialized solvers for BFNs that markedly surpass the original BFN sampler in terms of sample quality with a limited number of function evaluations (e.g., 10) on both image and text datasets. Notably, our best sampler achieves an increase in speed of 5~20 times for free. Our code is available at https://github.com/ML-GSAI/BFN-Solver., Comment: Published as a conference paper at ICML 2024

Deep neural networks (DNNs) are notoriously vulnerable to adversarial attacks that place carefully crafted perturbations on normal examples to fool DNNs. To better understand such attacks, a characterization of the features carried by adversarial examples is needed. In this paper, we tackle this challenge by inspecting the subspaces of sample features through spectral analysis. We first empirically show that the features of either clean signals or adversarial perturbations are redundant and span in low-dimensional linear subspaces respectively with minimal overlap, and the classical low-dimensional subspace projection can suppress perturbation features out of the subspace of clean signals. This makes it possible for DNNs to learn a subspace where only features of clean signals exist while those of perturbations are discarded, which can facilitate the distinction of adversarial examples. To prevent the residual perturbations that is inevitable in subspace learning, we propose an independence criterion to disentangle clean signals from perturbations. Experimental results show that the proposed strategy enables the model to inherently suppress adversaries, which not only boosts model robustness but also motivates new directions of effective adversarial defense., Comment: Accepted by COLING 2024

Knowledge graphs have garnered significant research attention and are widely used to enhance downstream applications. However, most current studies mainly focus on static knowledge graphs, whose facts do not change with time, and disregard their dynamic evolution over time. As a result, temporal knowledge graphs have attracted more attention because a large amount of structured knowledge exists only within a specific period. Knowledge graph representation learning aims to learn low-dimensional vector embeddings for entities and relations in a knowledge graph. The representation learning of temporal knowledge graphs incorporates time information into the standard knowledge graph framework and can model the dynamics of entities and relations over time. In this paper, we conduct a comprehensive survey of temporal knowledge graph representation learning and its applications. We begin with an introduction to the definitions, datasets, and evaluation metrics for temporal knowledge graph representation learning. Next, we propose a taxonomy based on the core technologies of temporal knowledge graph representation learning methods, and provide an in-depth analysis of different methods in each category. Finally, we present various downstream applications related to the temporal knowledge graphs. In the end, we conclude the paper and have an outlook on the future research directions in this area.

Temporal knowledge graph completion (TKGC) aims to fill in missing facts within a given temporal knowledge graph at a specific time. Existing methods, operating in real or complex spaces, have demonstrated promising performance in this task. This paper advances beyond conventional approaches by introducing more expressive quaternion representations for TKGC within hypercomplex space. Unlike existing quaternion-based methods, our study focuses on capturing time-sensitive relations rather than time-aware entities. Specifically, we model time-sensitive relations through time-aware rotation and periodic time translation, effectively capturing complex temporal variability. Furthermore, we theoretically demonstrate our method's capability to model symmetric, asymmetric, inverse, compositional, and evolutionary relation patterns. Comprehensive experiments on public datasets validate that our proposed approach achieves state-of-the-art performance in the field of TKGC.

In this paper, we propose R$^3$: Learning Reasoning through Reverse Curriculum Reinforcement Learning (RL), a novel method that employs only outcome supervision to achieve the benefits of process supervision for large language models. The core challenge in applying RL to complex reasoning is to identify a sequence of actions that result in positive rewards and provide appropriate supervision for optimization. Outcome supervision provides sparse rewards for final results without identifying error locations, whereas process supervision offers step-wise rewards but requires extensive manual annotation. R$^3$ overcomes these limitations by learning from correct demonstrations. Specifically, R$^3$ progressively slides the start state of reasoning from a demonstration's end to its beginning, facilitating easier model exploration at all stages. Thus, R$^3$ establishes a step-wise curriculum, allowing outcome supervision to offer step-level signals and precisely pinpoint errors. Using Llama2-7B, our method surpasses RL baseline on eight reasoning tasks by $4.1$ points on average. Notebaly, in program-based reasoning on GSM8K, it exceeds the baseline by $4.2$ points across three backbone models, and without any extra data, Codellama-7B + R$^3$ performs comparable to larger models or closed-source models., Comment: Preprint. Codes released: https://github.com/WooooDyy/LLM-Reverse-Curriculum-RL

The advancement of large language models (LLMs) has significantly propelled the field of code generation. Previous work integrated reinforcement learning (RL) with compiler feedback for exploring the output space of LLMs to enhance code generation quality. However, the lengthy code generated by LLMs in response to complex human requirements makes RL exploration a challenge. Also, since the unit tests may not cover the complicated code, optimizing LLMs by using these unexecuted code snippets is ineffective. To tackle these challenges, we introduce StepCoder, a novel RL framework for code generation, consisting of two main components: CCCS addresses the exploration challenge by breaking the long sequences code generation task into a Curriculum of Code Completion Subtasks, while FGO only optimizes the model by masking the unexecuted code segments to provide Fine-Grained Optimization. In addition, we furthermore construct the APPS+ dataset for RL training, which is manually verified to ensure the correctness of unit tests. Experimental results show that our method improves the ability to explore the output space and outperforms state-of-the-art approaches in corresponding benchmarks. Our dataset APPS+ and StepCoder are available online., Comment: 13 pages, 5 figures

Current large vision-language models (VLMs) often encounter challenges such as insufficient capabilities of a single visual component and excessively long visual tokens. These issues can limit the model's effectiveness in accurately interpreting complex visual information and over-lengthy contextual information. Addressing these challenges is crucial for enhancing the performance and applicability of VLMs. This paper proposes the use of ensemble experts technique to synergizes the capabilities of individual visual encoders, including those skilled in image-text matching, OCR, image segmentation, etc. This technique introduces a fusion network to unify the processing of outputs from different visual experts, while bridging the gap between image encoders and pre-trained LLMs. In addition, we explore different positional encoding schemes to alleviate the waste of positional encoding caused by lengthy image feature sequences, effectively addressing the issue of position overflow and length limitations. For instance, in our implementation, this technique significantly reduces the positional occupancy in models like SAM, from a substantial 4096 to a more efficient and manageable 64 or even down to 1. Experimental results demonstrate that VLMs with multiple experts exhibit consistently superior performance over isolated visual encoders and mark a significant performance boost as more experts are integrated. We have open-sourced the training code used in this report. All of these resources can be found on our project website.

Reinforcement Learning from Human Feedback (RLHF) has become a crucial technology for aligning language models with human values and intentions, enabling models to produce more helpful and harmless responses. Reward models are trained as proxies for human preferences to drive reinforcement learning optimization. While reward models are often considered central to achieving high performance, they face the following challenges in practical applications: (1) Incorrect and ambiguous preference pairs in the dataset may hinder the reward model from accurately capturing human intent. (2) Reward models trained on data from a specific distribution often struggle to generalize to examples outside that distribution and are not suitable for iterative RLHF training. In this report, we attempt to address these two issues. (1) From a data perspective, we propose a method to measure the strength of preferences within the data, based on a voting mechanism of multiple reward models. Experimental results confirm that data with varying preference strengths have different impacts on reward model performance. We introduce a series of novel methods to mitigate the influence of incorrect and ambiguous preferences in the dataset and fully leverage high-quality preference data. (2) From an algorithmic standpoint, we introduce contrastive learning to enhance the ability of reward models to distinguish between chosen and rejected responses, thereby improving model generalization. Furthermore, we employ meta-learning to enable the reward model to maintain the ability to differentiate subtle differences in out-of-distribution samples, and this approach can be utilized for iterative RLHF optimization.

The integration of neural networks into safety-critical systems has shown great potential in recent years. However, the challenge of effectively verifying the safety of Neural Network Controlled Systems (NNCS) persists. This paper introduces a novel approach to NNCS safety verification, leveraging the inductive invariant method. Verifying the inductiveness of a candidate inductive invariant in the context of NNCS is hard because of the scale and nonlinearity of neural networks. Our compositional method makes this verification process manageable by decomposing the inductiveness proof obligation into smaller, more tractable subproblems. Alongside the high-level method, we present an algorithm capable of automatically verifying the inductiveness of given candidates by automatically inferring the necessary decomposition predicates. The algorithm significantly outperforms the baseline method and shows remarkable reductions in execution time in our case studies, shortening the verification time from hours (or timeout) to seconds.

Supervised fine-tuning (SFT) is a crucial step for large language models (LLMs), enabling them to align with human instructions and enhance their capabilities in downstream tasks. Increasing instruction data substantially is a direct solution to align the model with a broader range of downstream tasks or notably improve its performance on a specific task. However, we find that large-scale increases in instruction data can damage the world knowledge previously stored in LLMs. To address this challenge, we propose LoRAMoE, a novelty framework that introduces several low-rank adapters (LoRA) and integrates them by using a router network, like a plugin version of Mixture of Experts (MoE). It freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge-edge forgetting. Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM., Comment: 14 pages, 7 figures

We present a unified probabilistic formulation for diffusion-based image editing, where a latent variable is edited in a task-specific manner and generally deviates from the corresponding marginal distribution induced by the original stochastic or ordinary differential equation (SDE or ODE). Instead, it defines a corresponding SDE or ODE for editing. In the formulation, we prove that the Kullback-Leibler divergence between the marginal distributions of the two SDEs gradually decreases while that for the ODEs remains as the time approaches zero, which shows the promise of SDE in image editing. Inspired by it, we provide the SDE counterparts for widely used ODE baselines in various tasks including inpainting and image-to-image translation, where SDE shows a consistent and substantial improvement. Moreover, we propose SDE-Drag -- a simple yet effective method built upon the SDE formulation for point-based content dragging. We build a challenging benchmark (termed DragBench) with open-set natural, art, and AI-generated images for evaluation. A user study on DragBench indicates that SDE-Drag significantly outperforms our ODE baseline, existing diffusion-based methods, and the renowned DragGAN. Our results demonstrate the superiority and versatility of SDE in image editing and push the boundary of diffusion-based editing methods.

The success of AI assistants based on language models (LLMs) hinges crucially on Reinforcement Learning from Human Feedback (RLHF), which enables the generation of responses more aligned with human preferences. As universal AI assistants, there's a growing expectation for them to perform consistently across various domains. However, previous work shows that Reinforcement Learning (RL) often exploits shortcuts to attain high rewards and overlooks challenging samples. This focus on quick reward gains undermines both the stability in training and the model's ability to generalize to new, unseen data. In this work, we propose a novel approach that can learn a consistent policy via RL across various data groups or domains. Given the challenges associated with acquiring group annotations, our method automatically classifies data into different groups, deliberately maximizing performance variance. Then, we optimize the policy to perform well on challenging groups. Lastly, leveraging the established groups, our approach adaptively adjusts the exploration space, allocating more learning capacity to more challenging data and preventing the model from over-optimizing on simpler data. Experimental results indicate that our approach significantly enhances training stability and model generalization.

Classical federated learning (FL) enables training machine learning models without sharing data for privacy preservation, but heterogeneous data characteristic degrades the performance of the localized model. Personalized FL (PFL) addresses this by synthesizing personalized models from a global model via training on local data. Such a global model may overlook the specific information that the clients have been sampled. In this paper, we propose a novel scheme to inject personalized prior knowledge into the global model in each client, which attempts to mitigate the introduced incomplete information problem in PFL. At the heart of our proposed approach is a framework, the PFL with Bregman Divergence (pFedBreD), decoupling the personalized prior from the local objective function regularized by Bregman divergence for greater adaptability in personalized scenarios. We also relax the mirror descent (RMD) to extract the prior explicitly to provide optional strategies. Additionally, our pFedBreD is backed up by a convergence analysis. Sufficient experiments demonstrate that our method reaches the state-of-the-art performances on 5 datasets and outperforms other methods by up to 3.5% across 8 benchmarks. Extensive analyses verify the robustness and necessity of proposed designs., Comment: Accepted by NeurIPS 2023

For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI agents are artificial entities that sense their environment, make decisions, and take actions. Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks. Actually, what the community lacks is a general and powerful model to serve as a starting point for designing AI agents that can adapt to diverse scenarios. Due to the versatile capabilities they demonstrate, large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI), offering hope for building general AI agents. Many researchers have leveraged LLMs as the foundation to build AI agents and have achieved significant progress. In this paper, we perform a comprehensive survey on LLM-based agents. We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. Building upon this, we present a general framework for LLM-based agents, comprising three main components: brain, perception, and action, and the framework can be tailored for different applications. Subsequently, we explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation. Following this, we delve into agent societies, exploring the behavior and personality of LLM-based agents, the social phenomena that emerge from an agent society, and the insights they offer for human society. Finally, we discuss several key topics and open problems within the field. A repository for the related papers at https://github.com/WooooDyy/LLM-Agent-Paper-List., Comment: 86 pages, 12 figures

Federated Learning (FL) presents an innovative approach to privacy-preserving distributed machine learning and enables efficient crowd intelligence on a large scale. However, a significant challenge arises when coordinating FL with crowd intelligence which diverse client groups possess disparate objectives due to data heterogeneity or distinct tasks. To address this challenge, we propose the Federated cINN Clustering Algorithm (FCCA) to robustly cluster clients into different groups, avoiding mutual interference between clients with data heterogeneity, and thereby enhancing the performance of the global model. Specifically, FCCA utilizes a global encoder to transform each client's private data into multivariate Gaussian distributions. It then employs a generative model to learn encoded latent features through maximum likelihood estimation, which eases optimization and avoids mode collapse. Finally, the central server collects converged local models to approximate similarities between clients and thus partition them into distinct clusters. Extensive experimental results demonstrate FCCA's superiority over other state-of-the-art clustered federated learning algorithms, evaluated on various models and datasets. These results suggest that our approach has substantial potential to enhance the efficiency and accuracy of real-world federated learning tasks.

Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, honest, and harmless) assistant. Alignment with humans assumes paramount significance, and reinforcement learning with human feedback (RLHF) emerges as the pivotal technological paradigm underpinning this pursuit. Current technical routes usually include \textbf{reward models} to measure human preferences, \textbf{Proximal Policy Optimization} (PPO) to optimize policy model outputs, and \textbf{process supervision} to improve step-by-step reasoning capabilities. However, due to the challenges of reward design, environment interaction, and agent training, coupled with huge trial and error cost of large language models, there is a significant barrier for AI researchers to motivate the development of technical alignment and safe landing of LLMs. The stable training of RLHF has still been a puzzle. In the first report, we dissect the framework of RLHF, re-evaluate the inner workings of PPO, and explore how the parts comprising PPO algorithms impact policy agent training. We identify policy constraints being the key factor for the effective implementation of the PPO algorithm. Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model. Based on our main results, we perform a comprehensive analysis of RLHF abilities compared with SFT models and ChatGPT. The absence of open-source implementations has posed significant challenges to the investigation of LLMs alignment. Therefore, we are eager to release technical reports, reward models and PPO codes, aiming to make modest contributions to the advancement of LLMs.

To enhance the multi-step reasoning capabilities of large language models, researchers have extensively explored prompting methods, notably the Chain-of-Thought (CoT) method which explicitly elicits human-like rationales. However, they have inadvertently overlooked the potential of enhancing model reasoning performance by formulating higher-quality problems. In this work, we start from the problem side and propose Self-Polish (SP), a novel method that facilitates the model's reasoning by guiding it to progressively refine the given problems to be more comprehensible and solvable. We also explore several automatic prompting varients and propose the Self-Polish prompt bank for the community. SP is orthogonal to all other prompting methods of answer/reasoning side like CoT, allowing for seamless integration with state-of-the-art techniques for further improvement. Thorough experiments show that the proposed method attains notable and consistent effectiveness on five reasoning benchmarks across different models. Furthermore, our method also showcases impressive performance on robustness evaluation. Codes and prompts are available at https://github.com/WooooDyy/Self-Polish., Comment: Accepted to EMNLP 2023 Findings. Codes and prompts are available at https://github.com/WooooDyy/Self-Polish

Observations have shown that some filaments appear and disappear in the H$\alpha$ line wing images periodically. There have been no attempts to model these "winking filaments" thus far. The evaporation--condensation mechanism is widely used to explain the formation of solar filaments. Here, we demonstrate, for the first time, how multi-dimensional evaporation--condensation in an arcade setup invariably causes a stretching of the magnetic topology. We aim to check whether this magnetic stretching during cyclic evaporation--condensation could reproduce a winking filament. We used our open-source code MPI-AMRVAC to carry out 2D magnetohydrodynamic simulations based on a quadrupolar configuration. A periodic localized heating, which modulates the evaporation--condensation process, was imposed before, during, and after the formation of the filament. Synthetic H$\alpha$ and 304 \r{A}, images were produced to compare the results with observations. For the first time, we noticed the winking filament phenomenon in a simulation of the formation of on-disk solar filaments, which was in good agreement with observations. Typically, the period of the winking is different from the period of the impulsive heating. A forced oscillator model explains this difference and fits the results well. A parameter survey is also done to look into details of the magnetic stretching phenomenon. We found that the stronger the heating or the higher the layer where the heating occurs, the more significant the winking effect appears., Comment: 14 pages, 6 figures. Accepted by A&A

In this paper, we carry out a detailed analysis of the M1.6 class eruptive flare occurring in NOAA active region 13078 on 2022 August 19. The flare is associated with a fast coronal mass ejection (CME) propagating in the southwest direction with an apparent speed of $\sim$926 km s$^{-1}$. Meanwhile, a shock wave is driven by the CME at the flank. The eruption of CME generates an extreme-ultraviolet (EUV) wave expanding outward from the flare site with an apparent speed of $\geq$200 km s$^{-1}$. As the EUV wave propagates eastward, it encounters and interacts with the low-lying adjacent coronal loops (ACLs), which are composed of two loops. The compression of EUV wave results in contraction, expansion, and transverse vertical oscillations of ACLs. The commencements of contraction are sequential from western to eastern footpoints and the contraction lasts for $\sim$15 minutes. The speeds of contraction lie in the range of 13$-$40 km s$^{-1}$ in 171 {\AA} and 8$-$54 km s$^{-1}$ in 193 {\AA}. A long, gradual expansion follows the contraction at lower speeds. Concurrent vertical oscillations are superposed on contraction and expansion of ACLs. The oscillations last for 2$-$9 cycles and the amplitudes are $\leq$4 Mm. The periods are between 3 to 12 minutes with an average value of 6.7 minutes. The results show rich dynamics of coronal loops., Comment: 10 pages, 10 figures, accepted for publication in ApJ

Flux emergence is responsible for various solar eruptions. Combining observation and simulations, we investigate the influence of flux emergence at one footpoint of an arcade on coronal rain as well as induced eruptions. The emergence changes the pressure in the loops, and the internal coronal rain all moves to the other side. The emerging flux reconnects with the overlying magnetic field, forming a current sheet and magnetic islands. The plasma is ejected outwards and heated, forming a cool jet ~ 6000 K and a hot X-ray jet ~ 4 MK simultaneously. The jet dynamical properties agree very well between observation and simulation. In the simulation, the jet also displays transverse oscillations with a period of 8 minutes, a so-called whip-like motion. The movement of the jet and dense plasmoids changes the configuration of the local magnetic field, facilitating the occurrence of Kelvin--Helmholtz instability, and vortex-like structures form at the boundary of the jet. Our simulation clearly demonstrates the effect of emergence on coronal rain, the dynamical details of reconnecting plasmoid chains, the formation of multi-thermal jets, and the cycling of cool mass between the chromosphere and the corona., Comment: 11 pages, 5 figures, accepted for publication in The Astrophysical Journal Letters

Computational astrophysics routinely combines grid-adaptive capabilities with modern shock-capturing, high resolution spatio-temporal schemes on multi-dimensional hydro- and magnetohydrodynamics. We provide an update on developments within the open-source MPI-AMRVAC code. With online documentation, the MPI-AMRVAC 3.0 release includes several added equation sets, and many options to explore and quantify the influence of implementation details. Showcasing this on a variety of hydro and MHD tests, we document new modules of interest for state-of-the-art solar applications. Test cases address how higher order reconstructions impact long term simulations of shear layers, with and without gas-dust coupling, how runaway radiative losses transit to intricate multi-temperature, multi-phase dynamics, and how different flavors of spatio-temporal schemes and magnetic monopole control produce consistent MHD results in combination with adaptive meshes. We demonstrate Super-Time-Stepping strategies for specific parabolic terms and give details on all implemented Implicit-Explicit integrators. A new magnetofrictional module can be used for computing force-free magnetic fields or for data-driven time-dependent evolutions, while the Regularized-Biot-Savart-Law approach can insert fluxropes in 3D domains. Synthetic observations of 3D MHD simulations can be rendered on-the-fly, or in post-processing, in many spectral wavebands. A particle module and a generic fieldline tracing, compatible with the hierarchical meshes, can be used to sample information at prescribed locations, to follow dynamics of charged particles, or realize two-way coupled simulations between MHD setups and field-aligned non-thermal processes. Highlighting the latest additions and various technical aspects, our open-source strategy welcomes any further code usage, contribution, or spin-off development., Comment: Accepted for A&A

Reducing communication overhead in federated learning (FL) is challenging but crucial for large-scale distributed privacy-preserving machine learning. While methods utilizing sparsification or others can largely lower the communication overhead, the convergence rate is also greatly compromised. In this paper, we propose a novel method, named single-step synthetic features compressor (3SFC), to achieve communication-efficient FL by directly constructing a tiny synthetic dataset based on raw gradients. Thus, 3SFC can achieve an extremely low compression rate when the constructed dataset contains only one data sample. Moreover, 3SFC's compressing phase utilizes a similarity-based objective function so that it can be optimized with just one step, thereby considerably improving its performance and robustness. In addition, to minimize the compressing error, error feedback (EF) is also incorporated into 3SFC. Experiments on multiple datasets and models suggest that 3SFC owns significantly better convergence rates compared to competing methods with lower compression rates (up to 0.02%). Furthermore, ablation studies and visualizations show that 3SFC can carry more information than competing methods for every communication round, further validating its effectiveness.

In this paper, we report the multiwavelength observations of an erupting prominence and the associated CME on 13 May 2013. The event occurs behind the western limb in the field of view of SDO/AIA. The prominence is supported by a highly twisted magnetic flux rope and shows rapid rotation in the counterclockwise direction during the rising motion. The rotation of the prominence lasts for $\sim$47 minutes. The average period, angular speed, and linear speed are $\sim$806 s, $\sim$0.46 rad min$^{-1}$, and $\sim$355 km s$^{-1}$, respectively. The total twist angle reaches $\sim$7$\pi$, which is considerably larger than the threshold for kink instability. Writhing motion during 17:42$-$17:46 UT is clearly observed by SWAP in 174 {\AA} and EUVI on board the behind STEREO spacecraft in 304 {\AA} after reaching an apparent height of $\sim$405\,Mm. Therefore, the prominence eruption is most probably triggered by kink instability. A pair of conjugate flare ribbons and post-flare loops are created and observed by STA/EUVI. The onset time of writhing motion is consistent with the commencement of the impulsive phase of the related flare. The 3D morphology and positions of the associated CME are derived using the graduated cylindrical shell (GCS) modeling. The kinetic evolution of the reconstructed CME is divided into a slow-rise phase ($\sim$330 km s$^{-1}$) and a fast-rise phase ($\sim$1005 km s$^{-1}$) by the writhing motion. The edge-on angular width of the CME is a constant (60$^{\circ}$), while the face-on angular width increases from 96$^{\circ}$ to 114$^{\circ}$, indicating a lateral expansion. The latitude of the CME source region decreases slightly from $\sim$18$^{\circ}$ to $\sim$13$^{\circ}$, implying an equatorward deflection during propagation., Comment: 28 pages, 20 figures, accepted for publication in Solar Physics, comments are welcome

Federated learning (FL for simplification) is a distributed machine learning technique that utilizes global servers and collaborative clients to achieve privacy-preserving global model training without direct data sharing. However, heterogeneous data problem, as one of FL's main problems, makes it difficult for the global model to perform effectively on each client's local data. Thus, personalized federated learning (PFL for simplification) aims to improve the performance of the model on local data as much as possible. Bayesian learning, where the parameters of the model are seen as random variables with a prior assumption, is a feasible solution to the heterogeneous data problem due to the tendency that the more local data the model use, the more it focuses on the local data, otherwise focuses on the prior. When Bayesian learning is applied to PFL, the global model provides global knowledge as a prior to the local training process. In this paper, we employ Bayesian learning to model PFL by assuming a prior in the scaled exponential family, and therefore propose pFedBreD, a framework to solve the problem we model using Bregman divergence regularization. Empirically, our experiments show that, under the prior assumption of the spherical Gaussian and the first order strategy of mean selection, our proposal significantly outcompetes other PFL algorithms on multiple public benchmarks., Comment: 19 pages, 6 figures, 3 tables

Recent works on Lottery Ticket Hypothesis have shown that pre-trained language models (PLMs) contain smaller matching subnetworks(winning tickets) which are capable of reaching accuracy comparable to the original models. However, these tickets are proved to be notrobust to adversarial examples, and even worse than their PLM counterparts. To address this problem, we propose a novel method based on learning binary weight masks to identify robust tickets hidden in the original PLMs. Since the loss is not differentiable for the binary mask, we assign the hard concrete distribution to the masks and encourage their sparsity using a smoothing approximation of L0 regularization.Furthermore, we design an adversarial loss objective to guide the search for robust tickets and ensure that the tickets perform well bothin accuracy and robustness. Experimental results show the significant improvement of the proposed method over previous work on adversarial robustness evaluation., Comment: ACL 2022. https://aclanthology.org/2022.acl-long.157

Diffusion probabilistic models (DPMs) have achieved impressive success in high-resolution image synthesis, especially in recent large-scale text-to-image generation applications. An essential technique for improving the sample quality of DPMs is guided sampling, which usually needs a large guidance scale to obtain the best sample quality. The commonly-used fast sampler for guided sampling is DDIM, a first-order diffusion ODE solver that generally needs 100 to 250 steps for high-quality samples. Although recent works propose dedicated high-order solvers and achieve a further speedup for sampling without guidance, their effectiveness for guided sampling has not been well-tested before. In this work, we demonstrate that previous high-order fast samplers suffer from instability issues, and they even become slower than DDIM when the guidance scale grows large. To further speed up guided sampling, we propose DPM-Solver++, a high-order solver for the guided sampling of DPMs. DPM-Solver++ solves the diffusion ODE with the data prediction model and adopts thresholding methods to keep the solution matches training data distribution. We further propose a multistep variant of DPM-Solver++ to address the instability issue by reducing the effective step size. Experiments show that DPM-Solver++ can generate high-quality samples within only 15 to 20 steps for guided sampling by pixel-space and latent-space DPMs.

Diffusion probabilistic models (DPMs) are emerging powerful generative models. Despite their high-quality generation performance, DPMs still suffer from their slow sampling as they generally need hundreds or thousands of sequential function evaluations (steps) of large neural networks to draw a sample. Sampling from DPMs can be viewed alternatively as solving the corresponding diffusion ordinary differential equations (ODEs). In this work, we propose an exact formulation of the solution of diffusion ODEs. The formulation analytically computes the linear part of the solution, rather than leaving all terms to black-box ODE solvers as adopted in previous works. By applying change-of-variable, the solution can be equivalently simplified to an exponentially weighted integral of the neural network. Based on our formulation, we propose DPM-Solver, a fast dedicated high-order solver for diffusion ODEs with the convergence order guarantee. DPM-Solver is suitable for both discrete-time and continuous-time DPMs without any further training. Experimental results show that DPM-Solver can generate high-quality samples in only 10 to 20 function evaluations on various datasets. We achieve 4.70 FID in 10 function evaluations and 2.87 FID in 20 function evaluations on the CIFAR10 dataset, and a $4\sim 16\times$ speedup compared with previous state-of-the-art training-free samplers on various datasets., Comment: Accepted in Neurips 2022

We investigate nonlinear instrumental variable (IV) regression given high-dimensional instruments. We propose a simple algorithm which combines kernelized IV methods and an arbitrary, adaptive regression algorithm, accessed as a black box. Our algorithm enjoys faster-rate convergence and adapts to the dimensionality of informative latent features, while avoiding an expensive minimax optimization procedure, which has been necessary to establish similar guarantees. It further brings the benefit of flexible machine learning models to quasi-Bayesian uncertainty quantification, likelihood-based model selection, and model averaging. Simulation studies demonstrate the competitive performance of our method., Comment: NeurIPS camera ready. Code available at https://github.com/meta-inf/fil

Federated learning (FL) is identified as a crucial enabler for large-scale distributed machine learning (ML) without the need for local raw dataset sharing, substantially reducing privacy concerns and alleviating the isolated data problem. In reality, the prosperity of FL is largely due to a centralized framework called FedAvg, in which workers are in charge of model training and servers are in control of model aggregation. However, FedAvg's centralized worker-server architecture has raised new concerns, be it the low scalability of the cluster, the risk of data leakage, and the failure or even defection of the central server. To overcome these problems, we propose Decentralized Federated Trusted Averaging (DeFTA), a decentralized FL framework that serves as a plug-and-play replacement for FedAvg, instantly bringing better security, scalability, and fault-tolerance to the federated learning process after installation. In principle, it fundamentally resolves the above-mentioned issues from an architectural perspective without compromises or tradeoffs, primarily consisting of a new model aggregating formula with theoretical performance analysis, and a decentralized trust system (DTS) to greatly improve system robustness. Note that since DeFTA is an alternative to FedAvg at the framework level, \textit{prevalent algorithms published for FedAvg can be also utilized in DeFTA with ease}. Extensive experiments on six datasets and six basic models suggest that DeFTA not only has comparable performance with FedAvg in a more realistic setting, but also achieves great resilience even when 66% of workers are malicious. Furthermore, we also present an asynchronous variant of DeFTA to endow it with more powerful usability., Comment: 12 pages, 5 figures

We propose Fuse Local and Global Semantics in Representation Learning (FLAGS) to generate richer representations. FLAGS aims at extract both global and local semantics from images to benefit various downstream tasks. It shows promising results under common linear evaluation protocol. We also conduct detection and segmentation on PASCAL VOC and COCO to show the representations extracted by FLAGS are transferable.

Gradient estimation -- approximating the gradient of an expectation with respect to the parameters of a distribution -- is central to the solution of many machine learning problems. However, when the distribution is discrete, most common gradient estimators suffer from excessive variance. To improve the quality of gradient estimation, we introduce a variance reduction technique based on Stein operators for discrete distributions. We then use this technique to build flexible control variates for the REINFORCE leave-one-out estimator. Our control variates can be adapted online to minimize variance and do not require extra evaluations of the target function. In benchmark generative modeling tasks such as training binary variational autoencoders, our gradient estimator achieves substantially lower variance than state-of-the-art estimators with the same number of function evaluations., Comment: NeurIPS 2022. Source code: https://github.com/thjashin/rodeo

Adopting the MPI-AMRVAC code, we present a 2.5-dimensional magnetohydrodynamic (MHD) simulation, which includes thermal conduction and radiative cooling, to investigate the formation and evolution of the coronal rain phenomenon. We perform the simulation in initially linear force-free magnetic fields which host chromospheric, transition region, and coronal plasma, with turbulent heating localized on their footpoints. Due to thermal instability, condensations start to occur at the loop top, and rebound shocks are generated by the siphon inflows. Condensations fragment into smaller blobs moving downwards and as they hit the lower atmosphere, concurrent upflows are triggered. Larger clumps show us clear "coronal rain showers" as dark structures in synthetic EUV hot channels and bright blobs with cool cores in the 304 {\AA} channel, well resembling real observations. Following coronal rain dynamics for more than 10 hours, we carry out a statistical study of all coronal rain blobs to quantify their widths, lengths, areas, velocity distributions, and other properties. The coronal rain shows us continuous heating-condensation cycles, as well as cycles in EUV emissions. Compared to previous studies adopting steady heating, the rain happens faster and in more erratic cycles. Although most blobs are falling downward, upward-moving blobs exist at basically every moment. We also track the movement of individual blobs to study their dynamics and the forces driving their movements. The blobs have a prominence-corona transition-region-like structure surrounding them, and their movements are dominated by the pressure evolution in the very dynamic loop system., Comment: 23 pages, 15 figures, accepted by ApJ

As the complexity of production processes increases, the diversity of data types drives the development of network monitoring technology. This paper mainly focuses on an online algorithm to detect serially correlated directed networks robustly and sensitively. First, we consider a transition probability matrix to resolve the double correlation of primary data. Further, since the sum of each row of the transition probability matrix is one, it standardizes the data, facilitating subsequent modeling. Then we extend the spring length based method to the multivariate case and propose an adaptive cumulative sum (CUSUM) control chart on the strength of a weighted statistic to monitor directed networks. This novel approach assumes only that the process observation is associated with nearby points without any parametric time series model, which is in line with reality. Simulation results and a real example from metro transportation demonstrate the superiority of our design.

Most solar flares demonstrate a prolonged, hourlong post-flare (or gradual) phase, characterized by arcade-like, post-flare loops (PFLs) visible in many extreme ultraviolet (EUV) passbands. These coronal loops are filled with hot -- $\sim 30 \,\mathrm{MK}$ -- and dense plasma, evaporated from the chromosphere during the impulsive phase of the flare, and they very gradually recover to normal coronal density and temperature conditions. During this gradual cooling down to $\sim 1 \,\mathrm{MK}$ regimes, much cooler -- $\sim 0.01 \,\mathrm{MK}$ -- and denser coronal rain is frequently observed inside PFLs. Understanding PFL dynamics in this long-duration, gradual phase is crucial to the entire corona-chromosphere mass and energy cycle. Here we report a simulation in which a solar flare evolves from pre-flare, over impulsive phase all the way into its gradual phase, which successfully reproduces post-flare coronal rain. This rain results from catastrophic cooling caused by thermal instability, and we analyse the entire mass and energy budget evolution driving this sudden condensation phenomenon. We find that the runaway cooling and rain formation also induces the appearance of dark post-flare loop systems, as observed in EUV channels. We confirm and augment earlier observational findings, suggesting that thermal conduction and radiative losses alternately dominate the cooling of PFLs., Comment: accepted by ApJL

Achieving short-distance flight helps improve the efficiency of humanoid robots moving in complex environments (e.g., crossing large obstacles or reaching high places) for rapid emergency missions. This study proposes a design of a flying humanoid robot named Jet-HR2. The robot has 10 joints driven by brushless motors and harmonic drives for locomotion. To overcome the challenge of the stable-attitude takeoff in small thrust-to-weight conditions, the robot was designed based on the concept of thrust vectoring. The propulsion system consists of four ducted fans, that is, two fixed on the waist of the robot and the other two mounted on the feet, for thrust vector control. The thrust vector is controlled by adjusting the attitude of the foot during the flight. A simplified model and control strategies are proposed to solve the problem of attitude instability caused by mass errors and joint position errors during takeoff. The experimental results show that the robot's spin and dive behaviors during takeoff were effectively suppressed by controlling the thrust vector of the ducted fan on the foot. The robot successfully achieved takeoff at a thrust-to-weight ratio of 1.17 (17 kg / 20 kg) and maintained a stable attitude, reaching a takeoff height of over 1000 mm., Comment: The article has been submitted to IEEE Robotics and Automation Letters (RA-L) with ICRA 2022 conference option. Supporting video: https://youtu.be/Z5xm8um8Sv8&ab_channel=JetPowerandHumanoidRobotLab

Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline is designed as a coarse-to-fine paradigm. In the retrieval part, we cascade the architecture of ResNet101-GeM-ArcFace and employ DBSCAN followed by spatial verification to obtain a better initial coarse pose. We design a module called observation constraints, which combines geometry information and semantic consistency for filtering outliers. Comprehensive experiments are conducted on open datasets, including retrieval on R-Oxford5k and R-Paris6k, semantic segmentation on Cityscapes, localization on Aachen Day-Night and InLoc. By creatively modifying separate modules in the total pipeline, our method achieves many performance improvements on the challenging localization benchmarks., Comment: Accepted by the 2021 International Conference on Robotics and Automation (ICRA2021)

As one of the main formation mechanisms of solar filament formation, the chromospheric evaporation-coronal condensation model has been confirmed by numerical simulations to explain the formation of filament threads very well in flux tubes with single dips. However, coronal magnetic extrapolations indicated that some magnetic field lines might possess more than one dip. It is expected that the formation process would be significantly different in this case compared to a single-dipped magnetic flux tube. In this paper, based on the evaporation-condensation model, we study filament thread formation in double-dipped magnetic flux tubes by numerical simulations. We find that only with particular combinations of magnetic configuration and heating, e.g., concentrated localized heating and a long magnetic flux tube with deep dips, can two threads form and persist in a double-dipped magnetic flux tube. Comparing our parametric survey with observations, we conclude that such magnetically connected threads due to multiple dips are more likely to exist in quiescent filaments than in active-region filaments. Moreover, we find that these threads are usually shorter than independently trapped threads, which might be one of the reasons why quiescent filaments have short threads. These characteristics of magnetically connected threads could also explain barbs and vertical threads in quiescent filaments., Comment: 43 pages, 13 figures, 4 tables. Accepted for publication in ApJ

Recent years have witnessed an upsurge of interest in employing flexible machine learning models for instrumental variable (IV) regression, but the development of uncertainty quantification methodology is still lacking. In this work we present a novel quasi-Bayesian procedure for IV regression, building upon the recently developed kernelized IV models and the dual/minimax formulation of IV regression. We analyze the frequentist behavior of the proposed method, by establishing minimax optimal contraction rates in $L_2$ and Sobolev norms, and discussing the frequentist validity of credible balls. We further derive a scalable inference algorithm which can be extended to work with wide neural network models. Empirical evaluation shows that our method produces informative uncertainty estimates on complex high-dimensional problems., Comment: Extended version of the NeurIPS'21 paper. ZW and YZ contribute equally