Search

Code Language Models (CLMs) have achieved tremendous progress in source code understanding and generation, leading to a significant increase in research interests focused on applying CLMs to real-world software engineering tasks in recent years. However, in realistic scenarios, CLMs are exposed to potential malicious adversaries, bringing risks to the confidentiality, integrity, and availability of CLM systems. Despite these risks, a comprehensive analysis of the security vulnerabilities of CLMs in the extremely adversarial environment has been lacking. To close this research gap, we categorize existing attack techniques into three types based on the CIA triad: poisoning attacks (integrity \& availability infringement), evasion attacks (integrity infringement), and privacy attacks (confidentiality infringement). We have collected so far the most comprehensive (79) papers related to adversarial machine learning for CLM from the research fields of artificial intelligence, computer security, and software engineering. Our analysis covers each type of risk, examining threat model categorization, attack techniques, and countermeasures, while also introducing novel perspectives on eXplainable AI (XAI) and exploring the interconnections between different risks. Finally, we identify current challenges and future research opportunities. This study aims to provide a comprehensive roadmap for both researchers and practitioners and pave the way towards more reliable CLMs for practical applications., Comment: Under a reviewing process since Sep. 3, 2024

The rapid progress in the reasoning capability of the Multi-modal Large Language Models (MLLMs) has triggered the development of autonomous agent systems on mobile devices. MLLM-based mobile agent systems consist of perception, reasoning, memory, and multi-agent collaboration modules, enabling automatic analysis of user instructions and the design of task pipelines with only natural language and device screenshots as inputs. Despite the increased human-machine interaction efficiency, the security risks of MLLM-based mobile agent systems have not been systematically studied. Existing security benchmarks for agents mainly focus on Web scenarios, and the attack techniques against MLLMs are also limited in the mobile agent scenario. To close these gaps, this paper proposes a mobile agent security matrix covering 3 functional modules of the agent systems. Based on the security matrix, this paper proposes 4 realistic attack paths and verifies these attack paths through 8 attack methods. By analyzing the attack results, this paper reveals that MLLM-based mobile agent systems are not only vulnerable to multiple traditional attacks, but also raise new security concerns previously unconsidered. This paper highlights the need for security awareness in the design of MLLM-based systems and paves the way for future research on attacks and defense methods., Comment: Preprint. Work in progress

From pedestrians to Kuramoto oscillators, interactions between agents govern how a multitude of dynamical systems evolve in space and time. Discovering how these agents relate to each other can improve our understanding of the often complex dynamics that underlie these systems. Recent works learn to categorize relationships between agents based on observations of their physical behavior. These approaches are limited in that the relationship categories are modelled as outcomes of categorical distribution, when in real world systems categories often intermingle and interact. In this work, we introduce a level of abstraction between the observable behavior of agents and the latent categories that determine their behavior. To do this, we learn a mapping from agent behavior to agent preferences for each latent category in a graph neural network. We integrate the physical proximity of agents and their preferences in a nonlinear opinion dynamics model which provides a mechanism to identify mutually exclusive latent categories, predict an agent's evolution in time, and control an agent's physical behavior. We demonstrate the utility of our model for learning interpretable categories, and its efficacy on long-horizon prediction across several benchmarks where we outperform existing methods.

In this paper, we introduce group convolutional neural networks (GCNNs) equivariant to color variation. GCNNs have been designed for a variety of geometric transformations from 2D and 3D rotation groups, to semi-groups such as scale. Despite the improved interpretability, accuracy and generalizability of these architectures, GCNNs have seen limited application in the context of perceptual quantities. Notably, the recent CEConv network uses a GCNN to achieve equivariance to hue transformations by convolving input images with a hue rotated RGB filter. However, this approach leads to invalid RGB values which break equivariance and degrade performance. We resolve these issues with a lifting layer that transforms the input image directly, thereby circumventing the issue of invalid RGB values and improving equivariance error by over three orders of magnitude. Moreover, we extend the notion of color equivariance to include equivariance to saturation shift. Our hue-, saturation-, and color-equivariant networks achieve strong generalization to out-of-distribution perceptual variations and improved sample efficiency over conventional architectures. We demonstrate the utility of our approach on synthetic and real world datasets where we consistently outperform competitive baselines.

Transferable adversarial examples cause practical security risks since they can mislead a target model without knowing its internal knowledge. A conventional recipe for maximizing transferability is to keep only the optimal adversarial example from all those obtained in the optimization pipeline. In this paper, for the first time, we question this convention and demonstrate that those discarded, sub-optimal adversarial examples can be reused to boost transferability. Specifically, we propose ``Adversarial Example Soups'' (AES), with AES-tune for averaging discarded adversarial examples in hyperparameter tuning and AES-rand for stability testing. In addition, our AES is inspired by ``model soups'', which averages weights of multiple fine-tuned models for improved accuracy without increasing inference time. Extensive experiments validate the global effectiveness of our AES, boosting 10 state-of-the-art transfer attacks and their combinations by up to 13% against 10 diverse (defensive) target models. We also show the possibility of generalizing AES to other types, e.g., directly averaging multiple in-the-wild adversarial examples that yield comparable success. A promising byproduct of AES is the improved stealthiness of adversarial examples since the perturbation variances are naturally reduced., Comment: Under review

Transfer-based adversarial attacks raise a severe threat to real-world deep learning systems since they do not require access to target models. Adversarial training (AT), which is recognized as the strongest defense against white-box attacks, has also guaranteed high robustness to (black-box) transfer-based attacks. However, AT suffers from heavy computational overhead since it optimizes the adversarial examples during the whole training process. In this paper, we demonstrate that such heavy optimization is not necessary for AT against transfer-based attacks. Instead, a one-shot adversarial augmentation prior to training is sufficient, and we name this new defense paradigm Data-centric Robust Learning (DRL). Our experimental results show that DRL outperforms widely-used AT techniques (e.g., PGD-AT, TRADES, EAT, and FAT) in terms of black-box robustness and even surpasses the top-1 defense on RobustBench when combined with diverse data augmentations and loss regularizations. We also identify other benefits of DRL, for instance, the model generalization capability and robust fairness., Comment: 9 pages

Let $G$ be a finite simple graph with the vertex set $V$ and let $I_G$ be its edge ideal in the polynomial ring $S=\mathbb{K}[x_V]$. In this paper, we compute the depth and the Castelnuovo--Mumford regularity of $S/I_G$ when $G=G_1\circ G_2$ or $G=G_1* G_2$ is a graph obtained from Cohen-Macaulay bipartite graphs $G_1$, $G_2$ by $\circ$ operation or $*$ operation, respectively., Comment: arXiv admin note: text overlap with arXiv:2308.06010

Let $G$ be a finite simple graph on the vertex set $V$ and let $I_G$ denote its edge ideal in the polynomial ring $S=\mathbb{K}[x_V]$. In this paper, we compute the depth and the Castelnuovo--Mumford regularity of $S/I_G$ when $G=F_{k}^{W}(K_n)$ is a $k$-fan graph, or $G=G_1\circ G_2$ or $G=G_1* G_2$ is the graph obtained from fan graphs $G_1$, $G_2$ by $\circ$ operation or $*$ operation, respectively.

Adversarial training (AT) is widely considered the state-of-the-art technique for improving the robustness of deep neural networks (DNNs) against adversarial examples (AE). Nevertheless, recent studies have revealed that adversarially trained models are prone to unfairness problems, restricting their applicability. In this paper, we empirically observe that this limitation may be attributed to serious adversarial confidence overfitting, i.e., certain adversarial examples with overconfidence. To alleviate this problem, we propose HAM, a straightforward yet effective framework via adaptive Hard Adversarial example Mining.HAM concentrates on mining hard adversarial examples while discarding the easy ones in an adaptive fashion. Specifically, HAM identifies hard AEs in terms of their step sizes needed to cross the decision boundary when calculating loss value. Besides, an early-dropping mechanism is incorporated to discard the easy examples at the initial stages of AE generation, resulting in efficient AT. Extensive experimental results on CIFAR-10, SVHN, and Imagenette demonstrate that HAM achieves significant improvement in robust fairness while reducing computational cost compared to several state-of-the-art adversarial training methods. The code will be made publicly available.

Quantized neural networks (QNNs) have received increasing attention in resource-constrained scenarios due to their exceptional generalizability. However, their robustness against realistic black-box adversarial attacks has not been extensively studied. In this scenario, adversarial transferability is pursued across QNNs with different quantization bitwidths, which particularly involve unknown architectures and defense methods. Previous studies claim that transferability is difficult to achieve across QNNs with different bitwidths on the condition that they share the same architecture. However, we discover that under different architectures, transferability can be largely improved by using a QNN quantized with an extremely low bitwidth as the substitute model. We further improve the attack transferability by proposing \textit{quantization aware attack} (QAA), which fine-tunes a QNN substitute model with a multiple-bitwidth training objective. In particular, we demonstrate that QAA addresses the two issues that are commonly known to hinder transferability: 1) quantization shifts and 2) gradient misalignments. Extensive experimental results validate the high transferability of the QAA to diverse target models. For instance, when adopting the ResNet-34 substitute model on ImageNet, QAA outperforms the current best attack in attacking standardly trained DNNs, adversarially trained DNNs, and QNNs with varied bitwidths by 4.3\% $\sim$ 20.9\%, 8.7\% $\sim$ 15.5\%, and 2.6\% $\sim$ 31.1\% (absolute), respectively. In addition, QAA is efficient since it only takes one epoch for fine-tuning. In the end, we empirically explain the effectiveness of QAA from the view of the loss landscape. Our code is available at https://github.com/yyl-github-1896/QAA/, Comment: Accepted by IEEE Transactions on Information Forensics and Security in 2024

Magnetic ultrathin films grown on heavy metal substrates often exhibit rich spin structures due to the competition between various magnetic interactions such as Heisenberg exchange, Dzyaloshinskii-Moriya interaction and higher-order spin interactions. Here we employ spin-polarized scanning tunneling microscopy to study magnetic nano-skyrmion phase in Fe monolayer grown on Ir(111) substrate. Our observations show that the formation of nano-skyrmion lattice in the Fe/Ir(111) system depends sensitively on the growth conditions and various non-skyrmion spin states can be formed. Remarkably, the application of voltage pulses between the tip and the sample can trigger a non-skyrmion to skyrmion phase transition. The fact that nano-skyrmions can be created using voltage pulse indicates that the balance between the competing magnetic interactions can be affected by an external electric field, which is highly useful to design skyrmion-based spintronic devices with low energy consumption.

Room temperature grown Fe monolayer (ML) on the Ir(111) single crystal substrate has attracted great research interests as nano-skyrmion lattice can form under proper growth conditions. The formation of the nanoscale skyrmion, however, appears to be greatly affected by the diffusion length of the Fe adatoms on the Ir(111) surface. We made this observation by employing spin-polarized scanning tunneling microscopy to study skyrmion formation upon systematically changing the impurity density on the substrate surface prior to Fe deposition. Since the substrate surface impurities serve as pinning centers for Fe adatoms, the eventual size and shape of the Fe islands exhibit a direct correlation with the impurity density, which in turn determines whether skyrmion can be formed. Our observation indicates that skyrmion only forms when the impurity density is below 0.006/nm2, i.e., 12 nm averaged spacing between the neighboring defects. We verify the significance of Fe diffusion length by growing Fe on clean Ir(111) substrate at low temperature of 30 K, where no skyrmion was observed to form. Our findings signify the importance of diffusion of Fe atoms on the Ir(111) substrate, which affects the size, shape and lattice perfection of the Fe islands and thus the formation of skyrmion lattice.

An equigenerated monomial ideal $I$ in the polynomial ring $S= K[x_1,\ldots,x_n]$ is a Freiman ideal if $\mu(I^2)=\ell(I)\mu(I)-{\ell(I)\choose 2}$ where $\ell(I)$ is the analytic spread of $I$ and $\mu(I)$ is the number of minimal generators of $I$. In this paper, we classify certain classes of Borel type ideals, including Borel ideals with multiple Borel generators and principal $k$-Borel ideals, which are Freiman., Comment: 16 pages, 3 figure