Search

Large language models (LLMs) have been proposed as powerful tools for detecting software vulnerabilities, where task-specific fine-tuning is typically employed to provide vulnerability-specific knowledge to the LLMs for this purpose. However, traditional full-parameter fine-tuning is inefficient for modern, complex LLMs, which contain billions of parameters. Soft prompt tuning has been suggested as a more efficient alternative for fine-tuning LLMs in general cases. However, pure soft prompt tuning treats source code as plain text, losing structural information inherent in source code. Meanwhile, graph-enhanced soft prompt tuning methods, which aim to address this issue, are unable to preserve the rich semantic information within code graphs, as they are primarily designed for general graph-related tasks and focus more on adjacency information. They also fail to ensure computational efficiency while accounting for graph-text interactions. This paper, therefore, introduces a new code graph-enhanced, structure-aware soft prompt tuning method for vulnerability detection, referred to as CGP-Tuning. It employs innovative type-aware embeddings to capture the rich semantic information within code graphs, along with a novel and efficient cross-modal alignment module that achieves linear computational cost while incorporating graph-text interactions. The proposed CGP-Tuning is evaluated on the latest DiverseVul dataset and the most recent open-source code LLMs, CodeLlama and CodeGemma. Experimental results demonstrate that CGP-Tuning outperforms the best state-of-the-art method by an average of 3.5 percentage points in accuracy, without compromising its vulnerability detection capabilities for long source code., Comment: 14 pages, 5 figures

The marked power spectrum - a two-point correlation function of a transformed density field - has emerged as a promising tool for extracting cosmological information from the large-scale structure of the Universe. In this work, we present the first comprehensive analytical study of the marked power spectrum's sensitivity to primordial non-Gaussianity (PNG) of the non-local type. We extend previous effective field theory frameworks to incorporate PNG, developing a complete theoretical model that we validate against the Quijote simulation suite. Through a systematic Fisher analysis, we compare the constraining power of the marked power spectrum against traditional approaches combining the power spectrum and bispectrum (P+B). We explore different choices of mark parameters to evaluate their impact on parameter constraints, particularly focusing on equilateral and orthogonal PNG as well as neutrino masses. Our analysis shows that while marking up underdense regions yields optimal constraints in the low shot-noise regime, the marked power spectrum's performance for discrete tracers with BOSS-like number densities does not surpass that of P+B analysis at mildly non-linear scales ($k \lesssim 0.25 \,h/\text{Mpc}$). However, the marked approach offers several practical advantages, including simpler estimation procedures and potentially more manageable systematic effects. Our theoretical framework reveals how the marked power spectrum incorporates higher-order correlation information through terms resembling tree-level bispectra and power spectrum convolutions. This work establishes a robust foundation for applying marked statistics to future large-volume surveys., Comment: 18 pages, 7 figures

The recent detection of extended $\gamma$-ray emission around middle-aged pulsars is interpreted as inverse-Compton scattering of ambient photons by electron-positron pairs escaping the pulsar wind nebula, which are confined near the system by unclear mechanisms. This emerging population of $\gamma$-ray sources was first discovered at TeV energies and remains underexplored in the GeV range. To address this, we conducted a systematic search for extended sources along the Galactic plane using 14 years of Fermi-LAT data above 10 GeV, aiming to identify a number of pulsar halo candidates and extend our view to lower energies. The search covered the inner Galactic plane ($\lvert l\rvert\leq$ 100$^{\circ}$, $\lvert b\rvert\leq$ 1$^{\circ}$) and the positions of known TeV sources and bright pulsars, yielding broader astrophysical interest. We found 40 such sources, forming the Second Fermi Galactic Extended Sources Catalog (2FGES), most with 68% containment radii smaller than 1.0$^{\circ}$ and relatively hard spectra with photon indices below 2.5. We assessed detection robustness using field-specific alternative interstellar emission models and by inspecting significance maps. Noting 13 sources previously known as extended in the 4FGL-DR3 catalog and five dubious sources from complex regions, we report 22 newly detected extended sources above 10 GeV. Of these, 13 coincide with H.E.S.S., HAWC, or LHAASO sources; six coincide with bright pulsars (including four also coincident with TeV sources); six are associated with 4FGL point sources only; and one has no association in the scanned catalogs. Notably, six to eight sources may be related to pulsars as classical pulsar wind nebulae or pulsar halos., Comment: 43 pages, 17 figures, 11 tables

To achieve accurate and unbiased predictions, Machine Learning (ML) models rely on large, heterogeneous, and high-quality datasets. However, this could raise ethical and legal concerns regarding copyright and authorization aspects, especially when information is gathered from the Internet. With the rise of generative models, being able to track data has become of particular importance, especially since they may (un)intentionally replicate copyrighted contents. Therefore, this work proposes Copyright Audit via Prompts generation (CAP), a framework for automatically testing whether an ML model has been trained with unauthorized data. Specifically, we devise an approach to generate suitable keys inducing the model to reveal copyrighted contents. To prove its effectiveness, we conducted an extensive evaluation campaign on measurements collected in four IoT scenarios. The obtained results showcase the effectiveness of CAP, when used against both realistic and synthetic datasets.

This practical experience report explores Neural Machine Translation (NMT) models' capability to generate offensive security code from natural language (NL) descriptions, highlighting the significance of contextual understanding and its impact on model performance. Our study employs a dataset comprising real shellcodes to evaluate the models across various scenarios, including missing information, necessary context, and unnecessary context. The experiments are designed to assess the models' resilience against incomplete descriptions, their proficiency in leveraging context for enhanced accuracy, and their ability to discern irrelevant information. The findings reveal that the introduction of contextual data significantly improves performance. However, the benefits of additional context diminish beyond a certain point, indicating an optimal level of contextual information for model training. Moreover, the models demonstrate an ability to filter out unnecessary context, maintaining high levels of accuracy in the generation of offensive security code. This study paves the way for future research on optimizing context use in AI-driven code generation, particularly for applications requiring a high degree of technical precision such as the generation of offensive code., Comment: Accepted for publication at The 35th IEEE International Symposium on Software Reliability Engineering

We present the results of a set of cosmological N-body simulations with standard $\Lambda$CDM cosmology but characterized by a scale-dependent primordial non-Gaussianity of the local type featuring a power-law dependence of the $f_{\rm NL}^{\rm loc}(k)$ at large scales followed by a saturation to a constant value at smaller scales where non-linear growth leads to the formation of collapsed cosmic structures. Such models are built to ensure consistency with current Cosmic Microwave Background bounds on primordial non-Gaussianity yet allowing for large effects of the non-Gaussian statistics on the properties of non-linear structure formation. We show the impact of such scale-dependent non-Gaussian scenarios on a wide range of properties of the resulting cosmic structures, such as the non-linear matter power spectrum, the halo and sub-halo mass functions, the concentration-mass relation, the halo and void density profiles, and we highlight for the first time that some of these models might mimic the effects of Warm Dark Matter for several of such observables, Comment: 21 pages, 9 figures, 2 tables; to be submitted to JCAP

Solving the AC optimal power flow problem (AC-OPF) is critical to the efficient and safe planning and operation of power grids. Small efficiency improvements in this domain have the potential to lead to billions of dollars of cost savings, and significant reductions in emissions from fossil fuel generators. Recent work on data-driven solution methods for AC-OPF shows the potential for large speed improvements compared to traditional solvers; however, no large-scale open datasets for this problem exist. We present the largest readily-available collection of solved AC-OPF problems to date. This collection is orders of magnitude larger than existing readily-available datasets, allowing training of high-capacity data-driven models. Uniquely, it includes topological perturbations - a critical requirement for usage in realistic power grid operations. We hope this resource will spur the community to scale research to larger grid sizes with variable topology.

This paper discusses a simple and effective method for the summation of long sequences of floating point numbers. The method comprises two phases: an accumulation phase where the mantissas of the floating point numbers are added to accumulators indexed by the exponents and a reconstruction phase where the actual summation result is finalised. Various architectural details are given for both FPGAs and ASICs including fusing the operation with a multiplier, creating efficient MACs. Some results are presented for FPGAs, including a tensor core capable of multiplying and accumulating two 4x4 matrices of bfloat16 values every clock cycle using ~6,400 LUTs + 64 DSP48 in AMD FPGAs at 700+ MHz. The method is then extended to posits and logarithmic numbers.

In this work we study the properties of dissipatively stabilized steady states of noisy quantum algorithms, exploring the extent to which they can be well approximated as thermal distributions, and proposing methods to extract the effective temperature T. We study an algorithm called the Relaxational Quantum Eigensolver (RQE), which is one of a family of algorithms that attempt to find ground states and balance error in noisy quantum devices. In RQE, we weakly couple a second register of auxiliary "shadow" qubits to the primary system in Trotterized evolution, thus engineering an approximate zero-temperature bath by periodically resetting the auxiliary qubits during the algorithm's runtime. Balancing the infinite temperature bath of random gate error, RQE returns states with an average energy equal to a constant fraction of the ground state. We probe the steady states of this algorithm for a range of base error rates, using several methods for estimating both T and deviations from thermal behavior. In particular, we both confirm that the steady states of these systems are often well-approximated by thermal distributions, and show that the same resources used for cooling can be adopted for thermometry, yielding a fairly reliable measure of the temperature. These methods could be readily implemented in near-term quantum hardware, and for stabilizing and probing Hamiltonians where simulating approximate thermal states is hard for classical computers., Comment: 14 pages, 7 Figures

As the Windows OS stands out as one of the most targeted systems, the PowerShell language has become a key tool for malicious actors and cybersecurity professionals (e.g., for penetration testing). This work explores an uncharted domain in AI code generation by automatically generating offensive PowerShell code from natural language descriptions using Neural Machine Translation (NMT). For training and evaluation purposes, we propose two novel datasets with PowerShell code samples, one with manually curated descriptions in natural language and another code-only dataset for reinforcing the training. We present an extensive evaluation of state-of-the-art NMT models and analyze the generated code both statically and dynamically. Results indicate that tuning NMT using our dataset is effective at generating offensive PowerShell code. Comparative analysis against the most widely used LLM service ChatGPT reveals the specialized strengths of our fine-tuned models., Comment: 18th USENIX WOOT Conference on Offensive Technologies, GitHub Repo: https://github.com/dessertlab/powershell-offensive-code-generation

This paper starts with a simple lossless ~1.5:1 compression algorithm for the weights of the Large Language Model (LLM) Llama2 7B [1] that can be implemented in ~200 LUTs in AMD FPGAs, processing over 800 million bfloat16 numbers per second. This framework is then extended to variable precision, variable range, compressed numerical data types that are a user defined super set of both floats and posits [2]. The paper then discusses a simple hardware implementation of such format based on ANS (Asymmetrical Numeral Systems) [3] that acts as a bridge between this flexible data format and a computational engine while, at the same time, achieving bandwidth reduction. An example of a token factory using weight compression and sharing is also given.

Context: AI code generators are revolutionizing code writing and software development, but their training on large datasets, including potentially untrusted source code, raises security concerns. Furthermore, these generators can produce incomplete code snippets that are challenging to evaluate using current solutions. Objective: This research work introduces DeVAIC (Detection of Vulnerabilities in AI-generated Code), a tool to evaluate the security of AI-generated Python code, which overcomes the challenge of examining incomplete code. Method: We followed a methodological approach that involved gathering vulnerable samples, extracting implementation patterns, and creating regular expressions to develop the proposed tool. The implementation of DeVAIC includes a set of detection rules based on regular expressions that cover 35 Common Weakness Enumerations (CWEs) falling under the OWASP Top 10 vulnerability categories. Results: We utilized four popular AI models to generate Python code, which we then used as a foundation to evaluate the effectiveness of our tool. DeVAIC demonstrated a statistically significant difference in its ability to detect security vulnerabilities compared to the state-of-the-art solutions, showing an F1 Score and Accuracy of 94% while maintaining a low computational cost of 0.14 seconds per code snippet, on average. Conclusions: The proposed tool provides a lightweight and efficient solution for vulnerability detection even on incomplete code., Comment: Accepted for publication in the Information and Software Technology (IST) journal

Traditional software fault injection methods, while foundational, face limitations in adequately representing real-world faults, offering customization, and requiring significant manual effort and expertise. This paper introduces a novel methodology that harnesses the capabilities of Large Language Models (LLMs) augmented with Reinforcement Learning from Human Feedback (RLHF) to overcome these challenges. The usage of RLHF emphasizes an iterative refinement process, allowing testers to provide feedback on generated faults, which is then used to enhance the LLM's fault generation capabilities, ensuring the generation of fault scenarios that closely mirror actual operational risks. This innovative methodology aims to significantly reduce the manual effort involved in crafting fault scenarios as it allows testers to focus on higher-level testing strategies, hence paving the way to new possibilities for enhancing the dependability of software systems., Comment: 2024 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Disrupt

Optimal Power Flow (OPF) refers to a wide range of related optimization problems with the goal of operating power systems efficiently and securely. In the simplest setting, OPF determines how much power to generate in order to minimize costs while meeting demand for power and satisfying physical and operational constraints. In even the simplest case, power grid operators use approximations of the AC-OPF problem because solving the exact problem is prohibitively slow with state-of-the-art solvers. These approximations sacrifice accuracy and operational feasibility in favor of speed. This trade-off leads to costly "uplift payments" and increased carbon emissions, especially for large power grids. In the present work, we train a deep learning system (CANOS) to predict near-optimal solutions (within 1% of the true AC-OPF cost) without compromising speed (running in as little as 33--65 ms). Importantly, CANOS scales to realistic grid sizes with promising empirical results on grids containing as many as 10,000 buses. Finally, because CANOS is a Graph Neural Network, it is robust to changes in topology. We show that CANOS is accurate across N-1 topological perturbations of a base grid typically used in security-constrained analysis. This paves the way for more efficient optimization of more complex OPF problems which alter grid connectivity such as unit commitment, topology optimization and security-constrained OPF.

We investigate the Wavelet Scattering Transform (WST) as a tool for the study of Primordial non-Gaussianity (PNG) in Large Scale Structure (LSS), and compare its performance with that achievable via a joint analysis with power spectrum and bispectrum (P+B). We consider the three main primordial bispectrum shapes - local, equilateral and orthogonal - and produce Fisher forecast for the corresponding fNL amplitude parameters, jointly with standard cosmological parameters. We analyze simulations from the publicly available "Quijote" and "Quijote-png" N-body suites, studying both the dark matter and halo fields. We find that the WST outperforms the power spectrum alone on all parameters, both on the fNL's and on cosmological ones. In particular, on fNL_loc for halos, the improvement is about 27%. When B is combined with P, halo constraints from WST are weaker for fNL_loc (at ~ 15% level), but stronger for fNL_eq (~ 25%) and fNL_ortho (~ 28%). Our results show that WST, both alone and in combination with P+B, can improve the extraction of information on PNG from LSS data over the one attainable by a standard P+B analysis. Moreover, we identify a class of WST in which the origin of the extra information on PNG can be cleanly isolated., Comment: 25 pages, 13 figures

Bit Layer Multiplier Accumulator (BLMAC) is an efficient method to perform dot products without multiplications that exploits the bit level sparsity of the weights. A total of 1,980,000 low, high, band pass and band stop type I FIR filters were generated by systematically sweeping through the cut off frequencies and by varying the number of taps from 55 to 255. After their coefficients were quantized to 16 bits, applying the filter using a BLMAC required, on average, from ~123.3 to ~513.6 additions, depending on the number of taps. A BLMAC dot product machine, specialised for 127 taps FIR filters, was designed for AMD FPGAs. The design footprint is ~110 LUTs, including coefficient and sample storage and is able to apply the filter in ~232 clock cycles on average. This implies a filtering rate of 1.4-3.4 Msamples/s, depending on the FPGA family.

We apply a suite of different estimators to the Quijote-PNG halo catalogues to find the best approach to constrain Primordial non-Gaussianity (PNG) at non-linear cosmological scales, up to $k_{\rm max} = 0.5 \, h\,{\rm Mpc}^{-1}$. The set of summary statistics considered in our analysis includes the power spectrum, bispectrum, halo mass function, marked power spectrum, and marked modal bispectrum. Marked statistics are used here for the first time in the context of PNG study. We perform a Fisher analysis to estimate their cosmological information content, showing substantial improvements when marked observables are added to the analysis. Starting from these summaries, we train deep neural networks (NN) to perform likelihood-free inference of cosmological and PNG parameters. We assess the performance of different subsets of summary statistics; in the case of $f_\mathrm{NL}^\mathrm{equil}$, we find that a combination of the power spectrum and a suitable marked power spectrum outperforms the combination of power spectrum and bispectrum, the baseline statistics usually employed in PNG analysis. A minimal pipeline to analyse the statistics we identified can be implemented either with our ML algorithm or via more traditional estimators, if these are deemed more reliable., Comment: 13 pages, 10 figures

Recent advances of artificial intelligence (AI) code generators are opening new opportunities in software security research, including misuse by malicious actors. We review use cases for AI code generators for security and introduce an evaluation benchmark., Comment: Dataset available at: https://github.com/dessertlab/violent-python

Missing data are frequently observed by practitioners and researchers in the building energy modeling community. In this regard, advanced data-driven solutions, such as Deep Learning methods, are typically required to reflect the non-linear behavior of these anomalies. As an ongoing research question related to Deep Learning, a model's applicability to limited data settings can be explored by introducing prior knowledge in the network. This same strategy can also lead to more interpretable predictions, hence facilitating the field application of the approach. For that purpose, the aim of this paper is to propose the use of Physics-informed Denoising Autoencoders (PI-DAE) for missing data imputation in commercial buildings. In particular, the presented method enforces physics-inspired soft constraints to the loss function of a Denoising Autoencoder (DAE). In order to quantify the benefits of the physical component, an ablation study between different DAE configurations is conducted. First, three univariate DAEs are optimized separately on indoor air temperature, heating, and cooling data. Then, two multivariate DAEs are derived from the previous configurations. Eventually, a building thermal balance equation is coupled to the last multivariate configuration to obtain PI-DAE. Additionally, two commonly used benchmarks are employed to support the findings. It is shown how introducing physical knowledge in a multivariate Denoising Autoencoder can enhance the inherent model interpretability through the optimized physics-based coefficients. While no significant improvement is observed in terms of reconstruction error with the proposed PI-DAE, its enhanced robustness to varying rates of missing data and the valuable insights derived from the physics-based coefficients create opportunities for wider applications within building systems and the built environment., Comment: Accepted for publication in Energy and Buildings

Primordial non-Gaussianity of the local type induces a strong scale-dependent bias on the clustering of halos in the late-time Universe. This signature is particularly promising to provide constraints on the non-Gaussianity parameter $f_{\rm NL}$ from galaxy surveys, as the bias amplitude grows with scale and becomes important on large, linear scales. However, there is a well-known degeneracy between the real prize, the $f_{\rm NL}$ parameter, and the (non-Gaussian) assembly bias i.e., the halo formation history-dependent contribution to the amplitude of the signal, which could seriously compromise the ability of large-scale structure surveys to constrain $f_{\rm NL}$. We show how the assembly bias can be modeled and constrained, thus almost completely recovering the power of galaxy surveys to competitively constrain primordial non-Gaussianity. In particular, studying hydrodynamical simulations, we find that a proxy for the halo properties that determine assembly bias can be constructed from photometric properties of galaxies. Using a prior on the assembly bias guided by this proxy degrades the statistical errors on $f_{\rm NL}$ only mildly compared to an ideal case where the assembly bias is perfectly known. The systematic error on $f_{\rm NL}$ that the proxy induces can be safely kept under control., Comment: 30 pages, 13 figures. v2: minor updates to match accepted version