Search

Model merging has emerged as a crucial technique in Deep Learning, enabling the integration of multiple models into a unified system while preserving performance and scalability. In this respect, the compositional properties of low-rank adaptation techniques (e.g., LoRA) have proven beneficial, as simple averaging LoRA modules yields a single model that mostly integrates the capabilities of all individual modules. Building on LoRA, we take a step further by imposing that the merged model matches the responses of all learned modules. Solving this objective in closed form yields an indeterminate system with A and B as unknown variables, indicating the existence of infinitely many closed-form solutions. To address this challenge, we introduce LoRM, an alternating optimization strategy that trains one LoRA matrix at a time. This allows solving for each unknown variable individually, thus finding a unique solution. We apply our proposed methodology to Federated Class-Incremental Learning (FCIL), ensuring alignment of model responses both between clients and across tasks. Our method demonstrates state-of-the-art performance across a range of FCIL scenarios.

The extreme heat fluxes in the divertor region of tokamaks may require an alternative to solid plasma-facing components, for the extraction of heat and the protection of the surrounding walls. Flowing liquid metals are proposed as an alternative, but raise additional challenges that require investigation and numerical simulations. Free surface designs are desirable for plasma-facing components (PFCs), but steady flow profiles and surface stability must be ensured to limit undesirable interactions with the plasma. Previous studies have mainly used steady-state, 2D, or simplified models for internal flows and have not been able to adequately model free-surface liquid metal (LM) experiments. Therefore, FreeMHD has been recently developed as an open-source magnetohydrodynamics (MHD) solver for free-surface electrically conductive flows subject to a strong external magnetic field. The FreeMHD solver computes incompressible free-surface flows with multi-region coupling for the investigation of MHD phenomena involving fluid and solid domains. The model utilizes the finite-volume OpenFOAM framework under the low magnetic Reynolds number approximation. FreeMHD is validated using analytical solutions for the velocity profiles of closed channel flows with various Hartmann numbers and wall conductance ratios. Next, experimental measurements are then used to verify FreeMHD, through a series of cases involving dam breaking, 3D magnetic fields, and free-surface LM flows. These results demonstrate that FreeMHD is a reliable tool for the design of LM systems under free surface conditions at the reactor scale. Furthermore, it is flexible, computationally inexpensive, and can be used to solve fully 3D transient MHD flows.

Complex human activity recognition (CHAR) remains a pivotal challenge within ubiquitous computing, especially in the context of smart environments. Existing studies typically require meticulous labeling of both atomic and complex activities, a task that is labor-intensive and prone to errors due to the scarcity and inaccuracies of available datasets. Most prior research has focused on datasets that either precisely label atomic activities or, at minimum, their sequence approaches that are often impractical in real world settings.In response, we introduce VCHAR (Variance-Driven Complex Human Activity Recognition), a novel framework that treats the outputs of atomic activities as a distribution over specified intervals. Leveraging generative methodologies, VCHAR elucidates the reasoning behind complex activity classifications through video-based explanations, accessible to users without prior machine learning expertise. Our evaluation across three publicly available datasets demonstrates that VCHAR enhances the accuracy of complex activity recognition without necessitating precise temporal or sequential labeling of atomic activities. Furthermore, user studies confirm that VCHAR's explanations are more intelligible compared to existing methods, facilitating a broader understanding of complex activity recognition among non-experts.

Reinforcement Learning from Human Feedback (RLHF) is popular in large language models (LLMs), whereas traditional Reinforcement Learning (RL) often falls short. Current autonomous driving methods typically utilize either human feedback in machine learning, including RL, or LLMs. Most feedback guides the car agent's learning process (e.g., controlling the car). RLHF is usually applied in the fine-tuning step, requiring direct human "preferences," which are not commonly used in optimizing autonomous driving models. In this research, we innovatively combine RLHF and LLMs to enhance autonomous driving safety. Training a model with human guidance from scratch is inefficient. Our framework starts with a pre-trained autonomous car agent model and implements multiple human-controlled agents, such as cars and pedestrians, to simulate real-life road environments. The autonomous car model is not directly controlled by humans. We integrate both physical and physiological feedback to fine-tune the model, optimizing this process using LLMs. This multi-agent interactive environment ensures safe, realistic interactions before real-world application. Finally, we will validate our model using data gathered from real-life testbeds located in New Jersey and New York City.

Federated Learning (FL) aims at unburdening the training of deep models by distributing computation across multiple devices (clients) while safeguarding data privacy. On top of that, Federated Continual Learning (FCL) also accounts for data distribution evolving over time, mirroring the dynamic nature of real-world environments. While previous studies have identified Catastrophic Forgetting and Client Drift as primary causes of performance degradation in FCL, we shed light on the importance of Incremental Bias and Federated Bias, which cause models to prioritize classes that are recently introduced or locally predominant, respectively. Our proposal constrains both biases in the last layer by efficiently finetuning a pre-trained backbone using learnable prompts, resulting in clients that produce less biased representations and more biased classifiers. Therefore, instead of solely relying on parameter aggregation, we leverage generative prototypes to effectively balance the predictions of the global model. Our method significantly improves the current State Of The Art, providing an average increase of +7.8% in accuracy. Code to reproduce the results is provided in the suppl. material.

General purpose Large Language Models (LLM) such as the Generative Pretrained Transformer (GPT) and Large Language Model Meta AI (LLaMA) have attracted much attention in recent years. There is strong evidence that these models can perform remarkably well in various natural language processing tasks. However, how to leverage them to approach domain-specific use cases and drive value remains an open question. In this work, we focus on a specific use case, pharmaceutical manufacturing investigations, and propose that leveraging historical records of manufacturing incidents and deviations in an organization can be beneficial for addressing and closing new cases, or de-risking new manufacturing campaigns. Using a small but diverse dataset of real manufacturing deviations selected from different product lines, we evaluate and quantify the power of three general purpose LLMs (GPT-3.5, GPT-4, and Claude-2) in performing tasks related to the above goal. In particular, (1) the ability of LLMs in automating the process of extracting specific information such as root cause of a case from unstructured data, as well as (2) the possibility of identifying similar or related deviations by performing semantic search on the database of historical records are examined. While our results point to the high accuracy of GPT-4 and Claude-2 in the information extraction task, we discuss cases of complex interplay between the apparent reasoning and hallucination behavior of LLMs as a risk factor. Furthermore, we show that semantic search on vector embedding of deviation descriptions can be used to identify similar records, such as those with a similar type of defect, with a high level of accuracy. We discuss further improvements to enhance the accuracy of similar record identification., Comment: 13 pages, 3 figures

The increasing cloudification and softwarization of networks foster the interplay among multiple independently managed deployments. An appealing reason for such an interplay lies in distributed Machine Learning (ML), which allows the creation of robust ML models by leveraging collective intelligence and computational power. In this paper, we study the application of the two cornerstones of distributed learning, namely Federated Learning (FL) and Knowledge Distillation (KD), on the Wi-Fi Access Point (AP) load prediction use case. The analysis conducted in this paper is done on a dataset that contains real measurements from a large Wi-Fi campus network, which we use to train the ML model under study based on different strategies. Performance evaluation includes relevant aspects for the suitability of distributed learning operation in real use cases, including the predictive performance, the associated communication overheads, or the energy consumption. In particular, we prove that distributed learning can improve the predictive accuracy centralized ML solutions by up to 93% while reducing the communication overheads and the energy cost by 80%.

Foley sound generation, the art of creating audio for multimedia, has recently seen notable advancements through text-conditioned latent diffusion models. These systems use multimodal text-audio representation models, such as Contrastive Language-Audio Pretraining (CLAP), whose objective is to map corresponding audio and text prompts into a joint embedding space. AudioLDM, a text-to-audio model, was the winner of the DCASE2023 task 7 Foley sound synthesis challenge. The winning system fine-tuned the model for specific audio classes and applied a post-filtering method using CLAP similarity scores between output audio and input text at inference time, requiring the generation of extra samples, thus reducing data generation efficiency. We introduce a new loss term to enhance Foley sound generation in AudioLDM without post-filtering. This loss term uses a new module based on the CLAP mode-Latent CLAP encode-to align the latent diffusion output with real audio in a shared CLAP embedding space. Our experiments demonstrate that our method effectively reduces the Frechet Audio Distance (FAD) score of the generated audio and eliminates the need for post-filtering, thus enhancing generation efficiency.

In our pursuit of quantum supremacy during the NISQ era, this research introduces a novel approach rooted in the Quantum Approximate Optimization Algorithm (QAOA) framework to address the Traveling Salesman Problem (TSP). By strategically reducing the requisite qubit count from $n^2$ to $n\log_{2} (n)$, our QAOA-based algorithm not only contributes to the ongoing discourse on qubit efficiency but also demonstrates improved performance based on established metrics, underscoring its potential for achieving NISQ-era supremacy in solving real-world optimization challenges., Comment: 4+2 pages, 1 figure

The construction industry is a vital sector of the global economy, but it faces many productivity challenges in various processes, such as design, planning, procurement, inspection, and maintenance. Generative artificial intelligence (AI), which can create novel and realistic data or content, such as text, image, video, or code, based on some input or prior knowledge, offers innovative and disruptive solutions to address these challenges. However, there is a gap in the literature on the current state, opportunities, and challenges of generative AI in the construction industry. This study aims to fill this gap by providing a state-of-the-art analysis of generative AI in construction, with three objectives: (1) to review and categorize the existing and emerging generative AI opportunities and challenges in the construction industry; (2) to propose a framework for construction firms to build customized generative AI solutions using their own data, comprising steps such as data collection, dataset curation, training custom large language model (LLM), model evaluation, and deployment; and (3) to demonstrate the framework via a case study of developing a generative model for querying contract documents. The results show that retrieval augmented generation (RAG) improves the baseline LLM by 5.2, 9.4, and 4.8% in terms of quality, relevance, and reproducibility. This study provides academics and construction professionals with a comprehensive analysis and practical framework to guide the adoption of generative AI techniques to enhance productivity, quality, safety, and sustainability across the construction industry., Comment: 74 pages, 11 figures, 20 tables

Emulating chip functionality before silicon production is crucial, especially with the increasing prevalence of RISC-V-based designs. FPGAs are promising candidates for such purposes due to their high-speed and reconfigurable architecture. In this paper, we introduce our Makinote, an FPGA-based Cluster platform, hosted at Barcelona Supercomputing Center (BSC-CNS), which is composed of a large number of FPGAs (in total 96 AMD/Xilinx Alveo U55c) to emulate massive size RTL designs (up to 750M ASIC cells). In addition, we introduce our FPGA shell as a powerful tool to facilitate the utilization of such a large FPGA cluster with minimal effort needed by the designers. The proposed FPGA shell provides an easy-to-use interface for the RTL developers to rapidly port such design into several FPGAs by automatically connecting to the necessary ports, e.g., PCIe Gen4, DRAM (DDR4 and HBM), ETH10g/100g. Moreover, specific drivers for exploiting RISC-V based architectures are provided within the set of tools associated with the FPGA shell. We release the tool online for further extensions. We validate the efficiency of our hardware platform (i.e., FPGA cluster) and the software tool (i.e., FPGA Shell) by emulating a RISC-V processor and experimenting HPC Challenge application running on 32 FPGAs. Our results demonstrate that the performance improves by 8 times over the single-FPGA case., Comment: 7 pages, 5 figures, presented in Rapid Simulation and Performance Evaluation for Design 2024 (RAPIDO24) and published in ACM Proceedings of Rapid Simulation and Performance Evaluation for Design

Quantum discord is a form of correlation that is defined as the difference between quantum and classical mutual information of two parties. Due to the optimization involved in the definition of classical mutual information of quantum systems, calculating and distinguishing between discordant and non-discordant states is not a trivial task. Additionally, complete tomography of a quantum state is the prerequisite for the calculation of its quantum discord, and it is indeed resource consuming. Here, by using the relation between the kernels of the convolutional layers of an artificial neural network and the expectation value of operators in quantum mechanical measurements, we design a Convolutional Neural Network (CNN) that uses 16 kernels to completely distinguish between the discordant and non-discordant general two-qubit states. We have also designed a Branching Convolutional Neural Network (BCNN) that can effectively detect quantum discord. Our BCNN achieves an accuracy of around 85% or 99%, by utilizing only 5 or 8 kernels, respectively. Our results show that to detect the existence of quantum discord up to the desired accuracy, instead of complete tomography, one can use suitable quantum circuits to directly measure the expectation values of the kernels, and then a fully connected network will solve the detection problem., Comment: Minor revisions, two references added

Hybrid Monte Carlo (HMC) simulations of lattice gauge theories with fermionic matter rely on the invertibility of the lattice Dirac operator. Near-zero modes of the latter can therefore significantly slow down the update algorithm and cause instabilities. This is in particular a problem when dealing with massless fermions. Homogeneous temporal Dirichlet boundary conditions can be used to remove zero modes from massless lattice Dirac operators, but the standard implementation of these boundary conditions can cause severe finite-volume cutoff effects in regions of parameter space where the physics at the ultraviolet (UV) cutoff scale is dominated by the fermionic instead of the gauge action. In lattice quantum chromodynamics (QCD) this is usually not an issue, as the gauge action dominates the UV physics and the problem does not show up. In studies of beyond standard model (BSM) theories, on the other hand, the finite-volume artifacts can be severe. We have identified the origin of these IR cutoff effects and propose a simple improvement on the homogeneous temporal Dirichlet boundary conditions to prevent them. We demonstrate the benefits of using our improved boundary conditions at the example of SU(2) lattice gauge theory with $N_f=24$ massless Wilson-clover flavors. Due to the large number of fermions in this theory, the boundary-related finite volume artifacts are particularly strong, and the effect from switching from the normal to our improved homogeneous Dirichlet boundary conditions is therefore distinct., Comment: 9 pages, 11 figures

We experimentally demonstrate the effects of read disturbance (RowHammer and RowPress) and uncover the inner workings of undocumented read disturbance defense mechanisms in High Bandwidth Memory (HBM). Detailed characterization of six real HBM2 DRAM chips in two different FPGA boards shows that (1) the read disturbance vulnerability significantly varies between different HBM2 chips and between different components (e.g., 3D-stacked channels) inside a chip, (2) DRAM rows at the end and in the middle of a bank are more resilient to read disturbance, (3) fewer additional activations are sufficient to induce more read disturbance bitflips in a DRAM row if the row exhibits the first bitflip at a relatively high activation count, (4) a modern HBM2 chip implements undocumented read disturbance defenses that track potential aggressor rows based on how many times they are activated. We describe how our findings could be leveraged to develop more powerful read disturbance attacks and more efficient defense mechanisms. We open source all our code and data to facilitate future research at https://github.com/CMU-SAFARI/HBM-Read-Disturbance., Comment: To appear in DSN 2024

Enterprise Wi-Fi networks can greatly benefit from Artificial Intelligence and Machine Learning (AI/ML) thanks to their well-developed management and operation capabilities. At the same time, AI/ML-based traffic/load prediction is one of the most appealing data-driven solutions to improve the Wi-Fi experience, either through the enablement of autonomous operation or by boosting troubleshooting with forecasted network utilization. In this paper, we study the suitability and feasibility of adopting AI/ML-based load prediction in practical enterprise Wi-Fi networks. While leveraging AI/ML solutions can potentially contribute to optimizing Wi-Fi networks in terms of energy efficiency, performance, and reliability, their effective adoption is constrained to aspects like data availability and quality, computational capabilities, and energy consumption. Our results show that hardware-constrained AI/ML models can potentially predict network load with less than 20% average error and 3% 85th-percentile error, which constitutes a suitable input for proactively driving Wi-Fi network optimization.

We formulate Vojta's conjecture for smooth weighted projective varieties, weighted multiplier ideal sheaves, and weighted log pairs and prove that all three versions of the conjecture are equivalent. In the process, we introduce generalized weighted general common divisors and express them as heights of weighted projective spaces blown-up relative to an exceptional divisor. Furthermore, we prove that assuming Vojta's conjecture for weighted projective varieties one can bound the $\log {\rm h_{wgcd}}$ for any subvariety of codimension $\geq 2$ and a finite set of places $S$. An analogue result is proved for weighted homogeneous polynomials with integer coefficients.

Lattice gauge field theories may suffer from unphysical "bulk" phase transitions at strong lattice gauge coupling. We introduce a one-parameter family of lattice SU(N) gauge actions which, when used in combination with an HMC update algorithm, prevents the appearance of the bulk phase transition. We briefly discuss the (presumed) mechanism behind the prevention of the bulk transition and present test results for different SU(N) gauge groups., Comment: 14 pages, 8 figures, contains a refined discussion of the working principle of bulk-prevention in SU(N) gauge theories and a new appendix on the relation between bulk configurations and magnetic monopoles