Search

The demand for artificially generated data for the development, training and testing of new algorithms is omnipresent. Quantum computing (QC), does offer the hope that its inherent probabilistic functionality can be utilised in this field of generative artificial intelligence. In this study, we use quantum-classical hybrid generative adversarial networks (QuGANs) to artificially generate graphs of shipping routes. We create a training dataset based on real shipping data and investigate to what extent QuGANs are able to learn and reproduce inherent distributions and geometric features of this data. We compare hybrid QuGANs with classical Generative Adversarial Networks (GANs), with a special focus on their parameter efficiency. Our results indicate that QuGANs are indeed able to quickly learn and represent underlying geometric properties and distributions, although they seem to have difficulties in introducing variance into the sampled data. Compared to classical GANs of greater size, measured in the number of parameters used, some QuGANs show similar result quality. Our reference to concrete use cases, such as the generation of shipping data, provides an illustrative example and demonstrate the potential and diversity in which QC can be used.

Reinforcement Learning (RL) policies are designed to predict actions based on current observations to maximize cumulative future rewards. In real-world applications (i.e., non-simulated environments), sensors are essential for measuring the current state and providing the observations on which RL policies rely to make decisions. A significant challenge in deploying RL policies in real-world scenarios is handling sensor dropouts, which can result from hardware malfunctions, physical damage, or environmental factors like dust on a camera lens. A common strategy to mitigate this issue is the use of backup sensors, though this comes with added costs. This paper explores the optimization of backup sensor configurations to maximize expected returns while keeping costs below a specified threshold, C. Our approach uses a second-order approximation of expected returns and includes penalties for exceeding cost constraints. We then optimize this quadratic program using Tabu Search, a meta-heuristic algorithm. The approach is evaluated across eight OpenAI Gym environments and a custom Unity-based robotic environment (RobotArmGrasping). Empirical results demonstrate that our quadratic program effectively approximates real expected returns, facilitating the identification of optimal sensor configurations., Comment: Accepted at ICAART conference 2025

In sequential decision-making environments, the primary approaches for training agents are Reinforcement Learning (RL) and Imitation Learning (IL). Unlike RL, which relies on modeling a reward function, IL leverages expert demonstrations, where an expert policy $\pi_e$ (e.g., a human) provides the desired behavior. Formally, a dataset $D$ of state-action pairs is provided: $D = {(s, a = \pi_e(s))}$. A common technique within IL is Behavior Cloning (BC), where a policy $\pi(s) = a$ is learned through supervised learning on $D$. Further improvements can be achieved by using an ensemble of $N$ individually trained BC policies, denoted as $E = {\pi_i(s)}{1 \leq i \leq N}$. The ensemble's action $a$ for a given state $s$ is the aggregated output of the $N$ actions: $a = \frac{1}{N} \sum{i} \pi_i(s)$. This paper addresses the issue of increasing action differences -- the observation that discrepancies between the $N$ predicted actions grow in states that are underrepresented in the training data. Large action differences can result in suboptimal aggregated actions. To address this, we propose a method that fosters greater alignment among the policies while preserving the diversity of their computations. This approach reduces action differences and ensures that the ensemble retains its inherent strengths, such as robustness and varied decision-making. We evaluate our approach across eight diverse environments, demonstrating a notable decrease in action differences and significant improvements in overall performance, as measured by mean episode returns., Comment: Accepted at ICAART 2025

According to the Strong Lottery Ticket Hypothesis, every sufficiently large neural network with randomly initialized weights contains a sub-network which - still with its random weights - already performs as well for a given task as the trained super-network. We present the first approach based on a genetic algorithm to find such strong lottery ticket sub-networks without training or otherwise computing any gradient. We show that, for smaller instances of binary classification tasks, our evolutionary approach even produces smaller and better-performing lottery ticket networks than the state-of-the-art approach using gradient information., Comment: 12 pages, 7 figures, 5 tables, accepted for publication at the 16th International Joint Conference on Computational Intelligence (IJCCI 2024)

We have developed a new set of Application-Specific Integrated Circuits (ASICs) of the TARGET family (CTC and CT5TEA), designed for the readout of signals from photosensors in cameras of Imaging Atmospheric Cherenkov Telescopes (IACTs) for ground-based gamma-ray astronomy. We present the performance and design details. Both ASICs feature 16 channels, with CTC being a Switched-Capacitor Array (SCA) sampler at 0.5 to 1 GSa/s with a 16,384 sample deep storage buffer, including the functionality to digitize full waveforms at arbitrary times. CT5TEA is its companion trigger ASIC (though may be used on its own), which provides trigger information for the analog sum of four (and 16) adjacent channels. Since sampling and triggering takes place in two separate ASICs, the noise due to interference from the SCA is suppressed, and allows a minimal trigger threshold of $\leq$ 2.5 mV (0.74 photo electrons (p.e.)) with a trigger noise of $\leq$ 0.5 mV (0.15 p.e.). For CTC, a maximal input voltage range from $-$0.5 V up to 1.7 V is achieved with an effective bit range of $>$ 11.6 bits and a baseline noise of 0.7 mV. The cross-talk improved to $\leq$ 1% over the whole $-$3 dB bandwidth of 220 MHz and even down to 0.2% for 1.5 V pulses of 10 ns width. Not only is the performance presented, but a temperature-stable calibration routine for pulse mode operation is introduced and validated. The resolution is found to be $\sim$ 2.5% at 33.7 mV (10 p.e.) and $\leq$ 0.3% at 337 mV (100 p.e.) with an integrated non-linearity of $<$ 1.6 mV. Developed for the Small-Sized Telescope (SST) and Schwarzschild-Couder Telescope (SCT) cameras of the Cherenkov Telescope Array Observatory (CTAO), CTC and CT5TEA are deployed for both prototypes and shall be integrated into the final versions., Comment: 18 pages, 26 figures, 1 table

A central challenge in quantum machine learning is the design and training of parameterized quantum circuits (PQCs). Much like in deep learning, vanishing gradients pose significant obstacles to the trainability of PQCs, arising from various sources. One such source is the presence of non-local loss functions, which require the measurement of a large subset of qubits involved. To address this issue and facilitate parameter training for quantum applications using global loss functions, we propose Sequential Hamiltonian Assembly (SHA). SHA iteratively approximates the loss by assembling it from local components. To further demonstrate the feasibility of our approach, we extend our previous case study by introducing a new partitioning strategy, a new merger between QAOA and SHA, and an evaluation of SHA onto the Max-Cut optimization problem. Simulation results show that SHA outperforms conventional parameter training by 43.89% and the empirical state-of-the-art, Layer-VQE by 29.08% in the mean accuracy for Max-Cut. This paves the way for locality-aware learning techniques, mitigating vanishing gradients for a large class of practically relevant problems., Comment: This is an extended version of our previously published article arXiv:2312.05552, additionally providing a new partitioning strategy, a new merger between QAOA and SHA, and an evaluation of SHA in the Max-Cut optimization problem

Emergent effects can arise in multi-agent systems (MAS) where execution is decentralized and reliant on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these effects, we identify misalignments between the global inherent specification (the true specification) and its local approximation (such as the configuration of different reward components or observations). Using established safety terminology, we develop a framework to understand these emergent effects. To showcase the resulting implications, we use two broadly configurable exemplary gridworld scenarios, where insufficient specification leads to unintended behavior deviations when derived independently. Recognizing that a global adaptation might not always be feasible, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving the system's alignment and reducing the risk of emergent failures., Comment: 18 pages, 3 figures, accepted for publication at the International Symposium on Leveraging Applications of Formal Methods (ISoLA 2024)

Quantum Reinforcement Learning (QRL) offers potential advantages over classical Reinforcement Learning, such as compact state space representation and faster convergence in certain scenarios. However, practical benefits require further validation. QRL faces challenges like flat solution landscapes, where traditional gradient-based methods are inefficient, necessitating the use of gradient-free algorithms. This work explores the integration of metaheuristic algorithms -- Particle Swarm Optimization, Ant Colony Optimization, Tabu Search, Genetic Algorithm, Simulated Annealing, and Harmony Search -- into QRL. These algorithms provide flexibility and efficiency in parameter optimization. Evaluations in $5\times5$ MiniGrid Reinforcement Learning environments show that, all algorithms yield near-optimal results, with Simulated Annealing and Particle Swarm Optimization performing best. In the Cart Pole environment, Simulated Annealing, Genetic Algorithms, and Particle Swarm Optimization achieve optimal results, while the others perform slightly better than random action selection. These findings demonstrate the potential of Particle Swarm Optimization and Simulated Annealing for efficient QRL learning, emphasizing the need for careful algorithm selection and adaptation., Comment: Accepted at QCE24 - QCRL24 Workshop

The Quantum Approximate Optimization Algorithm (QAOA) is a prominent quantum algorithm designed to find approximate solutions to combinatorial optimization problems, which are challenging for classical computers. In the current era, where quantum hardware is constrained by noise and limited qubit availability, simulating the QAOA remains essential for research. However, existing state-of-the-art simulation frameworks suffer from long execution times or lack comprehensive functionality, usability, and versatility, often requiring users to implement essential features themselves. Additionally, these frameworks are primarily restricted to Python, limiting their use in safer and faster languages like Rust, which offer, e.g., advanced parallelization capabilities. In this paper, we develop a GPU accelerated QAOA simulation framework utilizing the NVIDIA CUDA toolkit. This framework offers a complete interface for QAOA simulations, enabling the calculation of (exact) expectation values, direct access to the statevector, fast sampling, and high-performance optimization methods using an advanced state-of-the-art gradient calculation technique. The framework is designed for use in Python and Rust, providing flexibility for integration into a wide range of applications, including those requiring fast algorithm implementations leveraging QAOA at its core. The new framework's performance is rigorously benchmarked on the MaxCut problem and compared against the current state-of-the-art general-purpose quantum circuit simulation frameworks Qiskit and Pennylane as well as the specialized QAOA simulation tool QOKit. Our evaluation shows that our approach outperforms the existing state-of-the-art solutions in terms of runtime up to multiple orders of magnitude. Our implementation is publicly available at https://github.com/JFLXB/cuaoa and Zenodo., Comment: 6 pages, 3 figures

A common way of solving satisfiability instances with quantum methods is to transform these instances into instances of QUBO, which in itself is a potentially difficult and expensive task. State-of-the-art transformations from MAX-3SAT to QUBO currently work by mapping clauses of a 3SAT formula associated with the MAX-3SAT instance to an instance of QUBO and combining the resulting QUBOs into a single QUBO instance representing the whole MAX-3SAT instance. As creating these transformations is currently done manually or via exhaustive search methods and, therefore, algorithmically inefficient, we see potential for including search-based optimization. In this paper, we propose two methods of using evolutionary algorithms to automatically create QUBO representations of MAX-3SAT problems. We evaluate our created QUBOs on 500 and 1000-clause 3SAT formulae and find competitive performance to state-of-the-art baselines when using both classical and quantum annealing solvers.

Recent advances in multi-agent systems (MAS) have shown that incorporating peer incentivization (PI) mechanisms vastly improves cooperation. Especially in social dilemmas, communication between the agents helps to overcome sub-optimal Nash equilibria. However, incentivization tokens need to be carefully selected. Furthermore, real-world applications might yield increased privacy requirements and limited exchange. Therefore, we extend the PI protocol for mutual acknowledgment token exchange (MATE) and provide additional analysis on the impact of the chosen tokens. Building upon those insights, we propose mutually endorsed distributed incentive acknowledgment token exchange (MEDIATE), an extended PI architecture employing automatic token derivation via decentralized consensus. Empirical results show the stable agreement on appropriate tokens yielding superior performance compared to static tokens and state-of-the-art approaches in different social dilemma environments with various reward distributions., Comment: 12 pages, 5 figures

To enhance the interpretability of Reinforcement Learning (RL), we propose Revealing Evolutionary Action Consequence Trajectories (REACT). In contrast to the prevalent practice of validating RL models based on their optimal behavior learned during training, we posit that considering a range of edge-case trajectories provides a more comprehensive understanding of their inherent behavior. To induce such scenarios, we introduce a disturbance to the initial state, optimizing it through an evolutionary algorithm to generate a diverse population of demonstrations. To evaluate the fitness of trajectories, REACT incorporates a joint fitness function that encourages both local and global diversity in the encountered states and chosen actions. Through assessments with policies trained for varying durations in discrete and continuous environments, we demonstrate the descriptive power of REACT. Our results highlight its effectiveness in revealing nuanced aspects of RL models' behavior beyond optimal performance, thereby contributing to improved interpretability., Comment: 12 pages, 12 figures

This paper investigates the thermoelectric properties of solid polymer electrolytes (SPE) containing lithium bis(trifluoromethanesulfonyl)imide (LiTFSI) and sodium bis(trifluoromethanesulfonyl)imide (NaTFSI) salts, along with carbon-based additives of various dimensionalities. Increasing salt concentration leads to higher Seebeck coefficients as a result of the increasing number of free charge carriers and additional, superimposed effects by ion-ion and ion-polymer interactions. NaTFSI-based electrolytes exhibit negative Seebeck coefficients (up to $S = -1.5\,\mathrm{mV\,K^{-1}}$), indicating dominant mobility of $\mathrm{TFSI^-}$ ions. Quasi-one-dimensional carbon nanotubes (CNTs) increase the Seebeck coefficient by a factor of 3. Planar, two-dimensional graphite flakes (GF) moderately enhance it, affecting $\mathrm{Na^+}$ and $\mathrm{TFSI^-}$ ion mobilities and electronic conductivity. Bulky, three-dimensional carbon black (CB) additives induce a unique behavior where the sign of the Seebeck coefficient changes with temperature, presumably due to interaction with $\mathrm{TFSI^-}$ ions within the CB structure. Changes in activation energy and Vogel temperature with salt concentration suggest structural and mechanical modifications in the polymer matrix. The choice of carbon-based additives and salt concentration significantly influences the thermoelectric properties of SPEs thermoelectric properties, providing insights into their potential for thermoelectric applications. Sodium-based electrolytes emerge as promising, sustainable alternatives to lithium-based systems, aligning with sustainable energy research demands.

Large language models (LLMs) are rapidly replacing help forums like StackOverflow, and are especially helpful for non-professional programmers and end users. These users are often interested in data-centric tasks, such as spreadsheet manipulation and data wrangling, which are hard to solve if the intent is only communicated using a natural-language description, without including the data. But how do we decide how much data and which data to include in the prompt? This paper makes two contributions towards answering this question. First, we create a dataset of real-world NL-to-code tasks manipulating tabular data, mined from StackOverflow posts. Second, we introduce a cluster-then-select prompting technique, which adds the most representative rows from the input data to the LLM prompt. Our experiments show that LLM performance is indeed sensitive to the amount of data passed in the prompt, and that for tasks with a lot of syntactic variation in the input table, our cluster-then-select technique outperforms a random selection baseline., Comment: Paper accepted to NAACL 2024 (Findings)

Generative AI, such as image generation models and large language models, stands to provide tremendous value to end-user programmers in creative and knowledge workflows. Current research methods struggle to engage end-users in a realistic conversation that balances the actually existing capabilities of generative AI with the open-ended nature of user workflows and the many opportunities for the application of this technology. In this work-in-progress paper, we introduce participatory prompting, a method for eliciting opportunities for generative AI in end-user workflows. The participatory prompting method combines a contextual inquiry and a researcher-mediated interaction with a generative model, which helps study participants interact with a generative model without having to develop prompting strategies of their own. We discuss the ongoing development of a study whose aim will be to identify end-user programming opportunities for generative AI in data analysis workflows., Comment: Proceedings of the 34th Annual Conference of the Psychology of Programming Interest Group (PPIG 2023)

A central challenge in quantum machine learning is the design and training of parameterized quantum circuits (PQCs). Similar to deep learning, vanishing gradients pose immense problems in the trainability of PQCs, which have been shown to arise from a multitude of sources. One such cause are non-local loss functions, that demand the measurement of a large subset of involved qubits. To facilitate the parameter training for quantum applications using global loss functions, we propose a Sequential Hamiltonian Assembly, which iteratively approximates the loss function using local components. Aiming for a prove of principle, we evaluate our approach using Graph Coloring problem with a Varational Quantum Eigensolver (VQE). Simulation results show, that our approach outperforms conventional parameter training by 29.99% and the empirical state of the art, Layerwise Learning, by 5.12% in the mean accuracy. This paves the way towards locality-aware learning techniques, allowing to evade vanishing gradients for a large class of practically relevant problems.