Search

In real applications of Reinforcement Learning (RL), such as robotics, low latency and energy efficient inference is very desired. The use of sparsity and pruning for optimizing Neural Network inference, and particularly to improve energy and latency efficiency, is a standard technique. In this work, we perform a systematic investigation of applying these optimization techniques for different RL algorithms in different RL environments, yielding up to a 400-fold reduction in the size of neural networks., Comment: 14 pages, 6 figures

We consider linear search for an escaping target whose speed and initial position are unknown to the searcher. A searcher (an autonomous mobile agent) is initially placed at the origin of the real line and can move with maximum speed $1$ in either direction along the line. An oblivious mobile target that is moving away from the origin with an unknown constant speed $v<1$ is initially placed by an adversary on the infinite line at distance $d$ from the origin in an unknown direction. We consider two cases, depending on whether $d$ is known or unknown. The main contribution of this paper is to prove a new lower bound and give algorithms leading to new upper bounds for search in these settings. This results in an optimal (up to lower order terms in the exponent) competitive ratio in the case where $d$ is known and improved upper and lower bounds for the case where $d$ is unknown. Our results solve an open problem proposed in [Coleman et al., Proc. OPODIS 2022].

One common way to speed up the find operation within a set of text files involves a trigram index. This structure is merely a map from a trigram (sequence consisting of three characters) to a set of files which contain it. When searching for a pattern, potential file locations are identified by intersecting the sets related to the trigrams in the pattern. Then, the search proceeds only in these files. However, in a code repository, the trigram index evolves across different versions. Upon checking out a new version, this index is typically built from scratch, which is a time-consuming task, while we want our index to have almost zero-time startup. Thus, we explore the persistent version of a trigram index for full-text and key word patterns search. Our approach just uses the current version of the trigram index and applies only the changes between versions during checkout, significantly enhancing performance. Furthermore, we extend our data structure to accommodate CamelHump search for class and function names.

Personalization in machine learning (ML) tailors models' decisions to the individual characteristics of users. While this approach has seen success in areas like recommender systems, its expansion into high-stakes fields such as healthcare and autonomous driving is hindered by the extensive regulatory approval processes involved. To address this challenge, we propose a novel framework termed represented Markov Decision Processes (r-MDPs) that is designed to balance the need for personalization with the regulatory constraints. In an r-MDP, we cater to a diverse user population, each with unique preferences, through interaction with a small set of representative policies. Our objective is twofold: efficiently match each user to an appropriate representative policy and simultaneously optimize these policies to maximize overall social welfare. We develop two deep reinforcement learning algorithms that efficiently solve r-MDPs. These algorithms draw inspiration from the principles of classic K-means clustering and are underpinned by robust theoretical foundations. Our empirical investigations, conducted across a variety of simulated environments, showcase the algorithms' ability to facilitate meaningful personalization even under constrained policy budgets. Furthermore, they demonstrate scalability, efficiently adapting to larger policy budgets., Comment: Accepted to AAAI 2024. Code: https://github.com/dimonenka/RL_policy_budget

We have measured the absolute doubly differential angular sputtering yield for 20 keV Kr+ impacting a polycrystalline Cu slab at an incidence angle of {\theta}i = 45{\deg} relative to the surface normal. Sputtered Cu atoms were captured using collectors mounted on a half dome above the sample, and the sputtering distribution was measured as a function of the sputtering polar, {\theta}s, and azimuthal, phi, angles. Absolute results of the sputtering yield were determined from the mass gain of each collector, the ion dose, and the solid angle subtended, after irradiation to a total fluence of ~ 1 x 10^18 ions/cm^2. Our approach overcomes shortcomings of commonly used methods that only provide relative yields as a function of {\theta}s in the incidence plane (defined by the ion velocity and the surface normal). Our experimental results display an azimuthal variation that increases with increasing {\theta}s and is clearly discrepant with simulations using binary collision theory. We attribute the observed azimuthal anisotropy to ion-induced formation of micro- and nano-scale surface features that suppress the sputtering yield through shadowing and redeposition effects, neither of which are accounted for in the simulations. Our experimental results demonstrate the importance of doubly differential angular sputtering studies to probe ion sputtering processes at a fundamental level and to explore the effect of ion-beam-generated surface roughness., Comment: 29 pages, 9 figures

We present the full next-to-leading order (NLO) result for the impact factor of a forward Higgs boson, obtained in the infinite-top-mass limit, both in the momentum representation and as superposition of the eigenfunctions of the leading-order (LO) BFKL kernel., Comment: 8 pages, 1 Figure. Presented by Michael Fucilla at the 16th International Symposium on Radiative Corrections: Applications of Quantum Field Theory to Phenomenology (RADCOR 2023), Crieff (Scotland), 28 May- 02 June, 2023

Contract design involves a principal who establishes contractual agreements about payments for outcomes that arise from the actions of an agent. In this paper, we initiate the study of deep learning for the automated design of optimal contracts. We introduce a novel representation: the Discontinuous ReLU (DeLU) network, which models the principal's utility as a discontinuous piecewise affine function of the design of a contract where each piece corresponds to the agent taking a particular action. DeLU networks implicitly learn closed-form expressions for the incentive compatibility constraints of the agent and the utility maximization objective of the principal, and support parallel inference on each piece through linear programming or interior-point methods that solve for optimal contracts. We provide empirical results that demonstrate success in approximating the principal's utility function with a small number of training samples and scaling to find approximately optimal contracts on problems with a large number of actions and outcomes.

The majority of Multi-Agent Reinforcement Learning (MARL) literature equates the cooperation of self-interested agents in mixed environments to the problem of social welfare maximization, allowing agents to arbitrarily share rewards and private information. This results in agents that forgo their individual goals in favour of social good, which can potentially be exploited by selfish defectors. We argue that cooperation also requires agents' identities and boundaries to be respected by making sure that the emergent behaviour is an equilibrium, i.e., a convention that no agent can deviate from and receive higher individual payoffs. Inspired by advances in mechanism design, we propose to solve the problem of cooperation, defined as finding socially beneficial equilibrium, by using mediators. A mediator is a benevolent entity that may act on behalf of agents, but only for the agents that agree to it. We show how a mediator can be trained alongside agents with policy gradient to maximize social welfare subject to constraints that encourage agents to cooperate through the mediator. Our experiments in matrix and iterative games highlight the potential power of applying mediators in MARL.

It has been recently argued that the inclusive hadroproduction at the LHC of a Higgs boson in association with a jet can be sensitive to the high-energy dynamics. Moreover, the impact of the resummation at FCC energies is expected to be large also in the inclusive cross section for the main Higgs production channel in proton-proton collisions, namely the gluon fusion. As the energy increases, a pure $k_t$-factorization framework or a collinear-factorization approach supplemented by the high-energy resummation are adequate formalisms to describe these processes. In both cases, the fundamental missing ingredient for a next-to-leading logarithmic description is the Higgs boson impact factor. We present the full next-to-leading result for the forward Higgs impact factor, obtained in the infinite top-mass limit, discussing possible future extensions, such as the calculation of the impact factor in the central rapidity region., Comment: 4 pages. Contribution to the 2023 QCD session of the 57th Rencontres de Moriond

In this work, we present a new benchmarking suite with new real-life inspired skewed workloads to test the performance of concurrent index data structures. We started this project to prepare workloads specifically for self-adjusting data structures, i.e., they handle more frequent requests faster, and, thus, should perform better than their standard counterparts. We looked over the commonly used suites to test performance of concurrent indices trying to find an inspiration: Synchrobench, Setbench, YCSB, and TPC - and we found several issues with them. The major problem is that they are not flexible: it is difficult to introduce new workloads, it is difficult to set the duration of the experiments, and it is difficult to change the parameters. We decided to solve this issue by presenting a new suite based on Synchrobench. Finally, we highlight the problem of measuring performance of data structures. We show that the relative performance of data structures highly depends on the workload: it is not clear which data structure is best. For that, we take three state-of-the-art concurrent binary search trees and run them on the workloads from our benchmarking suite. As a result, we get six experiments with all possible relative performance of the chosen data structures.

Working in the hybrid high-energy/collinear factorization, where the next-to-leading resummation of energy logarithms is combined with collinear parton densities and fragmentation functions, we study observables sensitive to high-energy dynamics in the context of heavy-flavor physics., Comment: 6 pages, 1 figure. Presented by Michael Fucilla at "Diffraction and Low-$x$ 2022", Corigliano Calabro (Italy), September 24-30, 2022

We present a new study on helicity amplitudes and cross sections for the exclusive production of $\rho$ mesons at the EIC in high-energy factorization. In this framework the analytic expression of amplitudes takes the form of a convolution between an off-shell impact factor, depicting the ($\gamma^* \to \rho$) transition, and a nonperturbative density, known as Unintegrated Gluon Distribution (UGD) that encodes information about the proton structure at low $x$ and evolves according to the BFKL equation. We come out with an evidence that observables sensitive to the polarizations of the incoming virtual photon and of the emitted meson allow us to discriminate among different UGD models and to gather quantitative information on the proton content at high energies., Comment: 6 pages, 1 figure. Presented by F.G. Celiberto at DIS2022: XXIX International Workshop on Deep-Inelastic Scattering and Related Subjects, Santiago de Compostela, Spain, May 2-6 2022

Modern AI systems, based on von Neumann architecture and classical neural networks, have a number of fundamental limitations in comparison with the brain. This article discusses such limitations and the ways they can be mitigated. Next, it presents an overview of currently available neuromorphic AI projects in which these limitations are overcame by bringing some brain features into the functioning and organization of computing systems (TrueNorth, Loihi, Tianjic, SpiNNaker, BrainScaleS, NeuronFlow, DYNAP, Akida). Also, the article presents the principle of classifying neuromorphic AI systems by the brain features they use (neural networks, parallelism and asynchrony, impulse nature of information transfer, local learning, sparsity, analog and in-memory computing). In addition to new architectural approaches used in neuromorphic devices based on existing silicon microelectronics technologies, the article also discusses the prospects of using new memristor element base. Examples of recent advances in the use of memristors in euromorphic applications are also given.

We calculate the next-to-leading order correction to the impact factor (vertex) for the production of a forward Higgs boson, obtained in the infinite top-mass limit. We present the result both in the momentum representation and as superposition of the eigenfunctions of the leading-order BFKL kernel. This impact factor is a necessary ingredient for the description of the inclusive hadroproduction of a forward Higgs in the limit of small Bjorken $x$, as well as for the study of inclusive forward emissions of a Higgs boson in association with a backward identified object., Comment: 49 pages, 12 figures

Despite the numerous breakthroughs achieved with Reinforcement Learning (RL), solving environments with sparse rewards remains a challenging task that requires sophisticated exploration. Learning from Demonstrations (LfD) remedies this issue by guiding the agent's exploration towards states experienced by an expert. Naturally, the benefits of this approach hinge on the quality of demonstrations, which are rarely optimal in realistic scenarios. Modern LfD algorithms require meticulous tuning of hyperparameters that control the influence of demonstrations and, as we show in the paper, struggle with learning from suboptimal demonstrations. To address these issues, we extend Self-Imitation Learning (SIL), a recent RL algorithm that exploits the agent's past good experience, to the LfD setup by initializing its replay buffer with demonstrations. We denote our algorithm as SIL from Demonstrations (SILfD). We empirically show that SILfD can learn from demonstrations that are noisy or far from optimal and can automatically adjust the influence of demonstrations throughout the training without additional hyperparameters or handcrafted schedules. We also find SILfD superior to the existing state-of-the-art LfD algorithms in sparse environments, especially when demonstrations are highly suboptimal.

When dealing with binary classification of data with only one labeled class data scientists employ two main approaches, namely One-Class (OC) classification and Positive Unlabeled (PU) learning. The former only learns from labeled positive data, whereas the latter also utilizes unlabeled data to improve the overall performance. Since PU learning utilizes more data, we might be prone to think that when unlabeled data is available, the go-to algorithms should always come from the PU group. However, we find that this is not always the case if unlabeled data is unreliable, i.e. contains limited or biased latent negative data. We perform an extensive experimental study of a wide list of state-of-the-art OC and PU algorithms in various scenarios as far as unlabeled data reliability is concerned. Furthermore, we propose PU modifications of state-of-the-art OC algorithms that are robust to unreliable unlabeled data, as well as a guideline to similarly modify other OC algorithms. Our main practical recommendation is to use state-of-the-art PU algorithms when unlabeled data is reliable and to use the proposed modifications of state-of-the-art OC algorithms otherwise. Additionally, we outline procedures to distinguish the cases of reliable and unreliable unlabeled data using statistical tests.

RegretNet is a recent breakthrough in the automated design of revenue-maximizing auctions. It combines the flexibility of deep learning with the regret-based approach to relax the Incentive Compatibility (IC) constraint (that participants prefer to bid truthfully) in order to approximate optimal auctions. We propose two independent improvements of RegretNet. The first is a neural architecture denoted as RegretFormer that is based on attention layers. The second is a loss function that requires explicit specification of an acceptable IC violation denoted as regret budget. We investigate both modifications in an extensive experimental study that includes settings with constant and inconstant number of items and participants, as well as novel validation procedures tailored to regret-based approaches. We find that RegretFormer consistently outperforms RegretNet in revenue (i.e. is optimal-er) and that our loss function both simplifies hyperparameter tuning and allows to unambiguously control the revenue-regret trade-off by selecting the regret budget., Comment: NeurIPS 2022

This article proposes a sparse computation-based method for optimizing neural networks for reinforcement learning (RL) tasks. This method combines two ideas: neural network pruning and taking into account input data correlations; it makes it possible to update neuron states only when changes in them exceed a certain threshold. It significantly reduces the number of multiplications when running neural networks. We tested different RL tasks and achieved 20-150x reduction in the number of multiplications. There were no substantial performance losses; sometimes the performance even improved.

With the advent of TeV-energy colliding machines, such as the Large Hadron Collider (LHC), the possibility has opened up to test predictions of Quantum Chromodynamics (QCD) and, more in general, of the Standard Model (SM), in new, and so far unexplored, kinematical regimes. Among the many reactions that can be investigated at LHC, the Higgs production is one of the most important and challenging for the entire high-energy physics Community. Beside usual studies in the Higgs sector, it has recently been highlighted how differential Higgs distributions can be effectively used as ``stabilizers'' of the high-energy dynamics of QCD. The definition and the study of observables sensitive to high-energy dynamics in Higgs production has the double advantage of (i) allowing us to clearly disentangle the high-energy dynamics from the fixed-order one and (ii) providing us with an auxiliary tool to extend Higgs studies in wider kinematical regimes. In this work, we will show how a general hybrid collinear/high energy factorization can be built up for the inclusive production of a Higgs in association with a jet. Then, we will present some phenomenological analyses that corroborate the underlying assumption that this reaction can be used to investigate the semi-hard regime of QCD. Finally, we will focus on more formal developments, such as the inclusion of subleading corrections to previous studies, via the calculation of the forward next-to-leading order Higgs impact factor., Comment: 6 pages, 1 figures; Presented by M. Fucilla at the Particles and Nuclei International Conference (PANIC 2021), 5-10 September 2021; to be published in the conference proceedings by PoS