Search

The commissioning and operation of future large-scale scientific experiments will challenge current tuning and control methods. Reinforcement learning (RL) algorithms are a promising solution thanks to their capability of autonomously tackling a control problem based on a task parameterized by a reward function. The conventionally utilized machine learning (ML) libraries are not intended for microsecond latency applications, as they mostly optimize for throughput performance. On the other hand, most of the programmable logic implementations are meant for computation acceleration, not being intended to work in a real-time environment. To overcome these limitations of current implementations, RL needs to be deployed on-the-edge, i.e. on to the device gathering the training data. In this paper we present the design and deployment of an experience accumulator system in a particle accelerator. In this system deep-RL algorithms run using hardware acceleration and act within a few microseconds, enabling the use of RL for control of ultra-fast phenomena. The training is performed offline to reduce the number of operations carried out on the acceleration hardware. The proposed architecture was tested in real experimental conditions at the Karlsruhe research accelerator (KARA), serving also as a synchrotron light source, where the system was used to control induced horizontal betatron oscillations in real-time. The results showed a performance comparable to the commercial feedback system available at the accelerator, proving the viability and potential of this approach. Due to the self-learning and reconfiguration capability of this implementation, its seamless application to other control problems is possible. Applications range from particle accelerators to large-scale research and industrial facilities.

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time, the computational cost of simulations, and the high-dimensionality of optimisation problems pose significant challenges in generating the required data for training state-of-the-art machine learning models. In this work, we introduce Cheetah, a PyTorch-based high-speed differentiable linear-beam dynamics code. Cheetah enables the fast collection of large data sets by reducing computation times by multiple orders of magnitude and facilitates efficient gradient-based optimisation for accelerator tuning and system identification. This positions Cheetah as a user-friendly, readily extensible tool that integrates seamlessly with widely adopted machine learning tools. We showcase the utility of Cheetah through five examples, including reinforcement learning training, gradient-based beamline tuning, gradient-based system identification, physics-informed Bayesian optimisation priors, and modular neural network surrogate modelling of space charge effects. The use of such a high-speed differentiable simulation code will simplify the development of machine learning-based methods for particle accelerators and fast-track their integration into everyday operations of accelerator facilities., Comment: 16 pages, 9 figures, 3 tables

Accelerator physics relies on numerical algorithms to solve optimization problems in online accelerator control and tasks such as experimental design and model calibration in simulations. The effectiveness of optimization algorithms in discovering ideal solutions for complex challenges with limited resources often determines the problem complexity these methods can address. The accelerator physics community has recognized the advantages of Bayesian optimization algorithms, which leverage statistical surrogate models of objective functions to effectively address complex optimization challenges, especially in the presence of noise during accelerator operation and in resource-intensive physics simulations. In this review article, we offer a conceptual overview of applying Bayesian optimization techniques towards solving optimization problems in accelerator physics. We begin by providing a straightforward explanation of the essential components that make up Bayesian optimization techniques. We then give an overview of current and previous work applying and modifying these techniques to solve accelerator physics challenges. Finally, we explore practical implementation strategies for Bayesian optimization algorithms to maximize their performance, enabling users to effectively address complex optimization challenges in real-time beam control and accelerator design.

Online tuning of real-world plants is a complex optimisation problem that continues to require manual intervention by experienced human operators. Autonomous tuning is a rapidly expanding field of research, where learning-based methods, such as Reinforcement Learning-trained Optimisation (RLO) and Bayesian optimisation (BO), hold great promise for achieving outstanding plant performance and reducing tuning times. Which algorithm to choose in different scenarios, however, remains an open question. Here we present a comparative study using a routine task in a real particle accelerator as an example, showing that RLO generally outperforms BO, but is not always the best choice. Based on the study's results, we provide a clear set of criteria to guide the choice of algorithm for a given tuning task. These can ease the adoption of learning-based autonomous tuning solutions to the operation of complex real-world plants, ultimately improving the availability and pushing the limits of operability of these facilities, thereby enabling scientific and engineering advancements., Comment: 17 pages, 8 figures, 2 tables

We have evaluated the data-efficient Bayesian optimization method for the specific task of injection tuning in a circular accelerator. In this paper, we describe the implementation of this method at the Karlsruhe Research Accelerator with up to nine tuning parameters, including the determination of the associated hyperparameters. We show that the Bayesian optimization method outperforms manual tuning and the commonly used Nelder-Mead optimization algorithm both in simulation and experiment. The algorithm was also successfully used to ease the commissioning phase after the installation of new injection magnets and is regularly used during accelerator operations. We demonstrate that the introduction of context variables that include intra-bunch scattering effects, such as the Touschek effect, further improves the control and robustness of the injection process.

Studies of the crab cavities at KEKB revealed that the RF phase could shift by up to 50o within ~50 us during a quench; while the cavity voltage is still at approximately 75% of its nominal amplitude. If such a failure were to occur on the HL-LHC crab cavities, it is likely that the machine would sustain substantial damage to the beam line and surrounding infrastructure due to uncontrolled beam loss before the machine protection system could dump the beam. We have developed a low-level RF system model, including detuning mechanisms and beam loading, and use this to simulate the behaviour of a crab cavity during a quench, modeling the low-level RF system, detuning mechanisms and beam loading. We supplement this with measurement data of the actual RF response of the proof of principle Double-Quarter Wave Crab Cravity during a quench. Extrapolating these measurements to the HL-LHC, we show that Lorentz Force detuning is the dominant effect leading to phase shifts in the crab cavity during quenches; rather than pressure detuning which is expected to be dominant for the KEKB crab cavities. The total frequency shift for the HL-LHC crab cavities during quenches is expected to be about 460 Hz, leading to a phase shift of no more than 3o. The results of the quench model are read into a particle tracking simulation, SixTrack, and used to determine the effect of quenches on the HL-LHC beam. The quench model has been benchmarked against the KEKB experimental measurements. In this paper we present the results of the simulations on a crab cavity failure for HL-LHC as well as for the SPS and show that beam loss is negligible when using a realistic low-level RF response., Comment: 21 Pages, 22 figures, Submitted to PRAB