Search

Exploration is a cornerstone of reinforcement learning (RL). Intrinsic motivation attempts to decouple exploration from external, task-based rewards. However, established approaches to intrinsic motivation that follow general principles such as information gain, often only uncover low-level interactions. In contrast, children's play suggests that they engage in meaningful high-level behavior by imitating or interacting with their caregivers. Recent work has focused on using foundation models to inject these semantic biases into exploration. However, these methods often rely on unrealistic assumptions, such as language-embedded environments or access to high-level actions. We propose SEmaNtically Sensible ExploratIon (SENSEI), a framework to equip model-based RL agents with an intrinsic motivation for semantically meaningful behavior. SENSEI distills a reward signal of interestingness from Vision Language Model (VLM) annotations, enabling an agent to predict these rewards through a world model. Using model-based RL, SENSEI trains an exploration policy that jointly maximizes semantic rewards and uncertainty. We show that in both robotic and video game-like simulations SENSEI discovers a variety of meaningful behaviors from image observations and low-level actions. SENSEI provides a general tool for learning from foundation model feedback, a crucial research direction, as VLMs become more powerful., Comment: Preprint, under review. Project webpage at https://sites.google.com/view/sensei-paper

Robust and scalable function evaluation at any arbitrary point in the finite/spectral element mesh is required for querying the partial differential equation solution at points of interest, comparison of solution between different meshes, and Lagrangian particle tracking. This is a challenging problem, particularly for high-order unstructured meshes partitioned in parallel with MPI, as it requires identifying the element that overlaps a given point and computing the corresponding reference space coordinates. We present a robust and efficient technique for general field evaluation in large-scale high-order meshes with quadrilaterals and hexahedra. In the proposed method, a combination of globally partitioned and processor-local maps are used to first determine a list of candidate MPI ranks, and then locally candidate elements that could contain a given point. Next, element-wise bounding boxes further reduce the list of candidate elements. Finally, Newton's method with trust region is used to determine the overlapping element and corresponding reference space coordinates. Since GPU-based architectures have become popular for accelerating computational analyses using meshes with tensor-product elements, specialized kernels have been developed to utilize the proposed methodology on GPUs. The method is also extended to enable general field evaluation on surface meshes. The paper concludes by demonstrating the use of proposed method in various applications ranging from mesh-to-mesh transfer during r-adaptivity to Lagrangian particle tracking., Comment: 52 pages, 17 figures, 1 table

While many algorithms for diversity maximization under imitation constraints are online in nature, many applications require offline algorithms without environment interactions. Tackling this problem in the offline setting, however, presents significant challenges that require non-trivial, multi-stage optimization processes with non-stationary rewards. In this work, we present a novel offline algorithm that enhances diversity using an objective based on Van der Waals (VdW) force and successor features, and eliminates the need to learn a previously used skill discriminator. Moreover, by conditioning the value function and policy on a pre-trained Functional Reward Encoding (FRE), our method allows for better handling of non-stationary rewards and provides zero-shot recall of all skills encountered during training, significantly expanding the set of skills learned in prior work. Consequently, our algorithm benefits from receiving a consistently strong diversity signal (VdW), and enjoys more stable and efficient training. We demonstrate the effectiveness of our method in generating diverse skills for two robotic tasks in simulation: locomotion of a quadruped and local navigation with obstacle traversal.

Bertsimas and Lo's seminal work laid the groundwork for addressing the implementation shortfall dilemma in institutional investing, emphasizing the significance of market microstructure and price dynamics in minimizing execution costs. However, the ability to derive a theoretical Optimum market order policy is an unrealistic assumption for many investors. This study aims to bridge this gap by proposing an approach that leverages stochastic gradient descent (SGD) to derive alternative solutions for optimizing execution cost policies in dynamic markets where explicit mathematical solutions may not yet exist. The proposed methodology assumes the existence of a mathematically derived optimal solution that is a function of the underlying market dynamics. By iteratively refining strategies using SGD, economists can adapt their approaches over time based on evolving execution strategies. While these SGD-based solutions may not achieve optimality, they offer valuable insights into optimizing policies under complex market frameworks. These results serve as a bridge for economists and mathematicians, facilitating the study of the Optimum policy volatile markets while offering SGD driven implementable policies that closely approximate optimal outcomes within shorter time frames.

Bertsimas and Lo's seminal work established a foundational framework for addressing the implementation shortfall dilemma faced by large institutional investors. Their models emphasized the critical role of accurate knowledge of market microstructure and price/information dynamics in optimizing trades to minimize execution costs. However, this paper recognizes that perfect initial knowledge may not be a realistic assumption for new investors entering the market. Therefore, this study aims to bridge this gap by proposing an approach that iteratively derives OLS estimates of the market parameters from period to period. This methodology enables uninformed investors to engage in the market dynamically, adjusting their strategies over time based on evolving estimates, thus offering a practical solution for navigating the complexities of execution cost optimization without perfect initial knowledge.

This paper presents the multiplicity of neutral and charged hadrons produced in 90 GeV$/c$ proton-carbon interactions from a dataset taken by the NA61/SHINE experiment in 2017. Particle identification via dE/dx was performed for the charged hadrons $\pi^\pm$, $K^\pm$, and $p / \bar{p}$; the neutral hadrons $K^0_S$, $\Lambda$, and $\bar{\Lambda}$ were identified via an invariant mass analysis of their decays to charged hadrons. Double-differential multiplicity results as a function of laboratory momentum and polar angle are presented for each particle species; these results provide vital constraints on the predicted neutrino beam flux for current and future long-baseline neutrino oscillation experiments.

This article addresses the overlooked but crucial role of the Haar measure in solid mechanics, a concept well-established in mathematical literature but frequently misunderstood by mechanicians. The aim is to provide practical insights and methodologies for the application of the Haar measure in various mechanical scenarios. The article begins with an introduction to the Haar measure, underlying its significance and a theoretical foundation for its definition and formulas. Moving beyond mathematical abstraction, the core of the article relies on practical applications, including the computation of the Haar measure for the orthogonal transformations of the 2D and 3D space under several commonly used parametrisations, an application about invariant theory in mechanics and sampling on a group orbit. These applications are presented with practical examples, enabling mechanicians to integrate the Haar measure into their research seamlessly. The article caters to a broad audience, with sections designed for both introductory comprehension and in-depth exploration. Our goal is to equip mechanicians with a valuable tool, bridging the gap between mathematical theory and practical applications in the field of mechanics.

Offline reinforcement learning algorithms hold the promise of enabling data-driven RL methods that do not require costly or dangerous real-world exploration and benefit from large pre-collected datasets. This in turn can facilitate real-world applications, as well as a more standardized approach to RL research. Furthermore, offline RL methods can provide effective initializations for online finetuning to overcome challenges with exploration. However, evaluating progress on offline RL algorithms requires effective and challenging benchmarks that capture properties of real-world tasks, provide a range of task difficulties, and cover a range of challenges both in terms of the parameters of the domain (e.g., length of the horizon, sparsity of rewards) and the parameters of the data (e.g., narrow demonstration data or broad exploratory data). While considerable progress in offline RL in recent years has been enabled by simpler benchmark tasks, the most widely used datasets are increasingly saturating in performance and may fail to reflect properties of realistic tasks. We propose a new benchmark for offline RL that focuses on realistic simulations of robotic manipulation and locomotion environments, based on models of real-world robotic systems, and comprising a variety of data sources, including scripted data, play-style data collected by human teleoperators, and other data sources. Our proposed benchmark covers state-based and image-based domains, and supports both offline RL and online fine-tuning evaluation, with some of the tasks specifically designed to require both pre-training and fine-tuning. We hope that our proposed benchmark will facilitate further progress on both offline RL and fine-tuning algorithms. Website with code, examples, tasks, and data is available at \url{https://sites.google.com/view/d5rl/}, Comment: RLC 2024

We propose a transformer architecture for time series forecasting with a focus on time series tokenisation and apply it to a real-world prediction problem from the pricing domain. Our architecture aims to learn effective representations at many scales across all available data simultaneously. The model contains a number of novel modules: a differentiated form of time series patching which employs multiple resolutions, a multiple-resolution module for time-varying known variables, a mixer-based module for capturing cross-series information, and a novel output head with favourable scaling to account for the increased number of tokens. We present an application of this model to a real world prediction problem faced by the markdown team at a very large retailer. On the experiments conducted our model outperforms in-house models and the selected existing deep learning architectures.

Equilibriums in magnetic confinement devices result from force balancing between the Lorentz force and the plasma pressure gradient. In an axisymmetric configuration like a tokamak, such an equilibrium is described by an elliptic equation for the poloidal magnetic flux, commonly known as the Grad--Shafranov equation. It is challenging to develop a scalable and accurate free-boundary Grad--Shafranov solver, since it is a fully nonlinear optimization problem that simultaneouly solves for the magnetic field coil current outside the plasma to control the plasma shape. In this work, we develop a Newton-based free-boundary Grad--Shafranov solver using adaptive finite elements and preconditioning strategies. The free-boundary interaction leads to the evaluation of a domain-dependent nonlinear form of which its contribution to the Jacobian matrix is achieved through shape calculus. The optimization problem aims to minimize the distance between the plasma boundary and specified control points while satisfying two non-trivial constraints, which correspond to the nonlinear finite element discretization of the Grad--Shafranov equation and a constraint on the total plasma current involving a nonlocal coupling term. The linear system is solved by a block factorization, and AMG is called for subblock elliptic operators. The unique contributions of this work include the treatment of a global constraint, preconditioning strategies, nonlocal reformulation, and the implementation of adaptive finite elements. It is found that the resulting Newton solver is robust, successfully reducing the nonlinear residual to 1e-6 and lower in a small handful of iterations while addressing the challenging case to find a Taylor state equilibrium where conventional Picard-based solvers fail to converge.

The Black Hole Explorer (BHEX) is a next-generation space very long baseline interferometry (VLBI) mission concept that will extend the ground-based millimeter/submillimeter arrays into space. The mission, closely aligned with the science priorities of the Japanese VLBI community, involves an active engagement of this community in the development of the mission, resulting in the formation of the Black Hole Explorer Japan Consortium. Here we present the current Japanese vision for the mission, ranging from scientific objectives to instrumentation. The Consortium anticipates a wide range of scientific investigations, from diverse black hole physics and astrophysics studied through the primary VLBI mode, to the molecular universe explored via a potential single-dish observation mode in the previously unexplored 50-70\,GHz band that would make BHEX the highest-sensitivity explorer ever of molecular oxygen. A potential major contribution for the onboard instrument involves supplying essential elements for its high-sensitivity dual-band receiving system, which includes a broadband 300\,GHz SIS mixer and a space-certified multi-stage 4.5K cryocooler akin to those used in the Hitomi and XRISM satellites by the Japan Aerospace Exploration Agency. Additionally, the Consortium explores enhancing and supporting BHEX operations through the use of millimeter/submillimeter facilities developed by the National Astronomical Observatory of Japan, coupled with a network of laser communication stations operated by the National Institute of Information and Communication Technology., Comment: Submitted to SPIE Conference Series, 17 pages for the main text, 33 pages for the references, 4 figures, SPIE Astronomical Telescopes + Instrumentation 2024

We tackle the problem of policy learning from expert demonstrations without a reward function. A central challenge in this space is that these policies fail upon deployment due to issues of distributional shift, environment stochasticity, or compounding errors. Adversarial imitation learning alleviates this issue but requires additional on-policy training samples for stability, which presents a challenge in realistic domains due to inefficient learning and high sample complexity. One approach to this issue is to learn a world model of the environment, and use synthetic data for policy training. While successful in prior works, we argue that this is sub-optimal due to additional distribution shifts between the learned model and the real environment. Instead, we re-frame imitation learning as a fine-tuning problem, rather than a pure reinforcement learning one. Drawing theoretical connections to offline RL and fine-tuning algorithms, we argue that standard online world model algorithms are not well suited to the imitation learning problem. We derive a principled conservative optimization bound and demonstrate empirically that it leads to improved performance on two very challenging manipulation environments from high-dimensional raw pixel observations. We set a new state-of-the-art performance on the Franka Kitchen environment from images, requiring only 10 demos on no reward labels, as well as solving a complex dexterity manipulation task., Comment: Oral presentation, L4DC 2024

We present a computational study of assembling carbon clusters and nanophases in space from carbon aggregations. Geometry optimizations and Density-functional-based tight-binding (SCC-DFTB) dynamics methods are employed to predict carbon clusters, their time evolution, and their stability. The initial density of the aggregates is found to be of primary importance for the structural properties of the clusters. Aggregates with sufficiently low initial density yield clusters with approximately equal prevalence of sp and sp2 hybridized states and almost missing sp3 ones. The increase in the initial density results in sp2-dominant molecules that resemble the carbon skeleton of polycyclic aromatic hydrocarbons (PAHs). Larger initial aggregations with tetrahedral interatomic orientation result in sp2-dominant multi-dimensional polymers. Such materials are highly porous and resemble axially bound nanotubes. Some resultant clusters resemble fullerene building blocks. Spheroid nanoparticles resembling improper fullerenes are predicted by metadynamics, aimed at inter-fragment coupling reactions. One such structure has the lowest binding energy per atom among the studied molecules. All zero-dimensional forms, obtained by the simulations, conform to the experimentally detected types of molecules in space. The theoretical IR spectrum of the clusters closely resembles that of fullerene C70 and therefore such imperfect structures may be mistaken for known fullerenes in experimental infrared (IR) telescope studies.

Measurements of $K_S^0$ meson production via its $\pi^{+} \pi^{-}$ decay mode in inelastic $\textit{p+p}$ interactions at incident projectile momenta of 31, 40 and 80 GeV/$c$ ($\sqrt{s_{NN}}=7.7, 8.8$ and $12.3$ GeV, respectively) are presented. The data were recorded by the NA61/SHINE spectrometer at the CERN Super Proton Synchrotron. Double-differential distributions were obtained in transverse momentum and rapidity. The mean multiplicities of $K_S^0$ mesons were determined to be $(5.95 \pm 0.19 (stat) \pm 0.22 (sys)) \times 10^{-2}$ at 31 GeV/$c$, $(7.61 \pm 0.13 (stat) \pm 0.31 (sys)) \times 10^{-2}$ at 40 GeV/$c$ and $(11.58 \pm 0.12 (stat) \pm 0.37 (sys)) \times 10^{-2}$ at 80 GeV/$c$. The results on $K^{0}_{S}$ production are compared with model calculations (Epos1.99, SMASH 2.0 and PHSD) as well as with published data from other experiments., Comment: arXiv admin note: substantial text overlap with arXiv:2106.07535

The MFEM (Modular Finite Element Methods) library is a high-performance C++ library for finite element discretizations. MFEM supports numerous types of finite element methods and is the discretization engine powering many computational physics and engineering applications across a number of domains. This paper describes some of the recent research and development in MFEM, focusing on performance portability across leadership-class supercomputing facilities, including exascale supercomputers, as well as new capabilities and functionality, enabling a wider range of applications. Much of this work was undertaken as part of the Department of Energy's Exascale Computing Project (ECP) in collaboration with the Center for Efficient Exascale Discretizations (CEED)., Comment: 19 pages, 19 figures

Computational analysis with the finite element method requires geometrically accurate meshes. It is well known that high-order meshes can accurately capture curved surfaces with fewer degrees of freedom in comparison to low-order meshes. Existing techniques for high-order mesh generation typically output meshes with same polynomial order for all elements. However, high order elements away from curvilinear boundaries or interfaces increase the computational cost of the simulation without increasing geometric accuracy. In prior work, we have presented one such approach for generating body-fitted uniform-order meshes that takes a given mesh and morphs it to align with the surface of interest prescribed as the zero isocontour of a level-set function. We extend this method to generate mixed-order meshes such that curved surfaces of the domain are discretized with high-order elements, while low-order elements are used elsewhere. Numerical experiments demonstrate the robustness of the approach and show that it can be used to generate mixed-order meshes that are much more efficient than high uniform-order meshes. The proposed approach is purely algebraic, and extends to different types of elements (quadrilaterals/triangles/tetrahedron/hexahedra) in two- and three-dimensions., Comment: 14 pages, 11 figures

The critical point of strongly interacting matter is searched for at the CERN SPS by the NA61/SHINE experiment in central $^{40}$Ar +$^{45}$Sc collisions at 13$A$, 19$A$, 30$A$, 40$A$, and 75$A$ GeV/$c$. The dependence of the second-order scaled factorial moments of proton multiplicity distributions on the number of subdivisions in transverse momentum space is measured. The intermittency analysis uses statistically independent data sets for every subdivision in transverse and cumulative-transverse momentum variables. The results obtained do not indicate the searched intermittent pattern. An upper limit on the fraction of correlated protons and the intermittency index is obtained based on a comparison with the Power-law Model., Comment: 38 pages, 20 figures

We study the problem of offline pre-training and online fine-tuning for reinforcement learning from high-dimensional observations in the context of realistic robot tasks. Recent offline model-free approaches successfully use online fine-tuning to either improve the performance of the agent over the data collection policy or adapt to novel tasks. At the same time, model-based RL algorithms have achieved significant progress in sample efficiency and the complexity of the tasks they can solve, yet remain under-utilized in the fine-tuning setting. In this work, we argue that existing model-based offline RL methods are not suitable for offline-to-online fine-tuning in high-dimensional domains due to issues with distribution shifts, off-dynamics data, and non-stationary rewards. We propose an on-policy model-based method that can efficiently reuse prior data through model-based value expansion and policy regularization, while preventing model exploitation by controlling epistemic uncertainty. We find that our approach successfully solves tasks from the MetaWorld benchmark, as well as the Franka Kitchen robot manipulation environment completely from images. To the best of our knowledge, MOTO is the first method to solve this environment from pixels., Comment: This is an updated version of a manuscript that originally appeared at CoRL 2023. The project website is here https://sites.google.com/view/mo2o

This paper presents the energy dependence of multiplicity and net-electric charge fluctuations in p+p interactions at beam momenta 20, 31, 40, 80, and 158 GeV/c. Results are corrected for the experimental biases and quantified with the use of cumulants and factorial cumulants. Cumulant ratios are an essential tool in the search for the critical point of strongly interacting matter in heavy ion collisions. Measurements performed in p+p interactions provide a vital baseline estimation in these studies. The measured signals are compared with the string hadronic models EPOS1.99 and FTFP-BERT., Comment: 28 pages, 10 figures. The version to be published in Eur. Phys. J. C

Strong interactions preserve an approximate isospin symmetry between up ($u$) and down ($d$) quarks, part of the more general flavor symmetry. In the case of $K$ meson production, if this isospin symmetry were exact, it would result in equal numbers of charged ($K^+$ and $K^-$) and neutral ($K^0$ and $\overline K^{\,0}$) mesons in the final state. Here, we report results on the relative abundance of charged over neutral $K$ meson production in argon and scandium nuclei collisions at a center-of-mass energy of 11.9 GeV per nucleon pair. We find that the production of $K^+$ and $K^-$ mesons at mid-rapidity is $(18.4\pm 6.1)\%$ higher than that of the neutral $K$ mesons. Although with large uncertainties, earlier data on nucleus-nucleus collisions in the collision center-of-mass energy range $2.6 < \sqrt{s_{NN}} < 200$~\GeV are consistent with the present result. Using well-established models for hadron production, we demonstrate that known isospin-symmetry breaking effects and the initial nuclei containing more neutrons than protons lead only to a small (few percent) deviation of the charged-to-neutral kaon ratio from unity at high energies. Thus, they cannot explain the measurements. The significance of the flavor-symmetry violation beyond the known effects is 4.7$\sigma$ when the compilation of world data with uncertainties quoted by the experiments is used. New systematic, high-precision measurements and theoretical efforts are needed to establish the origin of the observed large isospin-symmetry breaking., Comment: The paper includes updates based on the final results of $K^+$ and $K^-$ yields published in Eur. Phys. J.C 84 (2024) 4, 416

Para-Hermitian polynomial matrices obtained by matrix spectral factorization lead to functions useful in control theory systems, basis functions in numerical methods or multiscaling functions used in signal processing. We introduce a fast algorithm for matrix spectral factorization based on Bauer$'$s method. We convert Bauer$'$ method into a nonlinear matrix equation (NME). The NME is solved by two different numerical algorithms (Fixed Point Iteration and Newton$'$s Method) which produce approximate scalar or matrix factors, as well as a symbolic algorithm which produces exact factors in closed form for some low-order scalar or matrix polynomial matrices, respectively. Convergence rates of the two numerical algorithms are investigated for a number of singular and nonsingular scalar and matrix polynomials taken from different areas. In particular, one of the singular examples leads to new orthogonal multiscaling and multiwavelet filters. Since the NME can also be solved as a Generalized Discrete Time Algebraic Riccati Equation (GDARE), numerical results using built-in routines in Maple 17.0 and 6 Matlab versions are presented., Comment: 24 pages,5 figures, 4 tables

People Nearby is a service offered by Telegram that allows a user to discover other Telegram users, based only on geographical proximity. Nearby users are reported with a rough estimate of their distance from the position of the reference user, allowing Telegram to claim location privacy In this paper, we systematically analyze the location privacy provided by Telegram to users of the People Nearby service. Through an extensive measurement campaign run by spoofing the user's location all over the world, we reverse-engineer the algorithm adopted by People Nearby to compute distances between users. Although the service protects against precise user localization, we demonstrate that location privacy is always lower than the one declared by Telegram of 500 meters. Specifically, we discover that location privacy is a function of the geographical position of the user. Indeed, the radius of the location privacy area (localization error) spans between 400 meters (close to the equator) and 128 meters (close to the poles), with a difference of up to 75% (worst case) compared to what Telegram declares. After our responsible disclosure, Telegram updated the FAQ associated with the service. Finally, we provide some solutions and countermeasures that Telegram can implement to improve location privacy. In general, the reported findings highlight the significant privacy risks associated with using People Nearby service.

During past decades a marked manifestation of widespread erosion phenomena was studied worldwide. Global conservation community has launched campaigns at local, regional and continental level in developing countries for preservation of soil resources in order not only to stop or mitigate human impact on nature but also to improve life in rural areas introducing new approaches for soil cultivation. After the adoption of Sustainable Development Goals of UNs and launching several world initiatives such as the Land Degradation Neutrality (LDN) the world came to realize the very importance of the soil resources on which the biosphere relies for its existence. The main goal of the chapter is to review different types and structures erosion models as well as their applications. Several methods using spatial analysis capabilities of geographic information systems (GIS) are in operation for soil erosion risk assessment, such as Universal Soil Loss Equation (USLE), Revised Universal Soil Loss Equation (RUSLE) in operation worldwide and in the USA and MESALES model. These and more models are being discussed in the present work alongside more experimental models and methods for assessing soil erosion risk such as Artificial Intelligence (AI), Machine and Deep Learning, etc. At the end of this work, a prospectus for the future development of soil erosion risk assessment is drawn., Comment: Chapter 21 (pages 54)

We report X-ray fluorescence (XRF) imaging of nanoscale inclusions of impurities for quantum technology. A very bright diffraction-limited focus of the X-ray beam produces very high sensitivity and resolution. We investigated gallium (Ga) dopants in silicon (Si) produced by a focused ion beam (FIB). These dopants might provide 3/2-spin qubits or p-type electrical contacts and quantum dots. We find that the ion beam spot is somewhat larger than expected, and the technique provides a useful calibration for the resolution of FIBs. Enticingly, we demonstrate that with a single shot detection of 1 second integration time, the sensitivity of the XRF would be sufficient to find amongst background a single isolated inclusion of unknown location comprising only 3000 Ga impurities (a mass of just 350 zg) without any need for specialized nm-thickness lamellae, and down from >105 atoms in previous reports of similar work. With increased integration we were able to detect 650 impurities. The results show that planned facility upgrades might achieve single atom sensitivity with a generally applicable, non-destructive technique in the near future., Comment: 8 pages, 5 figures

Despite many successful applications of data-driven control in robotics, extracting meaningful diverse behaviors remains a challenge. Typically, task performance needs to be compromised in order to achieve diversity. In many scenarios, task requirements are specified as a multitude of reward terms, each requiring a different trade-off. In this work, we take a constrained optimization viewpoint on the quality-diversity trade-off and show that we can obtain diverse policies while imposing constraints on their value functions which are defined through distinct rewards. In line with previous work, further control of the diversity level can be achieved through an attract-repel reward term motivated by the Van der Waals force. We demonstrate the effectiveness of our method on a local navigation task where a quadruped robot needs to reach the target within a finite horizon. Finally, our trained policies transfer well to the real 12-DoF quadruped robot, Solo12, and exhibit diverse agile behaviors with successful obstacle traversal., Comment: 7 pages, 6 figures, in submission to ICRA 2024

The NA61/SHINE experiment at the CERN Super Proton Synchrotron studies the onset of deconfinement in strongly interacting matter through a beam energy scan of particle production in collisions of nuclei of varied sizes. This paper presents results on inclusive double-differential spectra, transverse momentum and rapidity distributions and mean multiplicities of $\pi^\pm$, $K^\pm$, $p$ and $\bar{p}$ produced in $^{40}$Ar+$^{45}$Sc collisions at beam momenta of 13$A$, 19$A$, 30$A$, 40$A$, 75$A$ and 150$A$~\GeVc. The analysis uses the 10\% most central collisions, where the observed forward energy defines centrality. The energy dependence of the $K^\pm$/$\pi^\pm$ ratios as well as of inverse slope parameters of the $K^\pm$ transverse mass distributions are placed in between those found in inelastic $p$+$p$ and central Pb+Pb collisions. The results obtained here establish a system-size dependence of hadron production properties that so far cannot be explained either within statistical or dynamical models.

Experimentation on real robots is demanding in terms of time and costs. For this reason, a large part of the reinforcement learning (RL) community uses simulators to develop and benchmark algorithms. However, insights gained in simulation do not necessarily translate to real robots, in particular for tasks involving complex interactions with the environment. The Real Robot Challenge 2022 therefore served as a bridge between the RL and robotics communities by allowing participants to experiment remotely with a real robot - as easily as in simulation. In the last years, offline reinforcement learning has matured into a promising paradigm for learning from pre-collected datasets, alleviating the reliance on expensive online interactions. We therefore asked the participants to learn two dexterous manipulation tasks involving pushing, grasping, and in-hand orientation from provided real-robot datasets. An extensive software documentation and an initial stage based on a simulation of the real set-up made the competition particularly accessible. By giving each team plenty of access budget to evaluate their offline-learned policies on a cluster of seven identical real TriFinger platforms, we organized an exciting competition for machine learners and roboticists alike. In this work we state the rules of the competition, present the methods used by the winning teams and compare their results with a benchmark of state-of-the-art offline RL algorithms on the challenge datasets., Comment: Typo in author list fixed

Learning policies from previously recorded data is a promising direction for real-world robotics tasks, as online learning is often infeasible. Dexterous manipulation in particular remains an open problem in its general form. The combination of offline reinforcement learning with large diverse datasets, however, has the potential to lead to a breakthrough in this challenging domain analogously to the rapid progress made in supervised learning in recent years. To coordinate the efforts of the research community toward tackling this problem, we propose a benchmark including: i) a large collection of data for offline learning from a dexterous manipulation platform on two tasks, obtained with capable RL agents trained in simulation; ii) the option to execute learned policies on a real-world robotic system and a simulation for efficient debugging. We evaluate prominent open-sourced offline reinforcement learning algorithms on the datasets and provide a reproducible experimental setup for offline reinforcement learning on real systems., Comment: The Eleventh International Conference on Learning Representations. 2022. Published at ICLR 2023. Datasets available at https://github.com/rr-learning/trifinger_rl_datasets

This paper presents an innovative automatic fusion imaging system that combines 3D CT/MR images with real-time ultrasound (US) acquisition. The system eliminates the need for external physical markers and complex training, making image fusion feasible for physicians with different experience levels. The integrated system involves a portable 3D camera for patient-specific surface acquisition, an electromagnetic tracking system, and US components. The fusion algorithm comprises two main parts: skin segmentation and rigid co-registration, both integrated into the US machine. The co-registration software aligns the surface extracted from CT/MR images with patient-specific coordinates, facilitating rapid and effective fusion. Experimental testing in different settings validates the system's accuracy, computational efficiency, noise robustness, and operator independence. The co-registration error remains under the acceptable range of~$1$ cm.

There has been significant recent progress in the area of unsupervised skill discovery, utilizing various information-theoretic objectives as measures of diversity. Despite these advances, challenges remain: current methods require significant online interaction, fail to leverage vast amounts of available task-agnostic data and typically lack a quantitative measure of skill utility. We address these challenges by proposing a principled offline algorithm for unsupervised skill discovery that, in addition to maximizing diversity, ensures that each learned skill imitates state-only expert demonstrations to a certain degree. Our main analytical contribution is to connect Fenchel duality, reinforcement learning, and unsupervised skill discovery to maximize a mutual information objective subject to KL-divergence state occupancy constraints. Furthermore, we demonstrate the effectiveness of our method on the standard offline benchmark D4RL and on a custom offline dataset collected from a 12-DoF quadruped robot for which the policies trained in simulation transfer well to the real robotic system., Comment: RLC 2024