Search

Reinforcement Learning from Human feedback (RLHF) has become a powerful tool to fine-tune or train agentic machine learning models. Similar to how humans interact in social contexts, we can use many types of feedback to communicate our preferences, intentions, and knowledge to an RL agent. However, applications of human feedback in RL are often limited in scope and disregard human factors. In this work, we bridge the gap between machine learning and human-computer interaction efforts by developing a shared understanding of human feedback in interactive learning scenarios. We first introduce a taxonomy of feedback types for reward-based learning from human feedback based on nine key dimensions. Our taxonomy allows for unifying human-centered, interface-centered, and model-centered aspects. In addition, we identify seven quality metrics of human feedback influencing both the human ability to express feedback and the agent's ability to learn from the feedback. Based on the feedback taxonomy and quality criteria, we derive requirements and design choices for systems learning from human feedback. We relate these requirements and design choices to existing work in interactive machine learning. In the process, we identify gaps in existing work and future research opportunities. We call for interdisciplinary collaboration to harness the full potential of reinforcement learning with data-driven co-adaptive modeling and varied interaction mechanics.

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

In this work, we present a lattice QCD calculation of the Mellin moments of the twist-2 axial-vector generalized parton distribution (GPD), $\widetilde{H}(x,\xi,t)$, at zero skewness, $\xi$, with multiple values of the momentum transfer, $t$. Our analysis employs the short-distance factorization framework on ratio-scheme renormalized quasi-GPD matrix elements. The calculations are based on an $N_f=2+1+1$ twisted mass fermions ensemble with clover improvement, a lattice spacing of $a = 0.093$ fm, and a pion mass of $m_\pi = 260$ MeV. We consider both the iso-vector and iso-scalar cases, utilizing next-to-leading-order perturbative matching while ignoring the disconnected contributions and gluon mixing in the iso-scalar case. For the first time, we determine the Mellin moments of $\widetilde{H}$ up to the fifth order. From these moments, we discuss the quark helicity and orbital angular momentum contributions to the nucleon spin, as well as the spin-orbit correlations of the quarks. Additionally, we perform a Fourier transform over the momentum transfer, which allows us to explore the spin structure in the impact-parameter space., Comment: 17 pages, 13 figures

Telescope missions are currently being designed which will make direct imaging of habitable exoplanets possible in the near future, and studies are needed to quantify the detectability of biosignature features in the planet's reflectance spectrum. We simulated the detectability of a NIR-absorbing surface biosignature feature with simulated observations of the nearby exoplanet Proxima Centauri b. We modeled a biosignature spectral feature with a reflectance spectrum based on an anoxygenic photosynthetic bacterial species that has strong absorption at 1 um, which could make it well suited for life on an M-dwarf hosted planet. We modeled the distribution of this organism across the planet's surface based on climate states from a 3D General Circulation Model (GCM), which were Archean and Proterozoic-like exo-Earth analogues. We included the GCM runs' prognostically simulated water clouds and added organic haze into the Archean-like atmospheres. We simulated observations of these Proxima Centauri b scenarios with the LUVOIR-A and B telescope concepts, with LUVOIR-B serving as a proxy to the planned Habitable Worlds Observatory (HWO). We calculated integration times necessary to detect the biosignature, and found that it would be detectable on Proxima Centauri b if the organism is moderately abundant (greater than a 1-4% global surface area coverage), as long as the atmosphere is transmitting in the wavelength range under consideration. Small amounts of methane, clouds, and haze do not greatly impede detectability. We found preliminary evidence that such a biosignature would be detectable on exoplanets within 15 pc, but further investigations are needed to corroborate this., Comment: 31 pages, 9 figures, 2 tables

Most non-relativistic interacting quantum many-body systems, such as atomic and molecular ensembles or materials, are naturally described in terms of continuous-space Hamiltonians. The simulation of their ground-state properties on digital quantum computers is challenging because current algorithms require discretization, which usually amounts to choosing a finite basis set, inevitably introducing errors. In this work, we propose an alternative, discretization-free approach that combines classical and quantum resources in a global variational ansatz, optimized using the framework of variational Monte Carlo. We introduce both purely quantum as well as hybrid quantum-classical ansatze and benchmark them on three paradigmatic continuous-space systems that are either very challenging or beyond the reach of current quantum approaches: the one-dimensional quantum rotor model, a system of Helium-3 particles in one and two dimensions, and the two-dimensional homogeneous electron gas. We embed relevant constraints such as the antisymmetry of fermionic wave functions directly into the ansatz. Many-body correlations are introduced via backflow transformations represented by parameterized quantum circuits. We demonstrate that the accuracy of the simulation can be systematically improved by increasing the number of circuit parameters and study the effects of shot noise. Furthermore, we show that the hybrid ansatz improves the ground-state energies obtained using the purely classical wave function., Comment: 10 pages, 6 figures. App: 8 pages, 3 figures

The physics case for quarkonium-production studies accessible at the US Electron Ion Collider is described., Comment: Latex, 84 pages. Review prepared for Progress in Particle and Nuclear Physics

We introduce a random matrix model for the stationary covariance of multivariate Ornstein-Uhlenbeck processes with heterogeneous temperatures, where the covariance is constrained by the Sylvester-Lyapunov equation. Using the replica method, we compute the spectral density of the equal-time covariance matrix characterizing the stationary states, demonstrating that this model undergoes a transition between stable and unstable states. In the stable regime, the spectral density has a finite and positive support, whereas negative eigenvalues emerge in the unstable regime. We determine the critical line separating these regimes and show that the spectral density exhibits a power-law tail at marginal stability, with an exponent independent of the temperature distribution. Additionally, we compute the spectral density of the lagged covariance matrix characterizing the stationary states of linear transformations of the original dynamical variables. Our random-matrix model is potentially interesting to understand the spectral properties of empirical correlation matrices appearing in the study of complex systems., Comment: 10 pages, 6 figures

We study the spectral properties and eigenvector statistics of the Laplacian on highly-connected networks with random coupling strengths and a gamma distribution of rescaled degrees. The spectral density, the distribution of the local density of states, the singularity spectrum and the multifractal exponents of this model exhibit a rich behaviour as a function of the first two moments of the coupling strengths and the variance of the rescaled degrees. In the case of random coupling strengths, the spectral density diverges within the bulk of the spectrum when degree fluctuations are strong enough. The emergence of this singular behaviour marks a transition from non-ergodic delocalized states to localized eigenvectors that exhibit pronounced multifractal scaling. For constant coupling strengths, the bulk of the spectrum is characterized by a regular spectral density. In this case, the corresponding eigenvectors display localization properties reminiscent of the critical point of the Anderson localization transition on random graphs., Comment: 32 pages, 9 figures

Dynamically scheduled hardware enables high-level synthesis (HLS) for applications with irregular control flow and latencies, which perform poorly with conventional statically scheduled approaches. Since dynamically scheduled hardware is inherently data flow based, it is beneficial to have an intermediate representation (IR) that captures the global data flow to enable easier transformations. State-of-the-art dynamic HLS utilize control flow based IRs, which model data flow only at the basic block level, requiring the rediscovery of inter-block parallelism. The Regionalized Value State Dependence Graph (RVSDG) is an IR that models (1) control flow as part of the global data flow utilizing regions and (2) memory dependencies using state edges. We propose R-HLS, a new RVSDG dialect targeted for dynamic high-level synthesis. R-HLS explicitly models control flow decisions, routing, and memory, which are only abstractly represented in the RVSDG. Expressing the control flow as part of the data flow reduces the need for complex optimizations to extract performance and enables easy conversion to parallel circuits. Furthermore, we present a distributed memory disambiguation optimization that leverages memory state edges to decouple address generation from data accesses, resulting in resource efficient out-of-program-order execution of memory operations. Our results show that R-HLS effectively exposes parallelism, resulting in fewer executed cycles and a 10% speedup on average, compared to the state-of-the-art in dynamic HLS with optimized memory disambiguation. These results are achieved with a significant reduction in resource utilization, such as a 79% reduction in lookup-tables and 22% reduction in flip-flops, on average., Comment: ICCAD 2024

In the Standard Model of particle physics, the axial current is not conserved, due both to fermion masses and to the axial anomaly. Using perturbative quantum chromodynamics, we calculate matrix elements of the local and non-local axial current for a gluon target, clarifying their connection with the axial anomaly. In so doing, we also reconsider classic results obtained in the context of the nucleon spin sum rule as well as recent results for off-forward kinematics. An important role is played by the infrared regulator, for which we put a special emphasis on the nonzero quark mass. We highlight cancellations that take place between contributions from the axial anomaly and the quark mass, and we elaborate on the relation of those cancellations with the conservation of angular momentum., Comment: 12 pages

We give a new construction of the Brownian annulus based on removing a hull centered at the distinguished point in the free Brownian disk. We use this construction to prove that the Brownian annulus is the scaling limit of Boltzmann triangulations with two boundaries. We also prove that the space obtained by removing hulls centered at the two distinguished points of the Brownian sphere is a Brownian annulus. Our proofs rely on a detailed analysis of the peeling by layers algorithm for Boltzmann triangulations with a boundary., Comment: 47 pages, 4 figures

Efficient public transport systems are crucial for sustainable urban development as cities face increasing mobility demands. Yet, many public transport networks struggle to meet diverse user needs due to historical development, urban constraints, and financial limitations. Traditionally, planning of transport network structure is often based on limited surveys, expert opinions, or partial usage statistics. This provides an incomplete basis for decision-making. We introduce an data-driven approach to public transport planning and optimization, calculating detailed accessibility measures at the individual housing level. Our visual analytics workflow combines population-group-based simulations with dynamic infrastructure analysis, utilizing a scenario-based model to simulate daily travel patterns of varied demographic groups, including schoolchildren, students, workers, and pensioners. These population groups, each with unique mobility requirements and routines, interact with the transport system under different scenarios traveling to and from Points of Interest (POI), assessed through travel time calculations. Results are visualized through heatmaps, density maps, and network overlays, as well as detailed statistics. Our system allows us to analyze both the underlying data and simulation results on multiple levels of granularity, delivering both broad insights and granular details. Case studies with the city of Konstanz, Germany reveal key areas where public transport does not meet specific needs, confirmed through a formative user study. Due to the high cost of changing legacy networks, our analysis facilitates the identification of strategic enhancements, such as optimized schedules or rerouting, and few targeted stop relocations, highlighting consequential variations in accessibility to pinpointing critical service gaps., Comment: 9 pages, 8 figures

Using partial knowledge of a quantum state to control multiqubit entanglement is a largely unexplored paradigm in the emerging field of quantum interactive dynamics with the potential to address outstanding challenges in quantum state preparation and compression, quantum control, and quantum complexity. We present a deep reinforcement learning (RL) approach to constructing short disentangling circuits for arbitrary 4-, 5-, and 6-qubit states using an actor-critic algorithm. With access to only two-qubit reduced density matrices, our agent decides which pairs of qubits to apply two-qubit gates on; requiring only local information makes it directly applicable on modern NISQ devices. Utilizing a permutation-equivariant transformer architecture, the agent can autonomously identify qubit permutations within the state, and adjusts the disentangling protocol accordingly. Once trained, it provides circuits from different initial states without further optimization. We demonstrate the agent's ability to identify and exploit the entanglement structure of multiqubit states. For 4-, 5-, and 6-qubit Haar-random states, the agent learns to construct disentangling circuits that exhibit strong correlations both between consecutive gates and among the qubits involved. Through extensive benchmarking, we show the efficacy of the RL approach to find disentangling protocols with minimal gate resources. We explore the resilience of our trained agents to noise, highlighting their potential for real-world quantum computing applications. Analyzing optimal disentangling protocols, we report a general circuit to prepare an arbitrary 4-qubit state using at most 5 two-qubit (10 CNOT) gates., Comment: The source code as well as a demo in the form of an interactive Jupyter notebook are available on Github: https://github.com/mgbukov/RL_disentangle

Although real-world complex systems typically interact through sparse and heterogeneous networks, analytic solutions of their dynamics are limited to models with all-to-all interactions. Here, we solve the dynamics of a broad range of nonlinear models of complex systems on sparse directed networks with a random structure. By generalizing dynamical mean-field theory to sparse systems, we derive an exact equation for the path-probability describing the effective dynamics of a single degree of freedom. Our general solution applies to key models in the study of neural networks, ecosystems, epidemic spreading, and synchronization. Using the population dynamics algorithm, we solve the path-probability equation to determine the phase diagram of a seminal neural network model in the sparse regime, showing that this model undergoes a transition from a fixed-point phase to chaos as a function of the network topology., Comment: 19 pages, 5 figures

This paper explores the combination of neural network quantization and entropy coding for memory footprint minimization. Edge deployment of quantized models is hampered by the harsh Pareto frontier of the accuracy-to-bitwidth tradeoff, causing dramatic accuracy loss below a certain bitwidth. This accuracy loss can be alleviated thanks to mixed precision quantization, allowing for more flexible bitwidth allocation. However, standard mixed precision benefits remain limited due to the 1-bit frontier, that forces each parameter to be encoded on at least 1 bit of data. This paper introduces an approach that combines mixed precision, zero-point quantization and entropy coding to push the compression boundary of Resnets beyond the 1-bit frontier with an accuracy drop below 1% on the ImageNet benchmark. From an implementation standpoint, a compact decoder architecture features reduced latency, thus allowing for inference-compatible decoding., Comment: 5 pages, 5 figures, to be published in ISCAS 2024

Generalized parton distributions (GPDs) are key quantities for the description of a hadron's three-dimensional structure. They are the current focus of all areas of hadronic physics -- phenomenological, experimental, and theoretical, including lattice QCD. Synergies between these areas are desirable and essential to achieve precise quantification and understanding of the structure of, particularly nucleons, as the basic ingredients of matter. In this paper, we investigate, for the first time, the numerical implementation of the pseudo-distribution approach for the extraction of zero-skewness GPDs for unpolarized quarks. Pseudo-distributions are Euclidean parton correlators computable in lattice QCD that can be perturbatively matched to the light-cone parton distributions of interest. Being closely related to the quasi-distributions and coming from the same lattice-extracted matrix elements, they are, however, subject to different systematic effects. We use the data previously utilized for quasi-GPDs and extend it with other momentum transfers and nucleon boosts, in particular a higher one ($P_3=1.67$ GeV) with eight-fold larger statistics than the largest one used for quasi-distributions ($P_3=1.25$ GeV). We renormalize the matrix elements with a ratio scheme and match the resulting Ioffe time distributions to the light cone in coordinate space. The matched distributions are then used to reconstruct the $x$-dependence with a fitting ansatz.We investigate some systematic effects related to this procedure, and we also compare the results with the ones obtained in the framework of quasi-GPDs. Our final results involve the invariant four-momentum transfer squared ($-t$) dependence of the flavor non-singlet ($u-d$) $H$ and $E$ GPDs., Comment: 35 pages, 29 figures, compatible with journal version

We derive exact equations for the spectral density of sparse networks with an arbitrary distribution of the number of single edges and triangles per node. These equations enable a systematic investigation of the effect of clustering on the spectral properties of the network adjacency matrix. In the case of heterogeneous networks, we demonstrate that the spectral density becomes more symmetric as the fluctuations in the triangle-degree sequence increase. This phenomenon is explained by the small clustering coefficient of networks with a large variance of the triangle-degree distribution. In the homogeneous case of regular clustered networks, we find that both perturbative and non-perturbative approximations fail to predict the spectral density in the high-connectivity limit. This suggests that traditional large-degree approximations may be ineffective in studying the spectral properties of networks with more complex motifs. Our theoretical results are fully confirmed by numerical diagonalizations of finite adjacency matrices., Comment: 12 pages, 6 figures

First lattice QCD calculations of $x$-dependent GPD have been performed in the (symmetric) Breit frame, where the momentum transfer is evenly divided between the initial and final hadron states. However, employing the asymmetric frame, we are able to obtain proton GPDs for multiple momentum transfers in a computationally efficient setup. In these proceedings, we focus on the helicity twist-2 GPD at zero skewness that gives access to the $\widetilde{H}$ GPD. We will cover the implementation of the asymmetric frame, its comparison to the Breit frame, and the dependence of the GPD on the squared four-momentum transfer, $-t$. The calculation is performed on an $N_f = 2+1+1$ ensemble of twisted mass fermions with a clover improvement. The mass of the pion for this ensemble is roughly 260 MeV., Comment: 9 pages, 4 figures, Contribution to The 40th International Symposium on Lattice Field Theory (Lattice 2023)

Biophysical modeling, particularly involving partial differential equations (PDEs), offers significant potential for tailoring disease treatment protocols to individual patients. However, the inverse problem-solving aspect of these models presents a substantial challenge, either due to the high computational requirements of model-based approaches or the limited robustness of deep learning (DL) methods. We propose a novel framework that leverages the unique strengths of both approaches in a synergistic manner. Our method incorporates a DL ensemble for initial parameter estimation, facilitating efficient downstream evolutionary sampling initialized with this DL-based prior. We showcase the effectiveness of integrating a rapid deep-learning algorithm with a high-precision evolution strategy in estimating brain tumor cell concentrations from magnetic resonance images. The DL-Prior plays a pivotal role, significantly constraining the effective sampling-parameter space. This reduction results in a fivefold convergence acceleration and a Dice-score of 95%.

The superconducting state of the heavy-fermion metal UTe$_2$ has attracted considerable interest because of evidence for spin-triplet Cooper pairing and non-trivial topology. Progress on these questions requires identifying the presence or absence of nodes in the superconducting gap function and their dimension. In this article we report a comprehensive study of the influence of disorder on the thermal transport in the superconducting state of UTe$_2$. Through detailed measurements of the magnetic field dependence of the thermal conductivity in the zero-temperature limit, we obtain clear evidence for the presence of point nodes in the superconducting gap for all samples with transition temperatures ranging from 1.6~K to 2.1~K obtained by different synthesis methods, including a refined self-flux method. This robustness implies the presence of symmetry-imposed nodes throughout the range studied, further confirmed via disorder-dependent calculations of the thermal transport in a model with a single pair of nodes. In addition to capturing the temperature dependence of the thermal conductivity up to $T_c$, this model allows us to limit the possible locations of the nodes, suggesting a B$_{1u}$ or B$_{2u}$ symmetry for the superconducting order parameter. Additionally, comparing the new, ultra-high conductivity samples to older samples reveals a crossover between a low-field and a high field regime at a single value of the magnetic field in all samples. In the high field regime, the thermal conductivity at different disorder levels differ from each other by a simple offset, suggesting that some simple principle determines the physics of the mixed state, a fact which may illuminate trends observed in other clean nodal superconductors.

Most of the half-Heusler RPtBi compounds (R=rare earth) host various surface states due to spin-orbit coupling driven topological band structure. While recent ARPES measurements ubiquitously reported the existence of surface states in RPtBi, their evidence by other experimental techniques remains elusive. Here we report the angle-dependent magnetic field response of electrical transport properties of YPtBi in both the normal and superconducting states. The angle dependence of both magnetoresistance and the superconducting upper critical field breaks the rotational symmetry of the cubic crystal structure, and the angle between the applied magnetic field and the measurement plane of a plate-like sample prevails. Furthermore, the measured upper critical field is notably higher than the bulk response for an in-plane magnetic field configuration, suggesting the presence of quasi-2D superconductivity. Our work suggests the transport properties cannot be explained solely by the bulk carrier response, requiring robust normal and superconducting surface states to flourish in YPtBi.

AI-driven models are increasingly deployed in operational analytics solutions, for instance, in investigative journalism or the intelligence community. Current approaches face two primary challenges: ethical and privacy concerns, as well as difficulties in efficiently combining heterogeneous data sources for multimodal analytics. To tackle the challenge of multimodal analytics, we present MULTI-CASE, a holistic visual analytics framework tailored towards ethics-aware and multimodal intelligence exploration, designed in collaboration with domain experts. It leverages an equal joint agency between human and AI to explore and assess heterogeneous information spaces, checking and balancing automation through Visual Analytics. MULTI-CASE operates on a fully-integrated data model and features type-specific analysis with multiple linked components, including a combined search, annotated text view, and graph-based analysis. Parts of the underlying entity detection are based on a RoBERTa-based language model, which we tailored towards user requirements through fine-tuning. An overarching knowledge exploration graph combines all information streams, provides in-situ explanations, transparent source attribution, and facilitates effective exploration. To assess our approach, we conducted a comprehensive set of evaluations: We benchmarked the underlying language model on relevant NER tasks, achieving state-of-the-art performance. The demonstrator was assessed according to intelligence capability assessments, while the methodology was evaluated according to ethics design guidelines. As a case study, we present our framework in an investigative journalism setting, supporting war crime investigations. Finally, we conduct a formative user evaluation with domain experts in law enforcement. Our evaluations confirm that our framework facilitates human agency and steering in security-sensitive applications., Comment: 6 pages, 3 figures, 1 table

The fields of implementation science and the study of research use in policy and practice travel on many of the same roads and share similar goals, chief among which is improving societal outcomes through the application of research. Both fields also attract interdisciplinary teams and create strikingly similar knowledge across contexts. However, key differences have emerged in these two fields of study and the assumptions they make in the empirical work. These differences provide opportunities to strengthen the next generation of both implementation science and research on research use. In this report, the authors highlight how similarities and differences in the methods, approaches, and evolution of each of these fields can contribute to mutually beneficial insights and potential alignments. They describe each field in detail, with an eye toward highlighting key assumptions and differences, and we conclude by discussing opportunities for learning across fields. Overall, the authors aim to inspire a dialogue that may build a stronger foundation for supporting evidence use in ways that achieve equitable outcomes for people and communities.