Search

Dynamical loss functions are derived from standard loss functions used in supervised classification tasks, but they are modified such that the contribution from each class periodically increases and decreases. These oscillations globally alter the loss landscape without affecting the global minima. In this paper, we demonstrate how to transform cross-entropy and mean squared error into dynamical loss functions. We begin by discussing the impact of increasing the size of the neural network or the learning rate on the learning process. Building on this intuition, we propose several versions of dynamical loss functions and show how they significantly improve validation accuracy for networks of varying sizes. Finally, we explore how the landscape of these dynamical loss functions evolves during training, highlighting the emergence of instabilities that may be linked to edge-of-instability minimization.

Teams of mobile [aerial, ground, or aquatic] robots have applications in resource delivery, patrolling, information-gathering, agriculture, forest fire fighting, chemical plume source localization and mapping, and search-and-rescue. Robot teams traversing hazardous environments -- with e.g. rough terrain or seas, strong winds, or adversaries capable of attacking or capturing robots -- should plan and coordinate their trails in consideration of risks of disablement, destruction, or capture. Specifically, the robots should take the safest trails, coordinate their trails to cooperatively achieve the team-level objective with robustness to robot failures, and balance the reward from visiting locations against risks of robot losses. Herein, we consider bi-objective trail-planning for a mobile team of robots orienteering in a hazardous environment. The hazardous environment is abstracted as a directed graph whose arcs, when traversed by a robot, present known probabilities of survival. Each node of the graph offers a reward to the team if visited by a robot (which e.g. delivers a good to or images the node). We wish to search for the Pareto-optimal robot-team trail plans that maximize two [conflicting] team objectives: the expected (i) team reward and (ii) number of robots that survive the mission. A human decision-maker can then select trail plans that balance, according to their values, reward and robot survival. We implement ant colony optimization, guided by heuristics, to search for the Pareto-optimal set of robot team trail plans. As a case study, we illustrate with an information-gathering mission in an art museum., Comment: v0.0

Zero-knowledge proofs (ZKPs) represent a revolutionary advance in computational integrity and privacy technology, enabling the secure and private exchange of information without revealing underlying private data. ZKPs have unique advantages in terms of universality and minimal security assumptions when compared to other privacy-sensitive computational methods for distributed systems, such as homomorphic encryption and secure multiparty computation. Their application spans multiple domains, from enhancing privacy in blockchain to facilitating confidential verification of computational tasks. This survey starts with a high-level overview of the technical workings of ZKPs with a focus on an increasingly relevant subset of ZKPs called zk-SNARKS. While there have been prior surveys on the algorithmic and theoretical aspects of ZKPs, our work is distinguished by providing a broader view of practical aspects and describing many recently-developed use cases of ZKPs across various domains. These application domains span blockchain privacy, scaling, storage, and interoperability, as well as non-blockchain applications like voting, authentication, timelocks, and machine learning. Aimed at both practitioners and researchers, the survey also covers foundational components and infrastructure such as zero-knowledge virtual machines (zkVM), domain-specific languages (DSLs), supporting libraries, frameworks, and protocols. We conclude with a discussion on future directions, positioning ZKPs as pivotal in the advancement of cryptographic practices and digital privacy across many applications.

Since the breakthrough performance of AlexNet in 2012, convolutional neural networks (convnets) have grown into extremely powerful vision models. Deep learning researchers have used convnets to perform vision tasks with accuracy that was unachievable a decade ago. Confronted with the immense computation that convnets use, deep learning researchers also became interested in efficiency. However, the engineers who deployed efficient convnets soon realized that they were slower than the previous generation, despite using fewer operations. Many reverted to older models that ran faster. Hence researchers switched the objective of their search from arithmetic complexity to latency and produced a new wave of models that performed better. Paradoxically, these models also used more operations. Skepticism grew among researchers and engineers alike about the relevance of arithmetic complexity. Contrary to the prevailing view that latency and arithmetic complexity are irreconcilable, a simple formula relates both through computational efficiency. This insight enabled us to co-optimize the separate factors that determine latency. We observed that the degenerate conv2d layers that produce the best accuracy--complexity trade-off also use significant memory resources and have low computational efficiency. We devised block fusion algorithms to implement all the layers of a residual block in a single kernel, thereby creating temporal locality, avoiding communication, and reducing workspace size. Our ConvFirst model with block-fusion kernels has less arithmetic complexity and greater computational efficiency than baseline models and kernels, and ran approximately four times as fast as ConvNeXt. We also created novel tools, including efficiency gap plots and waterline analysis. Our unified approach to convnet efficiency envisions a new era of models and kernels that achieve greater accuracy at lower cost.

Group configurations play a pivotal role in shaping social skills and learning outcomes by fostering student interaction and collaboration. Yet, the factors influencing group formation, social cohesion, and group stability remain understudied. Here, we examine these factors in a cohort of 90 students from 9th and 10th grades at a private school in Chile. Students were either randomly assigned to groups (control) or allowed to choose their own groups (experimental) for a semester-long project. Our results show that self-selected groups had stronger friendship ties by the end of the semester. In contrast, randomly assigned groups exhibited higher levels of cooperation and more frequent membership changes, indicating weaker social cohesion. Additionally, 10th-grade students were more likely to stay in their original groups, possibly suggesting increased social maturity. These findings underscore the ongoing challenge in education of balancing student choice in group formation with the need for diverse group compositions that enhance learning. The study provides actionable insights for educators.

We demonstrate the selective control of the magnetic response and photoluminescence properties of Er3+ centers with light, by associating them with a highly conjugated beta-diketonate (1,3-di(2-naphthyl)-1,3-propanedione) ligand. We demonstrate this system to be an optically-pumped molecular compound emittingin infra-red, which can be employed as a precise heat-driving and detecting unit for low temperatures, Comment: 5 figures, version of record

Recent advances in machine learning and AI, including Generative AI and LLMs, are disrupting technological innovation, product development, and society as a whole. AI's contribution to technology can come from multiple approaches that require access to large training data sets and clear performance evaluation criteria, ranging from pattern recognition and classification to generative models. Yet, AI has contributed less to fundamental science in part because large data sets of high-quality data for scientific practice and model discovery are more difficult to access. Generative AI, in general, and Large Language Models in particular, may represent an opportunity to augment and accelerate the scientific discovery of fundamental deep science with quantitative models. Here we explore and investigate aspects of an AI-driven, automated, closed-loop approach to scientific discovery, including self-driven hypothesis generation and open-ended autonomous exploration of the hypothesis space. Integrating AI-driven automation into the practice of science would mitigate current problems, including the replication of findings, systematic production of data, and ultimately democratisation of the scientific process. Realising these possibilities requires a vision for augmented AI coupled with a diversity of AI approaches able to deal with fundamental aspects of causality analysis and model discovery while enabling unbiased search across the space of putative explanations. These advances hold the promise to unleash AI's potential for searching and discovering the fundamental structure of our world beyond what human scientists have been able to achieve. Such a vision would push the boundaries of new fundamental science rather than automatize current workflows and instead open doors for technological innovation to tackle some of the greatest challenges facing humanity today., Comment: 35 pages, first draft of the final report from the Alan Turing Institute on AI for Scientific Discovery

Large-scale asymmetries (i.e. lopsidedness) are a common feature in the stellar density distribution of nearby disk galaxies both in low- and high-density environments. In this work, we characterize the present-day lopsidedness in a sample of 1435 disk-like galaxies selected from the TNG50 simulation. We find that the percentage of lopsided galaxies (10%-30%) is in good agreement with observations if we use similar radial ranges to the observations. However, the percentage (58%) significantly increases if we extend our measurement to larger radii. We find a mild or lack of correlation between lopsidedness amplitude and environment at z=0 and a strong correlation between lopsidedness and galaxy morphology regardless of the environment. Present-day galaxies with more extended disks, flatter inner galactic regions and lower central stellar mass density (i.e. late-type disk galaxies) are typically more lopsided than galaxies with smaller disks, rounder inner galactic regions and higher central stellar mass density (i.e. early-type disk galaxies). Interestingly, we find that lopsided galaxies have, on average, a very distinct star formation history within the last 10 Gyr, with respect to their symmetric counterparts. Symmetric galaxies have typically assembled at early times (~8-6 Gyr ago) with relatively short and intense bursts of central star formation, while lopsided galaxies have assembled on longer timescales and with milder initial bursts of star formation, continuing building up their mass until z=0. Overall, these results indicate that lopsidedness in present-day disk galaxies is connected to the specific evolutionary histories of the galaxies that shaped their distinct internal properties., Comment: Submitted to MNRAS, 23 pages (including the Appendix), 23 figures

For general-sum, n-player, strategic games with transferable utility, the Harsanyi-Shapley value provides a computable method to both 1) quantify the strategic value of a player; and 2) make cooperation rational through side payments. We give a simple formula to compute the HS value in normal-form games. Next, we provide two methods to generalize the HS values to stochastic (or Markov) games, and show that one of them may be computed using generalized Q-learning algorithms. Finally, an empirical validation is performed on stochastic grid-games with three or more players. Source code is provided to compute HS values for both the normal-form and stochastic game setting., Comment: 8 pages in main work, 6 figures, ICML submission

Disc galaxies commonly show asymmetric features in their morphology, such as warps and lopsidedness. These features can provide key information regarding the recent evolution of a given disc galaxy. In the nearby Universe, up to $\sim30$ percent of late-type galaxies display a global non-axisymmetric lopsided mass distribution. However, the origin of this perturbation is not well understood. In this work, we study the origin of lopsided perturbations in simulated disc galaxies extracted from the TNG50 simulation of the IllustrisTNG project. We statistically explore different excitation mechanisms for this perturbation, such as direct satellite tidal interactions and distortions of the underlying dark matter distributions. We also characterize the main physical conditions that lead to lopsided perturbations. 50 percent of our sample galaxy have lopsided modes $m=1$ greater than $\sim 0.12$. We find a strong correlation between internal galaxy properties, such as central stellar surface density and disc radial extension with the strength of lopsided modes. The majority of lopsided galaxies have lower central surface densities and more extended discs than symmetric galaxies. As a result, such lopsided galaxies are less self-gravitationally cohesive, and their outer disc region is more susceptible to different types of external perturbations. However, we do not find strong evidence that tidal interactions with satellite galaxies are the main driving agent of lopsided modes. Lopsided galaxies tend to live in asymmetric dark matter halos with high spin, indicating strong galaxy-halo connections in late-type lopsided galaxies., Comment: 16 pages, 16 figures, accepted for publication in MNRAS

Soil and groundwater contamination is a pervasive problem at thousands of locations across the world. Contaminated sites often require decades to remediate or to monitor natural attenuation. Climate change exacerbates the long-term site management problem because extreme precipitation and/or shifts in precipitation/evapotranspiration regimes could re-mobilize contaminants and proliferate affected groundwater. To quickly assess the spatiotemporal variations of groundwater contamination under uncertain climate disturbances, we developed a physics-informed machine learning surrogate model using U-Net enhanced Fourier Neural Operator (U-FNO) to solve Partial Differential Equations (PDEs) of groundwater flow and transport simulations at the site scale.We develop a combined loss function that includes both data-driven factors and physical boundary constraints at multiple spatiotemporal scales. Our U-FNOs can reliably predict the spatiotemporal variations of groundwater flow and contaminant transport properties from 1954 to 2100 with realistic climate projections. In parallel, we develop a convolutional autoencoder combined with online clustering to reduce the dimensionality of the vast historical and projected climate data by quantifying climatic region similarities across the United States. The ML-based unique climate clusters provide climate projections for the surrogate modeling and help return reliable future recharge rate projections immediately without querying large climate datasets. In all, this Multi-scale Digital Twin work can advance the field of environmental remediation under climate change., Comment: 5 pages, 2 figures, 1 table, Machine Learning and the Physical Sciences workshop, NeurIPS 2022

A ''technology lottery'' describes a research idea or technology succeeding over others because it is suited to the available software and hardware, not necessarily because it is superior to alternative directions--examples abound, from the synergies of deep learning and GPUs to the disconnect of urban design and autonomous vehicles. The nascent field of Self-Driving Laboratories (SDL), particularly those implemented as Materials Acceleration Platforms (MAPs), is at risk of an analogous pitfall: the next logical step for building MAPs is to take existing lab equipment and workflows and mix in some AI and automation. In this whitepaper, we argue that the same simulation and AI tools that will accelerate the search for new materials, as part of the MAPs research program, also make possible the design of fundamentally new computing mediums. We need not be constrained by existing biases in science, mechatronics, and general-purpose computing, but rather we can pursue new vectors of engineering physics with advances in cyber-physical learning and closed-loop, self-optimizing systems. Here we outline a simulation-based MAP program to design computers that use physics itself to solve optimization problems. Such systems mitigate the hardware-software-substrate-user information losses present in every other class of MAPs and they perfect alignment between computing problems and computing mediums eliminating any technology lottery. We offer concrete steps toward early ''Physical Computing (PC) -MAP'' advances and the longer term cyber-physical R&D which we expect to introduce a new era of innovative collaboration between materials researchers and computer scientists.

There is a significant need for principled uncertainty reasoning in machine learning systems as they are increasingly deployed in safety-critical domains. A new approach with uncertainty-aware regression-based neural networks (NNs), based on learning evidential distributions for aleatoric and epistemic uncertainties, shows promise over traditional deterministic methods and typical Bayesian NNs, notably with the capabilities to disentangle aleatoric and epistemic uncertainties. Despite some empirical success of Deep Evidential Regression (DER), there are important gaps in the mathematical foundation that raise the question of why the proposed technique seemingly works. We detail the theoretical shortcomings and analyze the performance on synthetic and real-world data sets, showing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification. We go on to discuss corrections and redefinitions of how aleatoric and epistemic uncertainties should be extracted from NNs., Comment: 11 pages, 25 figures