Search

Estimating hyperparameters has been a long-standing problem in machine learning. We consider the case where the task at hand is modeled as the solution to an optimization problem. Here the exact gradient with respect to the hyperparameters cannot be feasibly computed and approximate strategies are required. We introduce a unified framework for computing hypergradients that generalizes existing methods based on the implicit function theorem and automatic differentiation/backpropagation, showing that these two seemingly disparate approaches are actually tightly connected. Our framework is extremely flexible, allowing its subproblems to be solved with any suitable method, to any degree of accuracy. We derive a priori and computable a posteriori error bounds for all our methods and numerically show that our a posteriori bounds are usually more accurate. Our numerical results also show that, surprisingly, for efficient bilevel optimization, the choice of hypergradient algorithm is at least as important as the choice of lower-level solver. [ABSTRACT FROM AUTHOR]

We present a model-based derivative-free method for optimization subject to general convex constraints, which we assume are unrelaxable and accessed only through a projection operator that is cheap to evaluate. We prove global convergence and a worst-case complexity of O(ε-2) iterations and objective evaluations for nonconvex functions, matching results for the unconstrained case. We introduce new, weaker requirements on model accuracy compared to existing methods. As a result, sufficiently accurate interpolation models can be constructed only using feasible points. We develop a comprehensive theory of interpolation set management in this regime for linear and composite linear models. We implement our approach for nonlinear least-squares problems and demonstrate strong practical performance compared to general-purpose solvers. [ABSTRACT FROM AUTHOR]

We investigate the potential of applying a state-of-the-art, local derivative-free solver, Py-BOBYQA to global optimization problems. In particular, we demonstrate the potential of a restarts procedure – as distinct from multistart methods – to allow Py-BOBYQA to escape local minima (where ordinarily it would terminate at the first local minimum found). We also introduce an adaptive variant of restarts which yields improved performance on global optimization problems. As Py-BOBYQA is a model-based trust-region method, we compare largely with other global optimization methods for which (global) models are important, such as Bayesian optimization and response surface methods; we also consider state-of-the-art representative deterministic and stochastic codes, such as DIRECT and CMA-ES. We find numerically that the restarts procedures in Py-BOBYQA are effective at helping it to escape local minima, when compared to using no restarts in Py-BOBYQA. Additionally, we find that Py-BOBYQA with adaptive restarts has comparable performance with global optimization solvers for all accuracy/budget regimes, in both smooth and noisy settings. In particular, Py-BOBYQA variants are best performing for smooth and multiplicative noise problems in high-accuracy regimes. As a by-product, some preliminary conclusions can be drawn on the relative performance of the global solvers we have tested with default settings. [ABSTRACT FROM AUTHOR]

Uncertainty in climate projections is large as shown by the likely uncertainty ranges in equilibrium climate sensitivity (ECS) of 2.5–4 K and in the transient climate response (TCR) of 1.4–2.2 K. Uncertainty in model projections could arise from the way in which unresolved processes are represented, the parameter values used, or the targets for model calibration. We show that, in two climate model ensembles that were objectively calibrated to minimize differences from observed large-scale atmospheric climatology, uncertainties in ECS and TCR are about 2–6 times smaller than in the CMIP5 or CMIP6 multimodel ensemble. We also find that projected uncertainties in surface temperature, precipitation, and annual extremes are relatively small. Residual uncertainty largely arises from unconstrained sea ice feedbacks. The more than 20-year-old HadAM3 standard model configuration simulates observed hemispheric-scale observations and preindustrial surface temperatures about as well as the median CMIP5 and CMIP6 ensembles while the optimized configurations simulate these better than almost all the CMIP5 and CMIP6 models. Hemispheric-scale observations and preindustrial temperatures are not systematically better simulated in CMIP6 than in CMIP5 although the CMIP6 ensemble seems to better simulate patterns of large-scale observations than the CMIP5 ensemble and the optimized HadAM3 configurations. Our results suggest that most CMIP models could be improved in their simulation of large-scale observations by systematic calibration. However, the uncertainty in climate projections (for a given scenario) likely largely arises from the choice of parameterization schemes for unresolved processes ("structural uncertainty"), with different tuning targets another possible contributor. Significance Statement: Climate models represent unresolved phenomena controlled by uncertain parameters. Changes in these parameters impact how well a climate model simulates current climate and its climate projections. Multiple calibrations of a single climate model, using an objective method, to large-scale atmospheric observations are performed. These models produce very similar climate projections at both global and regional scales. An analysis that combines uncertainties in observations with simulated sensitivity to observations and climate response also has small uncertainty showing that, for this model, current observations constrain climate projections. Recently developed climate models have a broad range of abilities to simulate large-scale climate with only some improvement in their ability to simulate this despite a decade of model development. [ABSTRACT FROM AUTHOR]

Ramming paste is a particular type of carbon paste which is used as lining for electric smelting furnaces and Hall-Héroult cells for the production of aluminium. The purpose of this lining is to form an impenetrable barrier, keeping the liquid within the furnace. If the lining has cracks or holes, then liquid can escape, which can lead to safety risks and financial losses, so the integrity of the lining is of great importance. In the present study, we develop a first principles mathematical model for the heat and mass transfer processes occurring during the baking of carbon paste. We then obtain numerical simulations using this model and compare the simulation results to experimental data, demonstrating that the model solutions do indeed describe and predict realistic behaviours of the carbon paste baking process. The simulations indicate a strong pressure buildup during the evaporation of water from fresh paste during the baking process, which is likely to lead to cracking of the paste as it hardens. Furthermore, we are able to show that more gradual heating during the baking process can lower the maximal pressures predicted by the model, which in turn may reduce the prevalence of cracks within the hardened paste. [ABSTRACT FROM AUTHOR]

A set of non-configurable transversely-displaced masks has been designed and fabricated to generate high-quality X-ray illumination patterns for use in imaging techniques such as ghost imaging (GI), ghost projection, and speckle tracking. The designs include a range of random binary and orthogonal patterns, fabricated through a combination of photolithography and gold electroplating techniques. We experimentally demonstrated that a single wafer can be used as an illumination mask for GI, employing individual illumination patterns and also a mixture of patterns, using a laboratory X-ray source. The quality of the reconstructed X-ray ghost images has been characterized and evaluated through a range of metrics.