Search

Research on low‐light image enhancement focuses on improving image quality in dim conditions. Recently, deep learning has driven significant advancements, with many studies using neural networks to enhance low‐light images. However, most focus on complex network designs to increase nonlinearity, often neglecting the implicit information from local image transformations. This paper introduces an improved U‐net‐based method for low‐light enhancement, retaining the original encoding network and adding branch links in the decoding network. The method uses attention feature fusion to handle image noise and gradients separately, adjusting brightness through a gradient‐adaptive transform. This approach optimizes performance using loss functions like peak signal‐to‐noise ratio and colour consistency. Unlike previous methods, the approach emphasizes extracting implicit information from image gradients, achieving enhancement that aligns with the original brightness distribution. The result is enhanced images with high detail similarity to the original, achieved through end‐to‐end inference in experiments. [ABSTRACT FROM AUTHOR]

Implementation of reconfigurable metasurfaces demands a complex structure. The design methodology for metasurface elements based on PIN diodes requires targeted optimisation tailored to the varying application environments and cannot be directly transplanted from one methodological approach to another. In addition, inclusion of varactors often imposes undesirable high‐power consumption with complex analog drive circuitry. Furthermore, the design method for the reconfigurable metasurface element has not been simplified and unified, hindering the rapid deployment of metasurface applications. The authors propose a general design model based on low‐power embedded circuitry for a multi‐function metasurface element and give the corresponding numerical analysis. The authors present a reconfigurable method based on the general design model and prove that the method achieves independent regulation of amplitude and phase for the reflected electromagnetic wave. Through analysis of the sensitive area at the bottom of the embedded circuit, a gradient descent algorithm is proposed to find the optimised phase points according to device sensitivity of the application scenario and suitable strategy. Finally, the authors demonstrate that the designed metasurface element possesses phase control capability and decent phase consistency obtained via far‐field scanning experiments. The proposed element design method will provide a theoretical reference for the integrated design of metasurface. [ABSTRACT FROM AUTHOR]

The development of innovative materials, based on the modern technologies and processes, is the key factor to improve the energetic sustainability and reduce the environmental impact of electrical equipment. In particular, the modeling of magnetic hysteresis is crucial for the design and construction of electrical and electronic devices. In recent years, additive manufacturing techniques are playing a decisive role in the project and production of magnetic elements and circuits for applications in various engineering fields. To this aim, the use of the deep learning paradigm, integrated with the most common models of the magnetic hysteresis process, has become increasingly present in recent years. The intent of this paper is to provide the features of a wide range of deep learning tools to be applied to magnetic hysteresis context and beyond. The possibilities of building neural networks in hybrid form are innumerable, so it is not plausible to illustrate them in a single paper, but in the present context, several neural networks used in the scientific literature, integrated with various hysteretic mathematical models, including the well-known Preisach model, are compared. It is shown that this hybrid approach not only improves the modeling of hysteresis by significantly reducing computational time and efforts, but also offers new perspectives for the analysis and prediction of the behavior of magnetic materials, with significant implications for the production of advanced devices. [ABSTRACT FROM AUTHOR]

Abstract Implementation of reconfigurable metasurfaces demands a complex structure. The design methodology for metasurface elements based on PIN diodes requires targeted optimisation tailored to the varying application environments and cannot be directly transplanted from one methodological approach to another. In addition, inclusion of varactors often imposes undesirable high‐power consumption with complex analog drive circuitry. Furthermore, the design method for the reconfigurable metasurface element has not been simplified and unified, hindering the rapid deployment of metasurface applications. The authors propose a general design model based on low‐power embedded circuitry for a multi‐function metasurface element and give the corresponding numerical analysis. The authors present a reconfigurable method based on the general design model and prove that the method achieves independent regulation of amplitude and phase for the reflected electromagnetic wave. Through analysis of the sensitive area at the bottom of the embedded circuit, a gradient descent algorithm is proposed to find the optimised phase points according to device sensitivity of the application scenario and suitable strategy. Finally, the authors demonstrate that the designed metasurface element possesses phase control capability and decent phase consistency obtained via far‐field scanning experiments. The proposed element design method will provide a theoretical reference for the integrated design of metasurface.

Abstract Research on low‐light image enhancement focuses on improving image quality in dim conditions. Recently, deep learning has driven significant advancements, with many studies using neural networks to enhance low‐light images. However, most focus on complex network designs to increase nonlinearity, often neglecting the implicit information from local image transformations. This paper introduces an improved U‐net‐based method for low‐light enhancement, retaining the original encoding network and adding branch links in the decoding network. The method uses attention feature fusion to handle image noise and gradients separately, adjusting brightness through a gradient‐adaptive transform. This approach optimizes performance using loss functions like peak signal‐to‐noise ratio and colour consistency. Unlike previous methods, the approach emphasizes extracting implicit information from image gradients, achieving enhancement that aligns with the original brightness distribution. The result is enhanced images with high detail similarity to the original, achieved through end‐to‐end inference in experiments.

The Delayed Weighted Gradient Method (DWGM) is a two-step gradient algorithm that is efficient for the minimization of large scale strictly convex quadratic functions. It has orthogonality properties that make it to compete with the Conjugate Gradient (CG) method. Both methods calculate in sequence two step-sizes, CG minimizes the objective function and DWGM the gradient norm, alongside two search directions defined over first order current and previous iteration information. The objective of this work is to accelerate the recently developed extension of DWGM to nonquadratic strongly convex minimization problems. Our idea is to define the step-sizes of DWGM in a unique two dimensional convex quadratic optimization problem, calculating them simultaneously. Convergence of the resulting algorithm is analyzed. Comparative numerical experiments illustrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

The geometric optimization of crystal structures is a procedure widely used in computational chemistry that changes the geometrical placement of the particles inside a structure. It is called structural relaxation and constitutes a local minimization problem with a non‐convex objective function whose domain complexity increases according to the number of particles involved. This work studies the performance of the two most popular gradient methods in structural relaxation, Steepest Descent and Conjugate Gradient. Although frequently employed, there is a lack of their study in this context from an algorithmic point of view. The algorithms are initially benchmarked on the basis of a constant step size. Three concepts for designing dynamic step size rules are then examined in detail and analyzed. Results show that there is a trade‐off between convergence rate and the possibility of an experiment to succeed. In order to address this, a function is proposed as a formal means for assigning utility to each method based on preference. The function is built according to a recently introduced model of preference indication concerning algorithms with deadline and their run time. It introduces the quantification of the optimization algorithms' performance according to convergence speed and success rate, thus enabling the appointment of a specific algorithmic recipe as the best choice for balanced preferences. [ABSTRACT FROM AUTHOR]

Crystal structures are important formations, bases of materials used in everyday life. A lot of effort is dedicated to the creation of new materials, starting from the study and combination of crystal structures' properties[1] through the crystals' geometric optimization. In the meantime, an equal amount of effort is dedicated to the study and development of methods employed in optimization problems; this, nevertheless, is carried out independently, in the context of computer science. There is a need, therefore, to acknowledge and highlight the overlap between the two fields, whilst creating a common language and establishing the theoretical foundations on which the optimization methods can become physically interpreted in the case of crystals. This article provides a detailed introduction to the theory behind the geometric optimization of crystals from the point of view of computer science, so as to facilitate interdisciplinary cooperation. Its target is to familiarize scientists from different fields with the problem, its parameters, and their derivations, in order to motivate their assistance in the creation, implementation, and application of new methods for the geometric optimization of crystal structures. [ABSTRACT FROM AUTHOR]

In the field of designing power supply systems of open pits an important role is attributed to the problems of components reliability analysis, which can be solved by integrated application of different analysis methods for individual types of electrical equipment and network devices in power supply systems. The purpose of the work is to establish optimal parameters for an open-pit mine power supply at mining and processing complexes, and to conduct research that allows a technical and economic model of the system to be created. It has been established that mathematical statistics provides a number of methods for establishing the homogeneity (or heterogeneity) of the population of random variable values. When analyzing the data of failure statistics of electrical equipment, the method of comparative estimate of two sample averages (compared to the method of comparing the empirical distribution with the normal distribution or the moving average method) is the most acceptable. The paper presents accident rates of electrical equipment of open-pit substations and network devices of the quarry processed on the basis of data, also presents the relationship between the reliability of the circuit and its elements and shows the influence of protection reliability on the reliability of the circuit. Methods of optimizing open-pit power supply systems are presented, which consist in the use of first-order gradient methods, second-order methods or the so-called quadratic optimization methods, and random search methods. A group of gradient methods is the most widespread for solving nonlinear programming problems, among which the method of steepest descent should be considered primarily. The combination of random search with the penalty function method makes it possible to determine conditional extrema for the problem of optimizing the electrical grid and system operation mode. The method of random descent, which consists in determining the minimum of the target function according to an appropriate algorithm, is also considered, in which the direction of motion is generally given by a random vector uniformly distributed over a hyposphere.

We derive efficient algorithms to compute weakly Pareto optimal solutions for smooth, convex and unconstrained multiobjective optimization problems in general Hilbert spaces. To this end, we define a novel inertial gradient-like dynamical system in the multiobjective setting, which trajectories converge weakly to Pareto optimal solutions. Discretization of this system yields an inertial multiobjective algorithm which generates sequences that converge weakly to Pareto optimal solutions. We employ Nesterov acceleration to define an algorithm with an improved convergence rate compared to the plain multiobjective steepest descent method (Algorithm 1). A further improvement in terms of efficiency is achieved by avoiding the solution of a quadratic subproblem to compute a common step direction for all objective functions, which is usually required in first-order methods. Using a different discretization of our inertial gradient-like dynamical system, we obtain an accelerated multiobjective gradient method that does not require the solution of a subproblem in each step (Algorithm 2). While this algorithm does not converge in general, it yields good results on test problems while being faster than standard steepest descent. [ABSTRACT FROM AUTHOR]

Abstract Recent work has shown that deep reinforcement learning (DRL) is vulnerable to adversarial attacks, so that exploiting vulnerabilities in DRL systems through adversarial attack techniques has become a necessary prerequisite for building robust DRL systems. Compared to traditional deep learning systems, DRL systems are characterised by long sequential decisions rather than one‐step decision, so attackers must perform multi‐step attacks on them. To successfully attack a DRL system, the number of attacks must be minimised to avoid detecting by the victim agent and to ensure the effectiveness of the attack. Some selective attack methods proposed in recent researches, that is, attacking an agent at partial time steps, are not applicable to real‐time attack scenarios, although they can avoid detecting by the victim agent. A real‐time selective attack method that is applicable to environments with discrete action spaces is proposed. Firstly, the optimal attack threshold T for performing selective attacks in the environment Env is determined. Then, the observation states corresponding to when the value of the action preference function of the victim agent in multiple eposides exceeds the threshold T are added to the training set according to this threshold. Finally, a universal perturbation is generated based on this training set, and it is used to perform real‐time selective attacks on the victim agent. Comparative experiments show that our attack method can perform real‐time attacks while maintaining the attack effect and stealthiness.

Abstract In this paper, a method of stroke‐based rendering is proposed for image representation and reconstruction. The proposed method involves compositing a set of ellipses that greatly vary in location, size, rotation angle, color, and opacity. This study proves that alpha compositing is differentiable if a two‐dimensional Gaussian function is used to draw a solid ellipse. The gradient method can then be employed for automatically identifying the parameters of each ellipse such that the difference between the input image and the composited image is minimized. Experimental results indicate that the proposed method can represent various types of images including portraits, landscapes, buildings, street scenes, artificial objects, medical images etc. The proposed method can particularly render the most details and significant features of an input image with fewer strokes compared to other stroke‐based rendering algorithms. This study also demonstrates that the proposed method can be applied in painting style simulation and sparse‐view computed tomography imaging.

This paper presents a methodology and generalized motion framework for quantifying passive biomechanical stability and Range of Motion of human cadaveric specimens, using a position-controlled industrial robot and a wrist-mounted force/torque sensor. Many biomechanical studies on diarthrodial joints using human cadaveric specimens are published in the literature, using various test protocols and machines to apply the loading conditions. In these studies, laxity or mobility of the joints are quantified by measuring the magnitude of translations and rotations with respect to force and torque. The protocols and anatomical motions of the specimens are usually described high-level, textually, and from a medical perspective to a broad audience. The present paper aims to describe, from a technical perspective to a robotics audience, our method to perform biomechanical studies and how existing protocols can be replicated through parameterization using the existing textual descriptions. To accomplish this, we propose a generalized task space motion framework for performing biomechanical studies on diarthrodial joints. The generalization is made by defining the robot Tool Center Point at the cadaveric joint rotation center and aligning the specimen so the anatomical motions can be modeled in world frame or tool frame. The framework was successfully evaluated in a technical pilot study on the shoulder, using one cadaveric shoulder specimen and an established protocol from the literature. The specimen was tested in the intact state and in an injury state, with increased passive instability observed for the injury state compared to intact state.

In this paper, a method of stroke‐based rendering is proposed for image representation and reconstruction. The proposed method involves compositing a set of ellipses that greatly vary in location, size, rotation angle, color, and opacity. This study proves that alpha compositing is differentiable if a two‐dimensional Gaussian function is used to draw a solid ellipse. The gradient method can then be employed for automatically identifying the parameters of each ellipse such that the difference between the input image and the composited image is minimized. Experimental results indicate that the proposed method can represent various types of images including portraits, landscapes, buildings, street scenes, artificial objects, medical images etc. The proposed method can particularly render the most details and significant features of an input image with fewer strokes compared to other stroke‐based rendering algorithms. This study also demonstrates that the proposed method can be applied in painting style simulation and sparse‐view computed tomography imaging. [ABSTRACT FROM AUTHOR]

The paper presents a comparative analysis of the application of two nonlocal gradient-formulated models to evaluate the concrete cover degradation time. Calculations were made taking into account the increase in the volume of the steel ring around the perimeter of the reinforcement bar. The results of the calculations were compared with the results of experimental studies published in the literature and with the elastic-plastic model based on the Menetrey-Willam surface, in which the objectivity of the obtained results depends on the fracture energy. In addition, the paper compares solutions using different contact models and cohesion models.

Abstract The transmit waveform with ambiguity function specifications in the range‐Doppler plane has been of great interest in traditional single antenna radar systems in recent years. This paper considers the design of waveform sets with a desired slow‐time ambiguity function for multiple‐input multiple‐output radar. Specifically, it is desirable that the ambiguity function has low sidelobes where clutter occurs, as this reduces the impact of the clutter on target detection. Instead of unimodular waveforms, the authors consider amplitude modulation signals and introduce the bound constraint on signal amplitudes into the optimisation for the consideration of transmitter efficiency. The design problem is formulated as minimising the weighted integrated sidelobes level of the ambiguity function over the range‐Doppler plane under bound constraints. The resulting problem has a highly non‐linear objective function and is constrained by bound constraints and quadratic equality (constant energy) constraints, making it difficult to solve. The authors develop an efficient algorithm based on the sequential quadratic programming technique and non‐linear conjugate gradient method to solve it. Simulation results show that the proposed algorithm is superior to the existing methods.

Abstract The approximating sequence Riccati equation method is an efficient approach for solving the nonlinear optimal control problems, but its neglect of nonlinear dynamics and necessary optimality condition makes the control law difficult to satisfy the optimality. In this paper, an approximating pseudospectral method with state‐dependent coefficient optimization algorithm is proposed to solve this defect. By introducing the approximating pseudospectral method, the original nonlinear problem is transformed into a sequence of linear subproblems, which preserves the nonlinearity of solution. Then a state‐dependent coefficient optimization algorithm based on the gradient projection technique is proposed, which ensures the optimality of the control law. A double‐layer optimization structure is designed to facilitate the coordination between the approximating method and the optimization algorithm. Theoretical analysis proves the convergence of the proposed method. Comparative case studies illustrate the effectiveness in reducing the performance index and ensuring the optimality of the control law.

The transmit waveform with ambiguity function specifications in the range‐Doppler plane has been of great interest in traditional single antenna radar systems in recent years. This paper considers the design of waveform sets with a desired slow‐time ambiguity function for multiple‐input multiple‐output radar. Specifically, it is desirable that the ambiguity function has low sidelobes where clutter occurs, as this reduces the impact of the clutter on target detection. Instead of unimodular waveforms, the authors consider amplitude modulation signals and introduce the bound constraint on signal amplitudes into the optimisation for the consideration of transmitter efficiency. The design problem is formulated as minimising the weighted integrated sidelobes level of the ambiguity function over the range‐Doppler plane under bound constraints. The resulting problem has a highly non‐linear objective function and is constrained by bound constraints and quadratic equality (constant energy) constraints, making it difficult to solve. The authors develop an efficient algorithm based on the sequential quadratic programming technique and non‐linear conjugate gradient method to solve it. Simulation results show that the proposed algorithm is superior to the existing methods. [ABSTRACT FROM AUTHOR]

Today, with the development of camera imaging techniques, object detection studies are becoming popular for unmanned aerial vehicles and autonomous systems. However, in these systems, it is extremely important to be able to identify objects effectively and with minimum error in detection for tracking. In this study, an effective human detection system is proposed based on the use of support vector machine, histogram oriented gradient features, and aggregate channel features with AdaBoost classifier. In this proposed system, an adaptive attention system based on the convolutional neural network GoogleNet architecture is developed. Hence efficiency in human detection and monitoring is increased and the central processing unit works faster. Using different data (UAV123, UAV123@10fps, COCO, OpenImagesV6) the most efficient performance is obtained with 97.4% accuracy. In addition, up to 75% savings are achieved with efficient use of the central processing unit. The model in this study is a suitable model for unmanned aerial vehicles, autonomous systems that carry out search and rescue activities, especially close-range target tracking in defense systems.

An algorithm has been developed for numerically solving the source determination problem in the model of information dissemination in synthetic online social networks, described by reaction–diffusion-type equations, using additional information about the process at fixed time points. The degree of ill-posedness of the source determination problem for a parabolic equation is studied based on the analysis of singular values of the linearized operator of the inverse problem. The algorithm developed is based on a combination of the tensor optimization method and gradient descent supplemented with the A.N. Tikhonov regularization. Numerical calculations demonstrate the smallest relative error in the reconstructed source obtained by the developed algorithm in comparison with classical approaches. [ABSTRACT FROM AUTHOR]

The approximating sequence Riccati equation method is an efficient approach for solving the nonlinear optimal control problems, but its neglect of nonlinear dynamics and necessary optimality condition makes the control law difficult to satisfy the optimality. In this paper, an approximating pseudospectral method with state‐dependent coefficient optimization algorithm is proposed to solve this defect. By introducing the approximating pseudospectral method, the original nonlinear problem is transformed into a sequence of linear subproblems, which preserves the nonlinearity of solution. Then a state‐dependent coefficient optimization algorithm based on the gradient projection technique is proposed, which ensures the optimality of the control law. A double‐layer optimization structure is designed to facilitate the coordination between the approximating method and the optimization algorithm. Theoretical analysis proves the convergence of the proposed method. Comparative case studies illustrate the effectiveness in reducing the performance index and ensuring the optimality of the control law. [ABSTRACT FROM AUTHOR]

The quadratic termination property is important to the efficiency of gradient methods. We consider equipping a family of gradient methods, where the stepsize is given by the ratio of two norms, with two dimensional quadratic termination. Such a desired property is achieved by cooperating with a new stepsize which is derived by maximizing the stepsize of the considered family in the next iteration. It is proved that each method in the family will asymptotically alternate in a two dimensional subspace spanned by the eigenvectors corresponding to the largest and smallest eigenvalues. Based on this asymptotic behavior, we show that the new stepsize converges to the reciprocal of the largest eigenvalue of the Hessian. Furthermore, by adaptively taking the long Barzilai–Borwein stepsize and reusing the new stepsize with retard, we propose an efficient gradient method for unconstrained quadratic optimization. We prove that the new method is R-linearly convergent with a rate of 1 - 1 / κ , where κ is the condition number of Hessian. Numerical experiments show the efficiency of our proposed method. [ABSTRACT FROM AUTHOR]

As the performance of computing devices such as graphics processing units (GPUs) has improved dramatically, many deep neural network models, especially convolutional neural networks (CNNs), have been widely applied to various applications such as image classification, semantic segmentation, and object recognition. However, effective first-order optimization methods for CNNs have rarely been studied, although many CNN models have been successfully developed. Accordingly, this paper investigates various advanced adaptive solution search methods and proposes a new first-order optimization algorithm for CNNs called Adam-ASC. Our approach uses four sophisticated adaptive solution search methods to adjust its search strength in the complicated large-dimensional weight solution space spanned by a loss function. At the same time, we explain how they can be combined compensatively to form a complete optimizer with a detailed implementation. From the experiments, we found that our Adam-ASC can significantly improve the image recognition performance of practical CNNs in both the image classification and segmentation tasks. These experimental results show that the four fundamental methods of Adam-ASC and their compensative combination strategy play a crucial role in training CNNs by effectively finding their optimal weights.

For an optimal control problem with constraints on phase coordinates, an iterative method for finding a numerical solution is considered, based on reduction to a finite-dimensional problem and applying to it a sequential linearization algorithm using a modified Lagrange function. To solve linear auxiliary problems at iterations of the method, the reduced gradient method is used. The efficiency of taking into account phase constraints in the calculation of optimal control is illustrated by the numerical solution of problems from the field of aerodynamics and robotics. [ABSTRACT FROM AUTHOR]

We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This not only provides an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but it also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze and extend the adaptive Barzilai–Borwein method to a new family of stepsizes. While this family exploits negative values for the target, we also consider positive targets. We present a convergence analysis for quadratic problems extending results by Dai and Liao (IMA J Numer Anal 22(1):1–10, 2002), and carry out experiments outlining the potential of the approaches. [ABSTRACT FROM AUTHOR]

As the penetration of distributed photovoltaic (PV) in the distribution network continues to increase, the problem of voltage violation is becoming increasingly prominent. PV output has strong volatility and uncertainty, leading to higher requirements on speed and accuracy of voltage control. The voltage qualification rate of the distribution network with PV is difficult to guarantee if relying on traditional measuring installation and discrete regulators. Thus, it is necessary to introduce high‐frequency sampling devices and various continuous regulators in real‐time voltage control. In this paper, a fast voltage control strategy by real‐time coordination of dynamic voltage regulators (DVRs) and PV inverters based on phasor measurement units (PMU) and remote terminal unit (RTU) information is proposed. The voltage sensitivity is calculated online based on the fast measurement information of PMU. The equivalent models of rapid voltage regulators including DVR and PV inverter are established for the online extraction of voltage optimal control gradients. According to the model predictive control (MPC) theory, a feeder‐level global voltage control strategy is proposed comprehensively considering the objectives of voltage qualification rate, control cost, and network loss, which can realize real‐time voltage control in the distribution network at the millisecond scale. The effectiveness and rapidity of the proposed global voltage control strategy are verified in the IEEE 33‐bus system and a real representative distribution system for preventing voltage violation. [ABSTRACT FROM AUTHOR]

Majority of modern techniques for creating and optimizing the geometry of medical devices are based on a combination of computer-aided designs and the utility of the finite element method This approach, however, is limited by the number of geometries that can be investigated and by the time required for design optimization. To address this issue, we propose a generative design approach that combines machine learning (ML) methods and optimization algorithms. We evaluate eight different machine learning methods, including decision tree-based and boosting algorithms, neural networks, and ensembles. For optimal design, we investigate six state-of-the-art optimization algorithms, including Random Search, Tree-structured Parzen Estimator, CMA-ES-based algorithm, Nondominated Sorting Genetic Algorithm, Multiobjective Tree-structured Parzen Estimator, and Quasi-Monte Carlo Algorithm. In our study, we apply the proposed approach to study the generative design of a prosthetic heart valve (PHV). The design constraints of the prosthetic heart valve, including spatial requirements, materials, and manufacturing methods, are used as inputs, and the proposed approach produces a final design and a corresponding score to determine if the design is effective. Extensive testing leads to the conclusion that utilizing a combination of ensemble methods in conjunction with a Tree-structured Parzen Estimator or a Nondominated Sorting Genetic Algorithm is the most effective method in generating new designs with a relatively low error rate. Specifically, the Mean Absolute Percentage Error was found to be 11.8% and 10.2% for lumen and peak stress prediction respectively. Furthermore, it was observed that both optimization techniques result in design scores of approximately 95%. From both a scientific and applied perspective, this approach aims to select the most efficient geometry with given input parameters, which can then be prototyped and used for subsequent in vitro experiments. By proposing this approach, we believe it will replace or complement CAD-FEM-based modeling, thereby accelerating the design process and finding better designs within given constraints. The repository, which contains the essential components of the study, including curated source code, dataset, and trained models, is publicly available at https://github.com/ViacheslavDanilov/generative_design.

In this article a preconditioned version of the Delayed Weighted Gradient Method (DWGM) is presented and analyzed. In addition to the convergence, some nice properties as the A- orthogonality of the current transformed gradient with all the previous gradient vectors as well as finite convergence are demonstrated. Numerical experimentation is also offered, exposing the benefits of preconditioning.

Abstract A multi‐agent iterative optimisation method based on deep reinforcement learning is proposed for the balancing and sequencing problem in mixed model assembly lines. Based on the Markov decision process model for balancing and sequencing, a balancing agent using a deep deterministic policy gradient algorithm, a sequencing agent using an Actor–Critic algorithm, as well as an iterative interaction mechanism between these agents' output solutions are designed for realising the global optimisation of mixed model assembly lines. The exchange of solution information including assembly time and station workload in the iterative interaction realises the coordination of the worker assignment policy at the balancing stage and the production arrangement policy at the sequencing stage for the minimisation of work overload and idle time at stations. Through the comparative experiments with heuristic rules, genetic algorithms, and the original deep reinforcement learning algorithm, the effectiveness of the proposed method is demonstrated and discussed for small‐scale instances as well as large‐scale ones.

We consider distributed optimization over several devices, each sending incremental model updates to a central server. This setting is considered, for instance, in federated learning. Various schemes have been designed to compress the model updates in order to reduce the overall communication cost. However, existing methods suffer from a significant slowdown due to additional variance ω > 0 coming from the compression operator and as a result, only converge sublinearly. What is needed is a variance reduction technique for taming the variance introduced by compression. We propose the first methods that achieve linear convergence for arbitrary compression operators. For strongly convex functions with condition number κ, distributed among n machines with a finite-sum structure, each worker having less than m components, we also (i) give analysis for the weakly convex and the non-convex cases and (ii) verify in experiments that our novel variance reduced schemes are more efficient than the baselines. Moreover, we show theoretically that as the number of devices increases, higher compression levels are possible without this affecting the overall number of communications in comparison with methods that do not perform any compression. This leads to a significant reduction in communication cost. Our general analysis allows to pick the most suitable compression for each problem, finding the right balance between additional variance and communication savings. Finally, we also (iii) give analysis for arbitrary quantized updates. [ABSTRACT FROM AUTHOR]

The performance of deep neural networks (DNNs) critically relies on high-quality annotations, while training DNNs with noisy labels remains challenging owing to their incredible capacity to memorize the entire training set. In this work, we use two synchronously trained networks to reveal that noisy labels may result in more divergent gradients when updating the parameters. To overcome this, we propose a novel co-training framework named gradient agreement learning (GAL). By dynamically evaluating the gradient agreement coefficient of every pair of parameters from two identical DNNs to determine whether to update them in the training process. GAL can effectively hinder the memorization of noisy labels. Furthermore, we utilize the pseudo labels produced by the two DNNs as the supervision for the training of another network, thereby gaining further improvement by correcting some noisy labels while overcoming the confirmation bias. Extensive experiments on various benchmark datasets demonstrate the superiority of the proposed GAL. [ABSTRACT FROM AUTHOR]

We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most n components, where the marginal constraints of standard Optimal Transport (OT) are relaxed via Kullback-Leibler divergence with regularization factor τ. Although only Sinkhorn-based UOT solvers have been analyzed in the literature with the iteration complexity of O(τ log(n) / ε lof (log(n)/ε)) and per-iteration cost of O(n²) for achieving the desired error ε, their positively dense output transportation plans strongly hinder the practicality. On the other hand, while being vastly used as heuristics for computing UOT in modern deep learning applications and having shown success in sparse OT problem, gradient methods applied to UOT have not been formally studied. In this paper, we propose a novel algorithm based on Gradient Extrapolation Method (GEM-UOT) to find an ε-approximate solution to the UOT problem in O(κ log (τ n/ε)) iterations with O(n²) per-iteration cost, where κ is the condition number depending on only the two input measures. Our proof technique is based on a novel dual formulation of the squared ℓ2-norm UOT objective, which fills the lack of sparse UOT literature and also leads to a new characterization of approximation error between UOT and OT. To this end, we further present a novel approach of OT retrieval from UOT, which is based on GEM-UOT with fine tuned τ and a post-process projection step. Extensive experiments on synthetic and real datasets validate our theories and demonstrate the favorable performance of our methods in practice. We showcase GEM-UOT on the task of color transfer in terms of both the quality of the transfer image and the sparsity of the transportation plan. [ABSTRACT FROM AUTHOR]

Abstract Partial discharge (PD) signals are an important index to evaluate the operation state of intelligent substations. The correct distinction of PD pulse and interference pulse has become a challenging task. Because of the noise and the low signal‐to‐noise ratio, the stable signals become non‐stationary. The selection of a wavelet basis, the selection rule of threshold λ and the design of the threshold function are the key factors affecting the final denoising effect. Therefore, an enhanced ant colony optimisition wavelet (ACOW) algorithm was applied to find the global optimal threshold through the continuous derivative threshold function and the ant colony optimisation (ACO) algorithm. At the same time the efficiency of adaptive search calculation, was also significantly improved. The method of the ACOW algorithm was compared with the soft wavelet method, gradient‐based wavelet method and the genetic optimisation wavelet (GOW) method. Using these four methods to denoise four typical signals, different mean square errors (MSE), magnitude errors (ME) and time costs were obtained. Interestingly, the results show that the ACOW method can achieve the minimum MSE and has less time cost. It generates significantly smaller waveform distortion than the other three threshold estimation methods. In addition, the high efficiency and good quality of the output signals are beneficial to the diagnosis of local discharge signals in intelligent substations.

This paper considers the problem of controlling a dynamical system when the state cannot be directly measured and the control performance metrics are unknown or only partially known. In particular, we focus on the design of data-driven controllers to regulate a dynamical system to the solution of a constrained convex optimization problem where: i) the state must be estimated from nonlinear and possibly high-dimensional data; and ii) the cost of the optimization problem – which models control objectives associated with inputs and states of the system – is not available and must be learned from data. We propose a data-driven feedback controller that is based on adaptations of a projected gradient-flow method; the controller includes neural networks as integral components for the estimation of the unknown functions. Leveraging stability theory for perturbed systems, we derive sufficient conditions to guarantee exponential input-to-state stability (ISS) of the control loop. In particular, we show that the interconnected system is ISS with respect to the approximation errors of the neural network and unknown disturbances affecting the system. The transient bounds combine the universal approximation property of deep neural networks with the ISS characterization. Illustrative numerical results are presented in the context of robotics and control of epidemics.

Model predictive control (MPC) is a competitive option in modern control systems due to its ability to account for future response and incorporation of complex control objectives. As applications become more intricate, nonlinearities limit the utility of linear control strategies, thus requiring more sophisticated architectures, often at a significant computational cost. This article investigates the computational cost of solving nonlinear MPC problems and provides a framework for designing nonlinear MPC architectures compatible with real-time performance. To motivate the computational complexity associated with nonlinear MPC, the design of an automotive collision imminent steering system and the controller is considered. Various trajectory optimization strategies are examined and compared for this application, identifying multiple-shooting-based Runge–Kutta explicit integration as the most suitable. The control algorithm is then mapped into a graphics processor unit-based hardware system, where special considerations of the parallel hardware architecture are discussed. Compared to the single-shooting solution as the benchmark, multiple shootings on parallel hardware achieve three orders of magnitude improvement in wall time, supporting real-time implementation. [ABSTRACT FROM AUTHOR]

In this paper, we consider the large-scale precoding matrix indicator (PMI) selection problem at the receiver in wireless communications. The selection is based on the channel capacity of the PMI matrix in a pre-designed codebook. The quality of the PMI matrix is essential in achieving higher spectral efficiency. We first derive two novel formulations including a partial permutation-matrix model and an indicator-vector model for the original problem. The discrete constraints in the formulations make the problem NP-hard. Then we propose a stochastic projected gradient method augmented by block coordinate descent under various strategies. We show that the algorithms terminate in finite steps and produce sufficient descent at each iteration when the step size is chosen properly. Extensive experiments demonstrate that our proposed algorithms are able to find better PMI matrices more efficiently compared to the existing methods. [ABSTRACT FROM AUTHOR]

Abstract Images can be blurred due to the imperfection of the imaging system and blurriness is one of the challenging problems for image quality assessment (IQA). No‐reference blurred IQA methods have been proposed in the literature to calculate image blurriness. Inspired by image processing‐based auto‐focussing and maximum local information theories, a no‐reference blurred colourful IQA method based on Dual Maximum Local Information is proposed here. First, a window extraction method that combines the maximum gradient with local entropy is proposed to obtain the region of interest (ROI) for subsequent processing. Second, an improved maximum gradient method that leverages information from different channel images is presented to calculate the maximum gradient variation within the ROI for final sharpness score. Experimental results illustrated that the proposed method has better performance under various measurements compared with the state‐of‐the‐art methods on LIVE, CSIQ, TID2008, TID2013, VCL@FER, IVC image databases.

Abstract Deep reinforcement learning has achieved some remarkable results in self‐driving. There is quite a lot of work to do in the area of autonomous driving with high real‐time requirements because of the inefficiency of reinforcement learning in exploring large continuous motion spaces. A deep imitation reinforcement learning (DIRL) framework is presented to learn control policies of self‐driving vehicles, which is based on a deep deterministic policy gradient algorithm (DDPG) by vision. The DIRL framework comprises two components, the perception module and the control module, using imitation learning (IL) and DDPG, respectively. The perception module employs the IL network as an encoder which processes an image into a low‐dimensional feature vector. This vector is then delivered to the control module which outputs control commands. Meanwhile, the actor network of the DDPG is initialized with the trained IL network to improve exploration efficiency. In addition, a reward function for reinforcement learning is defined to improve the stability of self‐driving vehicles, especially on curves. DIRL is verified by the open racing car simulator (TORCS), and the results show that the correct control strategy is learned successfully and has less training time.

Abstract With the growth of mobile data traffic in wireless networks, caches are used to bring data closer to mobile users and to minimise the traffic load on macro base station (MBS). Storing data in caches on user terminals (UTs) and small base stations (SBSs) faces challenges with respect to the decision of cache contents. Here, a multi‐objective cache content strategy that aims to maximise the cache hit rate of SBSs in mobile edge networks (MENs) is proposed. The multi‐objective cache placement optimisation is formulated as a classification problem. Unlike previous work, mobility input attributes such as user locations, contact duration, communication ranges, contact probability between UTs and SBSs, etc. as well as content popularity and the correlation between these input attributes separating the decision space into two regions of cache and not cache are used. Stochastic gradient descent algorithm is used for the training of three supervised machine learning techniques: artificial neural network ANN, support vector machine (SVM), and logistic regression LR to define the hyperplane that separates the cache content decision space. Simulation results show that compared with the weighted‐sum approach, the SBSs cache hit rates increase on the average by 18.58%, 18.52%, and 18.2%, and the total energy consumption values decrease on the average by 33.49%, 53.19%, and 49.9% for ANN, SVM, and LR, respectively.

Abstract A novel approach to de‐embed the frequency response of filters connected to a 2‐channel multiplexer without the need for unplugging is proposed. The technique to manifold coupled multiplexers with arbitrary number of filters is also theoretically generalised. In multiplexer tuning, there are practically important circumstances in which it is advantageous to de‐embed the frequency response of filters in multiplexers without detaching them. The authors show that using a novel technique and reasonable approximations, all filters of a multiplexer can be modelled by simple microwave networks, and then, the equivalent rational function of the channel filters can be estimated appropriately by the rational H2 approximation; extraction of this model facilitates tuning of the filters using the coupling matrix method. The impact of mechanical tolerances on the accuracy of the proposed technique is also studied. Next, this method is examined on a 2‐channel multiplexer and its application in de‐embedding and tuning of its connected filters is verified. Provided that the mechanical tolerances of the structure are sufficiently small, this technique is a powerful engineering solution towards tuning of manifold‐coupled multiplexers.

The primary aim of this paper is to provide a review of the last few decades of research focused on delayed gradient methods for solving symmetric positive definite linear systems. Recent developments on spectral properties are stressed, as well as their implications for the formulation of new methods. Numerical experiments are conducted with emphasis on the comparison with the conjugate gradient method for solving ill-conditioned problems. [ABSTRACT FROM AUTHOR]

Consider the following distributed optimization scenario. A worker has access to training data that it uses to compute the gradients while a server decides when to stop iterative computation based on its target accuracy or delay constraints. The server receives all its information about the problem instance from the worker via a rate-limited noiseless communication channel. We introduce the principle we call differential quantization (DQ) that prescribes compensating the past quantization errors to direct the descent trajectory of a quantized algorithm towards that of its unquantized counterpart. Assuming that the objective function is smooth and strongly convex, we prove that differentially quantized gradient descent (DQ-GD) attains a linear contraction factor of $\max \{\sigma _{\mathrm {GD}}, \rho _{n} 2^{-R}\}$ , where $\sigma _{\mathrm {GD}}$ is the contraction factor of unquantized gradient descent (GD), $\rho _{n} \geq 1$ is the covering efficiency of the quantizer, and $R$ is the bitrate per problem dimension $n$. Thus at any $R\geq \log _{2} \rho _{n} /\sigma _{\mathrm {GD}}$ bits, the contraction factor of DQ-GD is the same as that of unquantized GD, i.e., there is no loss due to quantization. We show a converse demonstrating that no algorithm within a certain class can converge faster than $\max \{\sigma _{\mathrm {GD}}, 2^{-R}\}$. Since quantizers exist with $\rho _{n} \to 1$ as $n \to \infty $ (Rogers, 1963), this means that DQ-GD is asymptotically optimal. In contrast, naively quantized GD where the worker directly quantizes the gradient barely attains $\sigma _{\mathrm {GD}} + \rho _{n}2^{-R}$. The principle of differential quantization continues to apply to gradient methods with momentum such as Nesterov’s accelerated gradient descent, and Polyak’s heavy ball method. For these algorithms as well, if the rate is above a certain threshold, there is no loss in contraction factor obtained by the differentially quantized algorithm compared to its unquantized counterpart, and furthermore, the differentially quantized heavy ball method attains the optimal contraction achievable among all (even unquantized) gradient methods. Experimental results on least-squares problems validate our theoretical analysis. [ABSTRACT FROM AUTHOR]

Traffic signal control is a critical method that ensures the efficiency of traffic flow in cities across the world. There are massive studies that focus on generating optimal signal timing plans. Most of the these studies are model-based, where the signal plan is determined by optimization models with fixed parameters. Reinforcement learning (RL) is a model-free method that learns the optimal control policy over time which avoids limitations of traditional methods. In the past a few years, deep RL methods become particularly attractive since deep neural networks can provide scalable learning capability given the complex traffic condition in real life. In this manuscript, we develop a deep actor-critic method that can provide efficient traffic signal plans. First, a novel deep neural network model is applied to mine the recent traffic condition information as a sequence of temporally sequential image representations of the intersection, which improves from relying solely on limited traffic information such as queue length; then the deep neural network model is integrated with an actor-critic model which avoids drawbacks from the value-based or the policy-based methods. Simulation experiments illustrate that the deep actor-critic method outperforms the classic model-based method and several existing deep reinforcement learning methods according to different measures such as queue length, average delay time, and throughput. [ABSTRACT FROM AUTHOR]

This paper focuses on a time-varying constrained nonconvex optimization problem, and considers the synthesis and analysis of online regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient method is implemented in a running fashion, in the sense that the underlying optimization problem changes during the execution of the algorithms. In order to study its performance, we first derive its continuous-time limit as a system of differential inclusions. We then study sufficient conditions for tracking a KKT trajectory, and also derive asymptotic bounds for the tracking error (as a function of the time-variability of a KKT trajectory). Further, we provide a set of sufficient conditions for the KKT trajectories not to bifurcate or merge, and also investigate the optimal choice of the parameters of the algorithm. Illustrative numerical results for a time-varying nonconvex problem are provided. [ABSTRACT FROM AUTHOR]

In this paper, we study a fully composite formulation of convex optimization problems, which includes, as a particular case, the problems with functional constraints, max-type minimization problems, and problems with simple nondifferentiable components. We treat all these formulations in a unified way, highlighting the existence of very natural optimization schemes of different order p ≥ 1. As the result, we obtain new high-order (p ≥ 2) optimization methods for composite formulation. We prove the global convergence rates for them under the most general conditions. Assuming that the upper-level component of our objective function is subhomogeneous, we develop efficient modification of the basic fully composite first-order and second-order methods and propose their accelerated variants. [ABSTRACT FROM AUTHOR]

An interactive computer program for multiple nonlinear curves fitting has been developed in this work. Several optimization algorithms have been implemented in this software for solving constrained and unconstrained nonlinear optimization models in order to evaluate and best-estimate the concerning and desired suggested mathematical model parameters. Two categories of algorithms have been used in this work. The first category is random and stochastic mathematical methods (nondeterministic methods) such as the simulated annealing and the adaptive simulated annealing. The second category is the direct search methods (deterministic methods) such as Hook-Jeeves pattern search, Fletcher and Reeves conjugate gradient and steepest descent method.

Abstract Recently, the gradient‐based neural architecture search has made remarkable progress with the characteristics of high efficiency and fast convergence. However, two common problems in the gradient‐based NAS algorithms are found. First, with the increase in the raining time, the NAS algorithm tends to skip‐connect operation, leading to performance degradation and instability results. Second, another problem is no reasonable allocation of computing resources on valuable candidate network models. The above two points lead to the difficulty in searching the optimal sub‐network and poor stability. To address them, the trick of pre‐training the super‐net is applied, so that each operation has an equal opportunity to develop its strength, which provides a fair competition condition for the convergence of the architecture parameters. In addition, a skip‐controller is proposed to ensure each sampled sub‐network with an appropriate number of skip‐connects. The experiments were performed on three mainstream datasets CIFAR‐10, CIFAR‐100 and ImageNet, in which the improved method achieves comparable results with higher accuracy and stronger robustness.

Abstract A weighted stochastic gradient algorithm is proposed for cost‐efficient tracking of unknown, correlated spatial signals from randomly distributed sensor observations in localized wireless sensor field. The algorithm is implemented in spatial modelling and spatial tracking phases. In spatial modelling phase, the algorithm finds the model parameters, and in spatial tracking phase, it updates these parameters. The spatial signal is modelled with its M iso‐contour lines at equally spaced levels {ℓ}k=1M and the sensors with sensor observations in Δ margin of these contour levels report to the fusion centre (FC) for spatial monitoring purpose. Based on progressive learning and in successive iterations, the algorithm improves its findings of the signal strength's range, and the spatial, temporal and spectral attributes of the signal. To reduce the cost, in each iteration, only a subset of wireless sensors transmits the observations to the FC, in response to its query. In this article, the percentage of the reporting sensors to the FC is defined as the algorithm's cost. With importance sampling perspective, the sample space is reduced to those sensors whose observations are within a Δ margin of atleast one of these M contour levels. The Δ margin is pruned or enhanced using the proposed weighted stochastic gradient algorithm, dynamically in order to reduce the spatial tracking cost. The evaluation results show that after spatial modelling, spatial tracking is drastically of low cost and its performance is better than that of the conventional stochastic gradient method. The modelling error, the cost and the convergence of the proposed algorithm are investigated extensively, in this article. Spatial correlation in signal distribution and the coordinates of the wireless sensors are the only initial assumptions in spatial monitoring of the unknown signal distribution. The main purpose of this algorithm is low‐cost identification of unknown correlated spatial signals from sensor observations, over time. An example for application of the proposed algorithm is environmental monitoring using wireless sensor observations.

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

In this paper, we consider gradient methods for minimizing smooth convex functions, which employ the information obtained at the previous iterations in order to accelerate the convergence towards the optimal solution. This information is used in the form of a piece-wise linear model of the objective function, which provides us with much better prediction abilities as compared with the standard linear model. To the best of our knowledge, this approach was never really applied in Convex Minimization to differentiable functions in view of the high complexity of the corresponding auxiliary problems. However, we show that all necessary computations can be done very efficiently. Consequently, we get new optimization methods, which are better than the usual Gradient Methods both in the number of oracle calls and in the computational time. Our theoretical conclusions are confirmed by preliminary computational experiments. [ABSTRACT FROM AUTHOR]