Search

This paper investigates the policy optimization paradigm, where a learning model is trained to find the solution of complex Markov decision problems, as a tool to address the berth allocation problem in multimodal terminals. To this purpose, we drop the typical formulation of the latter as a mixed-integer static scheduling one, and we model it instead as an evolving scenario in which berths are assigned to ships according to a parameterized policy function that drives the temporal evolution of the environment. We adopt a cross-entropy optimization scheme to optimize the policy parameters, which is a simple and highly parallelizable gradient-free technique. As compared to the static mixed-integer formulation, the proposed approach relies on a much lighter optimization problem in the continuous space of the policy parameters, thus making it feasible to replan in real time when needed. Furthermore, the generality of the policy optimization approach allows to take into account any performance metric and specific feature of the scenario straightforwardly, without the need to devise ad hoc heuristics. Simulation tests showcase the good performance of the policy approach under various conditions.

Modal decomposition is popular in fluid mechanics, especially for data-driven analysis. In particular, dynamic mode decomposition allows to identify the modes that describe complex phenomenona, such as those physically modelled by the Navier-Stokes equation. The identified modes are associated with residuals, which can be used to detect a meaningful change of regime, e.g., the formation of a vortex. Toward this end, moving horizon estimation is applied to identify the trend of the norm of the residuals that result from the application of dynamic mode decomposition to automatically classify the time evolution of fluid flows. The trend dynamics is modelled as a switching nonlinear system and hence a problem of moving horizon estimation is solved to monitor the behavior of the fluid and quickly identify changes of regime. The stability of the resulting estimation error given is proved. The combination of dynamic mode decomposition and moving horizon estimation provides successful results as shown by processing experimental datasets of the velocity field of fluid flows obtained by a particle image velocimetry.

The normal flow equation is a nonlinear partial differential equation that is quite popular in numerous research fields related to the so-called level set methods. Specifically, we have investigated the feedback control of such an equation by proposing two different regulators. The first approach consists in considering the velocity field of the equation as a control action; in such a case a simple proportional regulator is proved to be stable. In the second case, the control acts on the source term, and it relies on a Luenberger observer that provides an estimate of the norm of the gradient involved in the normal flow equation. Also this controller is proved to be stable by using Lyapunov arguments. Simulation results are presented to show the effectiveness of the proposed approaches.

Fast moving horizon state estimation for constrained discrete-time systems is investigated through an approach based on an incomplete optimization at each time step. In particular, only few iterations of the gradient, conjugate gradient, and Newton descent methods are executed. Constraints on the state variables are explicitly considered by exploiting the "dead time" between two consecutive time steps, where a projection requiring the minimization of a distance function is performed. Instead, a simpler line search is computed on line that consists in searching for a state estimate satisfying the constraints by solving a one-dimensional optimization. Stability conditions for the estimation error are derived. The proposed approach is well-suited to being applied for state estimation of rapidly-evolving constrained systems, as confirmed by simulations in comparison with the application of the online "full" projection at each time step.

We address the problem of estimating the constant parameters involved in the normal flow equation, which is a Hamilton-Jacobi PDE widely used in many different research areas. The identification of such parameters allows one to estimate the flow function, which is the velocity vector field that governs the dynamics of the level sets associated with the solution of the equation. The estimates are obtained by using a Luenberger observer and a parameter estimator based on the adaptation law proposed by Pomet and Praly in 1992. Such a law makes it possible to explicitly take into account bounds on the parameters. Conditions for the stability of the parameter estimation error are established. Simulation results are presented that confirm the theoretical achievements.

Robotic networks are increasingly used to under- take complex missions and are often composed of heterogeneous agents executing well-defined tasks. To react against external disturbances and hardware failures, a typical technique is to change at runtime how tasks are assigned to robots. In this perspective, the paper presents an approach based on predictive control for the on-line selection of the optimal time instants when perform reassignments. To evaluate the effectiveness of the proposed approach, simulation results are showcased in comparison with reactive and proactive strategies.

The computational issues involved in the solution of optimal control problems of propagating fronts described by level sets motivate the search for effective optimization algorithms. In this paper, we attack the problem of optimal control of moving fronts by searching for an approximate solution method that is computationally feasible and robust to local minima trapping. The presence of many local minima is a crucial difficulty one encounters in dealing with such a problem. Following previous results, we use the extended Ritz method to find approximate solutions. This approach consists in adopting a control law with fixed structure that depends nonlinearly on a number of parameters to be suitably chosen. To overcome the local minima issue, we propose to optimize the weights for the level set optimal control by a recursive minimization based on the extended Kalman filter (EKF). As compared with techniques based on gradient-descent methods, the EKF optimization turns out to be successful to reduce computational burden and increase robustness with respect to local minima trapping, as shown by simulation results in a test case involving a change of topology.

This work addresses the problem of learning an unknown function from data when local models are employed. In particular, kernel smoothing models are considered, which use kernels in a straightforward fashion by modeling the output as a weighted average of values observed in a neighborhood of the input. Such models are a popular alternative to other kernel paradigms, such as support vector machines (SVM), due to their very light computational burden. The purpose of this work is to prove that a smart deterministic selection of the observation points can be advantageous with respect to input data coming from a pure random sampling. Apart from the theoretical interest, this has a practical implication in all the cases in which one can control the generation of the input samples (e.g., in applications from robotics, dynamic programming, optimization, mechanics, etc.) To this purpose, lattice point sets (LPSs), a special kind of sampling schemes commonly employed for efficient numerical integration, are investigated. It is proved that building local kernel smoothers using LPSs guarantees universal approximation property with better rates with respect to i.i.d. sampling. Then, a rule for automatic kernel width selection, making the computational burden of building the model negligible, is introduced to show how the regular structure of the lattice can lead to practical advantages. Simulation results are also provided to test in practice the performance of the proposed methods.

The widespread diffusion of Infrastructure-as-a- Service and cloud computing paradigms requires large-scale data centers with thousands of running nodes and high energy demands, thus causing relevant economical and environmental costs. In this perspective, the paper presents an energy-aware consolidation strategy based on predictive control, in which virtual machines are migrated among nodes to reduce the number of active units. To describe a general cloud infras- tructure, a discrete-time dynamic model is presented together with constraints. The migration strategies of virtual machines are obtained by solving finite-horizon optimal control problems involving integer variables. To reduce the computational effort, approximate solutions are searched for via Monte Carlo opti- mization. Besides power savings, the proposed method allows one to reduce violations of the service level agreement and aggressive on/off cycles of nodes. To showcase the effectiveness of the proposed approach, preliminary simulation results are provided.

When the problem of learning from data is solved through a regression tree estimator, the quality of the available observations is an important issue, since it influences directly the accuracy of the resulting model. It becomes particuarly relevant when there is freedom to sample the input space arbitrarily to build the tree model or, alternatively, when we need to select a subsample to train the tree estimator on a computationally feasible input set, or to evaluate the goodness of the estimation on a test set. Here the accuracy of estimation based on regression trees is analyzed from the point of view of geometric properties of the available input data. In particular, the concept of F-discrepancy, a quantity that measures how well a set of points represents the distribution underlying the input generation process, is applied to derive conditions for convergence to the optimal piecewise-constant estimator for the unknown function we want to learn. The analysis has a constructive nature, allowing to select in practice good input sets for the problem at hand, as shown in a simulation example involving a real data set.

The problem of optimal tracking of level sets is investigated. More specifically, we propose a novel method to design a controller that is able to track a reference curve generated by a level set equation. Such a functional optimal control problem is nonstandard as it involves a moving domain over time, thus requiring the development of both an ad-hoc approximation scheme and an efficient numerical solver. We address the approximate solution of the resulting optimization problem by using the extended Ritz method, which has been recently applied to the solution of a number of optimal control problems in high dimensional setting. Such a method relies on the use a control law of fixed structure that depends on a number of parameters to be suitably chosen. Toward this end, we derive the adjoint equation for the optimal tracking problem. Such an equation allows one to compute the gradient of the cost function with respect to the vector of parameters of the control law. Numerical results are reported to show the effectiveness and computational effort of the proposed approach for the purpose of tracking curves generated by the normal and mean curvature flow equations.

This paper analyzes quasi-random sampling techniques for approximate dynamic programming. Specifically, low-discrepancy sequences and lattice point sets are investigated and compared as efficient schemes for uniform sampling of the state space in high-dimensional settings. The convergence analysis of the approximate solution is provided basing on geometric properties of the two discretization methods. It is also shown that such schemes are able to take advantage of regularities of the value functions, possibly through suitable transformations of the state vector. Simulation results concerning optimal management of a water reservoirs system and inventory control are presented to show the effectiveness of the considered techniques with respect to pure-random sampling.

The design of closed-loop finite-dimensional con- trollers for systems described by partial differential equations is tackled by combining tools borrowed by research areas such as approximation and operator theory. The proposed paradigm is based on the idea of using operators to account for the dynamics, regulator, and measurement mappings. Specifically, we rely on a well-established setting of Banach spaces, which is well-suited to supporting the generality of the approach. First, we define a class of Lipschitz operators with finite seminorm and formulate a tracking problem in the Banach spaces of real- valued functions. Second, we search for controllers that ensure stability and minimize a given performance index. The design of such regulators is achieved by resorting to an approximation scheme based on the extended Ritz method. Such a scheme consists in constraining the regulation operator to take on a fixed structure, where a finite number of free parameters can be suitably chosen. The problem is then reduced to a mathematical programming one of nonlinear type in general, in which the values of the parameters are optimized to guarantee stability. A family of nonlinear approximators to which the most common classes of feedforward neural networks belong are employed to accomplish the design via a convenient choice of their parameters (i.e., the weights), as shown by means of simulations with the optimal control of an unstable heat equation.

Connections between function approximation and classes of functional optimization problems, whose admissible solutions may depend on a large number of variables, are investigated. The insights obtained in this context are exploited to analyze families of nonlinear approximation schemes contain- ing tunable parameters and enjoying the following property: when they are used to approximate the (unknown) solutions to optimization problems, the number of parameters required to guarantee a desired accuracy grows at most polynomially with respect to the number of variables in admissible solutions. Both sigmoidal neural networks and networks with kernel units are considered as approximation structures to which the analysis applies. Finally, it is shown how the approach can be applied for the solution of finite-horizon optimal control problems via approximate dynamic programming enhancing the potentiali- ties of recent developments in nonlinear approximation in the framework of the solution of sequential decision problems with continuous state spaces.

The use of integer tree-based search and mixed- integer programming is investigated for the purpose of control of multi-item multi-echelon distribution chains. A discrete-time model is considered to describe the dynamics of a generic distribution chain. The decisions on the amounts of goods to transfer are made by referring to a performance index that accounts for transportation, storage, and backlog costs at two levels, i.e., strategic and tactical. As to the strategic level, a worst-case stock replenishment policy is adopted to exploit the uncertain information available on long-term predictions of the customers' demand. The solution of such a problem is obtained by using a top-down tree-based algorithm to select policy parameters such as the delivery cycle times of goods. At the tactical level, the on-line decisions on the transportation of goods are taken basing on model predictive control, which allows one to take into account recent, reliable, short-term predictions of the demand. The tactical optimal decisions are obtained by solving mixed-integer programming problems with fewer variables as compared with the strategic setting. Simulation results are presented to show the effectiveness of the proposed approach.

The goal of this paper is that of reporting the results obtained in the identification of a macroscopic dynamic model that can describe the freeway traffic using the informa- tion available from a wireless cellular network. To this end, we need to assume that the distribution of mobile terminals aboard cars be uniform along the freeway and to deal with the cells of the cellular network as the sections in which a freeway stretch is divided. The information on the mobile terminals concerns their positions and speed. Two different approaches are investigated. The former is an extension to quite a standard freeway macroscopic model, while the latter is a black-box approach that consists in using a neural network to represent the traffic dynamics. The parameters of the former and the neural weights of the latter are identified off line by a least-squares approach. Numerical results obtained after identification and validation are reported using the data generated by a microscopic simulator.