Search

In this paper, a novel methodology is introduced for the inverse optimal control of non-affine, nonlinear and discrete-time systems. Although inverse optimal control of affine systems is studied in detail in technical literature, there is no adequate research about its implementation on non-affine systems. here are two main contributions of this work. Firstly using the input–output data of the system to be controlled its NARMA-L2 model is obtained using a multi-layer feedforward neural network, this step provides a conversion from a non-affine to affine system model. After the affine system model is obtained, the inverse optimal control law is applied. The second contribution of this paper is the computation of the inverse optimal control signal. The selection of the P matrix in the control law is crucial since its value directly affects the control performance. Here a novel method is proposed where an adaptive and optimal P matrix is computed online using a recurrent neural network to minimize a predefined cost function. The performance of the proposed control method is evaluated by simulations performed on benchmark problems. The robustness of the method is also tested by additional simulations where noise and disturbance is imposed on the system. The obtained results justify the applicability of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper proposes an adaptive speed control strategy for DC motors using a DC/DC converter (in buck and boost modes) based on the inverse optimal control (IOC) approach. Our proposal leverages the robust theoretical framework of IOC to derive a control law that ensures the stability and optimal performance of nonlinear dynamical systems through the Lyapunov theory. The control law is designed to minimize a specified cost function, implicitly supporting the optimality of the control strategy. An integral action is incorporated into the IOC approach to enhance performance, ensuring asymptotic stability without affecting convergence properties. The control strategy was implemented on buck and boost converter/DC motor systems. In addition, a disturbance observer technique was utilized for real-time load torque estimation, ensuring precise and efficient control to tackle the challenge posed by unknown and time-varying load torques. Simulations in PLECS and experimental results demonstrate the superiority of the proposed IOC approach compared to conventional cascaded PI control. Our approach reports significantly faster response times and reduced settling times in both buck and boost converter configurations, showcasing its potential for efficient and robust DC motor speed control applications.

Inverse optimal control is a method for recovering the cost function used in an optimal control problem in expert demonstrations. Most studies on inverse optimal control have focused on building the unknown cost function through the linear combination of given features with unknown cost weights, which are generally considered to be constant. However, in many real-world applications, the cost weights may vary over time. In this study, we propose an adaptive online inverse optimal control approach based on a neural-network approximation to address the challenge of recovering time-varying cost weights. We conduct a well-posedness analysis of the problem and suggest a condition for the adaptive goal, under which the weights of the neural network generated to achieve this adaptive goal are unique to the corresponding inverse optimal control problem. Furthermore, we propose an updating law for the weights of the neural network to ensure the stability of the convergence of the solutions. Finally, simulation results for an example linear system are presented to demonstrate the effectiveness of the proposed strategy. The proposed method is applicable to a wide range of problems requiring real-time inverse optimal control calculations. [ABSTRACT FROM AUTHOR]

To effectively suppress the longitudinal vibration of the car under the conditions of high-speed operation and emergency braking, and improve the ride comfort of the car system, this paper proposes an adaptive fuzzy inverse optimal output feedback control strategy. Firstly, the dynamic model of the high-speed elevator system is established and the nonlinear dynamic model is approximated by the fuzzy logic system, and the auxiliary system model is established. Fuzzy state observer is designed to estimate the unmeasurable state. Furthermore, an adaptive inverse optimal output feedback controller based on fuzzy observer is designed by using adaptive backstepping technology and inverse optimal control principle. The stability analysis shows that the proposed adaptive fuzzy inverse optimal output feedback control strategy not only ensures the stability of the car attitude of high-speed elevator but also realizes the inverse optimization of the target cost function. Finally, the acceleration time–frequency response analysis of the two typical stages of high-speed elevator uniform running and emergency braking is carried out, and the numerical results are compared with the linear quadratic controller optimized by stepping quantum genetic algorithm (GA-LQR) and controller based on the state-dependent Ricatti equation (SDRE). The analysis verifies the effectiveness of the controller. [ABSTRACT FROM AUTHOR]

In this paper an active controller for ground vehicles stability is presented. The objective of this controller is to force the vehicle to track a desired reference, ensuring safe driving conditions in the case of adhesion loss during hazardous maneuvers. To this aim, a nonlinear discrete-time inverse optimal control based on a neural network identification is designed, using a recurrent high order neural network (RHONN) trained by an Extended Kalman Filter. The RHONN ensures stability of the identification error, while the controller ensures the stability of the tracking errors. Moreover, a discrete-time reduced order state observer is utilized to reconstruct the lateral vehicle dynamic not usually available. For the control problem, the references of the lateral velocity and yaw rate are given by a dynamic system mimicking an ideal vehicle having not-decreasing tire lateral characteristics. The proposed approach avoids the identification of the Pacejka's lateral parameters of the tires, so simplifying the input control determination. Moreover, an optimal control is proposed to optimize the actuator effort and power, usually bounded. Control gains are determined using optimal "nature-inspired" algorithms such as particle swarm optimization. Test maneuvers, performed through the full vehicle simulator CarSim®, have been used to test correctness, quality and performances of the observer, the neural identifier and the inverse optimal controller. Robustness of the reduced order discrete-time state observer is also discussed for different sample times. Finally, a fair comparison between optimal and non-optimal control schemes is presented, highlighting the numerical results obtained in simulation. [ABSTRACT FROM AUTHOR]

Robots deployed to the real world must be able to interact with other agents in their environment. Dynamic game theory provides a powerful mathematical framework for modeling scenarios in which agents have individual objectives and interactions evolve over time. However, a key limitation of such techniques is that they require a priori knowledge of all players' objectives. In this work, we address this issue by proposing a novel method for learning players' objectives in continuous dynamic games from noise-corrupted, partial state observations. Our approach learns objectives by coupling the estimation of unknown cost parameters of each player with inference of unobserved states and inputs through Nash equilibrium constraints. By coupling past state estimates with future state predictions, our approach is amenable to simultaneous online learning and prediction in receding horizon fashion. We demonstrate our method in several simulated traffic scenarios in which we recover players' preferences, for, e.g. desired travel speed and collision-avoidance behavior. Results show that our method reliably estimates game-theoretic models from noise-corrupted data that closely matches ground-truth objectives, consistently outperforming state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper addresses the adaptive inverse optimal output regulation problem for a class of uncertain nonlinear systems driven by an exosystem. The unknown parameters, internal disturbances, and unmeasured states are contained in the nonlinear system. Firstly, the output regulation problem is decomposed into a feedforward control design problem which can be solved by the internal model based on the output regulation theory, and an adaptive inverse optimal stabilization problem. Then an auxiliary system is designed, and a new state observer related to the auxiliary system is given. By combining adaptive control technology and inverse optimal control method, a novel adaptive output feedback inverse optimal controller is developed to make the output of the system track the reference signal fast. With this control strategy, all the signals of the closed-loop system are uniformly ultimately bounded (UUB), and the newly well-defined cost functional which is connected with the auxiliary system and the controller can be minimized. Finally, a simulation case is put forward to verify the feasibility of the newly raised controller and the state observer.

This study utilizes inverse optimal control (IOC) theory to address the issue of output voltage regulation in a boost converter feeding an unknown direct current (DC) load. The proposed approach involves developing a general feedback control law through IOC to ensure asymptotic stability in closed-loop operation, with the added advantage of incorporating an integral gain without compromising stability. Two estimators are introduced to minimize the number of sensors required for implementing the IOC controller with integral action. The first estimator, based on the immersion and invariance (I&I) method, determines the current demand of the DC load by measuring the boost converter’s output voltage. While the second estimator, using the disturbance observer (DO) method, estimates the voltage input value by measuring the inductor’s current flow. Both methods guarantee exponential convergence to the precise value of the estimated variable, irrespective of the initial estimation points. Experimental validation using varying DC loads and estimation techniques confirms the proposed IOC approach’s effectiveness and robustness in regulating voltage for DC loads connected to a boost converter. Furthermore, the proposed controller is compared to the sliding mode control and presents a better performance with a more straightforward design, and the stability in closed-loop ensured.

Inverse optimal control (IOC) is a problem of estimating a cost function based on the behaviors of an expert that behaves optimally with respect to the cost function. Although the Hamilton-Jacobi-Bellman (HJB) equation for the value function that evaluates the temporal integral of the cost function provides a necessary condition for the optimality of expert behaviors, the use of the HJB equation alone is insufficient for solving the IOC problem. In this study, we propose a triangle inequality which is useful for estimating the better representation of the value function, along with a new IOC method incorporating the triangle inequality. Through several IOC problems and imitation learning problems of time-dependent control behaviors, we show that our IOC method performs substantially better than an existing IOC method. Showing our IOC method is also applicable to an imitation of expert control of a 2-link manipulator, we demonstrate applicability of our method to real-world problems.

In this study, a novel inverse optimal controller based on NARMA-L2 modelling technique and online least squares support vector regression (LSSVR) method has been proposed for nonlinear non-affine systems. Firstly, the nonlinear autoregressive with exogenous inputs (NARX) model of the system is obtained using online LSSVR method, then this model is decomposed into NARMA-L2 submodels. Hence, the non-affine system model is converted to a nonlinear affine system model. The obtained NARMA-L2 submodels are used in computing the inverse optimal control law. Furthermore, the parameters of the inverse optimal controller have also been optimized online using the Levenberg-Marquadt algorithm. The performance of the proposed LSSVR based inverse optimal controller using NARMA-L2 model has been evaluated by simulations carried out on two benchmark systems, and the results show that the LSSVR based NARMA-L2 model and inverse optimal controller attain good modelling and control performances.

We present a method to solve a special class of parameter identification problems for an elliptic optimal control problem to global optimality. The bilevel problem is reformulated via the optimal-value function of the lower-level problem. The reformulated problem is nonconvex and standard regularity conditions like Robinson's CQ are violated. Via a relaxation of the constraints, the problem can be decomposed into a family of convex problems and this is the basis for a solution algorithm. The convergence properties are analyzed. It is shown that a penalty method can be employed to solve this family of problems while maintaining convergence speed. For an example problem, the use of the identity as penalty function allows for the solution by a semismooth Newton method. Numerical results are presented. Difficulties and limitations of our approach to solve a nonconvex problem to global optimality are discussed. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Harbin University of Science & Technology is the property of Journal of Harbin University of Science & Technology and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper presents the development of a neural inverse optimal control (NIOC) for a regenerative braking system installed in electric vehicles (EVs), which is composed of a main energy system (MES) including a storage system and an auxiliary energy system (AES). This last one is composed of a supercapacitor and a buck–boost converter. The AES aims to recover the energy generated during braking that the MES is incapable of saving and using later during the speed increase. To build up the NIOC, a neural identifier has been trained with an extended Kalman filter (EKF) to estimate the real dynamics of the buck–boost converter. The NIOC is implemented to regulate the voltage and current dynamics in the AES. For testing the drive system of the EV, a DC motor is considered where the speed is controlled using a PID controller to regulate the tracking source in the regenerative braking. Simulation results illustrate the efficiency of the proposed control scheme to track time-varying references of the AES voltage and current dynamics measured at the buck–boost converter and to guarantee the charging and discharging operation modes of the supercapacitor. In addition, it is demonstrated that the proposed control scheme enhances the EV storage system's efficacy and performance when the regenerative braking system is working. Furthermore, the mean squared error is calculated to prove and compare the proposed control scheme with the mean squared error for a PID controller. [ABSTRACT FROM AUTHOR]

This paper proposes an impulsive control scheme for a complex network that helps reduce the spread of two epidemic diseases: influenza type A and COVID-19. Both are respiratory infections; thus, they have a similar form of transmission, and it is possible to use the same control scheme in both study cases. The objective of this work is to use neural impulsive inverse optimal pinning control for complex networks to reduce the effects of propagation. The dynamic model is considered unknown, for which we design a neural identifier that, through training using the extended Kalman filter algorithm, provides the appropriate nonlinear model for this complex network. The dynamics of the network nodes are represented by the Susceptible-Infected-Removed (SIR) compartmental model in their discrete form. The results of the simulations are presented and addressed, applying the same control scheme but with different parameter values for each case study. [ABSTRACT FROM AUTHOR]

This article proposes robust inverse $Q$ -learning algorithms for a learner to mimic an expert’s states and control inputs in the imitation learning problem. These two agents have different adversarial disturbances. To do the imitation, the learner must reconstruct the unknown expert cost function. The learner only observes the expert’s control inputs and uses inverse $Q$ -learning algorithms to reconstruct the unknown expert cost function. The inverse $Q$ -learning algorithms are robust in that they are independent of the system model and allow for the different cost function parameters and disturbances between two agents. We first propose an offline inverse $Q$ -learning algorithm which consists of two iterative learning loops: 1) an inner $Q$ -learning iteration loop and 2) an outer iteration loop based on inverse optimal control. Then, based on this offline algorithm, we further develop an online inverse $Q$ -learning algorithm such that the learner mimics the expert behaviors online with the real-time observation of the expert control inputs. This online computational method has four functional approximators: a critic approximator, two actor approximators, and a state-reward neural network (NN). It simultaneously approximates the parameters of $Q$ -function and the learner state reward online. Convergence and stability proofs are rigorously studied to guarantee the algorithm performance. [ABSTRACT FROM AUTHOR]

In this paper, we present a Model-Free Stochastic Inverse Optimal Control (IOC) algorithm for the discrete-time infinite-horizon stochastic linear quadratic regulator (LQR). Our proposed algorithm exploits the richness of the available system trajectories to recover the control gain $K$ and cost function parameters $(Q,R)$ in a low (space, sample, and computational) complexity manner. By leveraging insights on the stochastic LQR, we guarantee well-posedness of the Model-Free Stochastic IOC LQR via satisfaction of the Certainty Equivalence optimality conditions. The exact solution of the control gain $K$ is recovered via a deterministic, low complexity Least Squares approach. Using $K$ , we solve a completely model-free non-iterative SemiDefinite Programming (SDP) problem to obtain a unique (up to a scalar ambiguity) $(Q,R)$ , in which optimality and feasibility are jointly ensured. Via derivation of the sample complexity bounds, we show that the non-asymptotic performance of the Model-Free Stochastic IOC LQR can be characterized by the signal-to-noise (SNR) ratio of the finite set of system state and input signals. We present a model-based version of the algorithm for the special case where $(A,B)$ is available, and we, further, provide the extension to the Stochastic Model-Free IOC linear quadratic tracking (LQT) case.

An efficient inverse optimal control method named Adaptive Reference IOC is introduced to study natural walking with musculoskeletal models. Adaptive Reference IOC utilizes efficient inner-loop direct collocation for optimal trajectory prediction along with a gradient-based weight update inspired by structured classification in the outer-loop to achieve about 7 times faster convergence than existing derivative-free methods while maintaining similar outcomes in terms of gait trajectory matching. The proposed method adequately reconstructed the reference data when applied to experimental walking data from ten participants walking at various speeds and stride lengths. The proposed framework can facilitate efficient personalized cost function optimization for specific walking tasks, and provide guidance to personalized reference trajectory design for assistive robotic systems such as lower-limb exoskeletons.

In this paper, the utilization of inverse-optimality-based prescribed-time explicit reference governor is investigated for missile intercepting against unknown maneuvering targets under performance and control input constraints. With an arctangent-based disturbance observer equipped for disturbance elimination, incorporating the inverse optimality approach into the missile interception guarantees the minimization of a performance index. In the framework of prescribed-time explicit reference governor, the control constraint is translated into a restriction of time-varying invariant set, and it follows the prescribed-time regulation of the applied reference along with the restriction satisfaction. The combined prescribed-time explicit reference governor approach could be transformed into a linear matrix inequality optimization problem, and its online solution over a receding horizon gives a Lyapunov function value for reference regulation and then control decisions in the continuous time. Simulation studies are performed to illustrate the performance of the proposed guidance control law. • Arctangent-based disturbance observer compensates lumped external disturbances for better performance. • Inverse optimality guidance approach guarantees the minimization of a performance index. • Prescribed-time ERG generates a time-varying invariant set for constraint satisfaction. • The online solution of LMI gives a Lyapunov function value for reference regulation. [ABSTRACT FROM AUTHOR]

This article addresses the problem regarding power regulation in classical DC-DC second-order converters by means of a nonlinear control technique based on inverse optimal control theory. There are few papers that describe inverse optimal control for DC-DC converters in the literature. Therefore, this study constitutes a contribution to the state of the art on nonlinear control techniques for DC-DC converters. In this vein, the main objective of this research was to implement inverse optimal control theory with integral action to the typical DC-DC conversion topologies for power regulation, regardless of the load variations and the application. The converter topologies analyzed were: (i) Buck; (ii) Boost; (iii) Buck-Boost; and (iv) Non-Inverting Buck-Boost. A dynamical model was proposed as a function of the state variable error, which helped to demonstrate that the inverse optimal control law with proportional-integral action implemented in the different converters ensures stability in each closed-loop operation via Lyapunov’s theorem. Numerical validations were carried out by means of simulations in the PSIM software, comparing the established control law, the passivity-based PI control law, and an open-loop control. As a conclusion, it was possible to determine that the proposed model is easier to implement and has a better dynamical behavior than the PI-PBC, ensuring asymptotic stability from the closed-loop control design.

Learning from demonstration, or imitation learning, is the process of learning to act in an environment from examples provided by a teacher. Inverse reinforcement learning (IRL) is a specific form of learning from demonstration that attempts to estimate the reward function of a Markov decision process from examples provided by the teacher. The reward function is often considered the most succinct description of a task. In simple applications, the reward function may be known or easily derived from properties of the system and hard coded into the learning process. However, in complex applications, this may not be possible, and it may be easier to learn the reward function by observing the actions of the teacher. This paper provides a comprehensive survey of the literature on IRL. This survey outlines the differences between IRL and two similar methods - apprenticeship learning and inverse optimal control. Further, this survey organizes the IRL literature based on the principal method, describes applications of IRL algorithms, and provides areas of future research. [ABSTRACT FROM AUTHOR]

An efficient inverse optimal control method named Adaptive Reference IOC is introduced to study natural walking with musculoskeletal models. Adaptive Reference IOC utilizes efficient inner-loop direct collocation for optimal trajectory prediction along with a gradient-based weight update inspired by structured classification in the outer-loop to achieve about 7 times faster convergence than existing derivative-free methods while maintaining similar outcomes in terms of gait trajectory matching. The proposed method adequately reconstructed the reference data when applied to experimental walking data from ten participants walking at various speeds and stride lengths. The proposed framework can facilitate efficient personalized cost function optimization for specific walking tasks, and provide guidance to personalized reference trajectory design for assistive robotic systems such as lower-limb exoskeletons. [ABSTRACT FROM AUTHOR]

Path Tracking Controller (PTC) plays a key role in achieving improved dynamic behaviour of Autonomous Vehicle (AV) while it is mainly responsible for implementing the planned paths. In this work, a learning-based Model Predictive Controller (MPC) is proposed to integrate a data-driven approach to the path tracking task using human driving demonstrations. The objective is to reproduce customised vehicle motion profile that is statistically comparable to a human driver with a proper trade-off between vehicle speed-acceleration profile and tracking accuracy. A combined longitudinal and lateral MPC controller with a novel feature-based parametric cost function is designed to perform the path tracking task under different road conditions. A high precision data logging system is developed to collect human driving demonstration data. From the analysis of the collected data, relevant features are chosen based on their applicability to produce a customised motion profile while performing AV’s path tracking task. Next, a bi-level optimisation-based Inverse Optimal Controller (IOC) is designed to learn the cost function’s parameters using the human demonstration data. The outcomes of the study show that the proposed controller can effectively regenerate some characteristics of human driving, including speed, longitudinal and lateral accelerations, and yaw rate. [ABSTRACT FROM AUTHOR]

A novel control strategy for secondary restoration of an islanded microgrid is proposed, focusing on restoring the frequency and voltage magnitude of an inverter-based distributed generator. A neural inverse optimal controller is integrated into the secondary control layer with a higher-order neural network trained using an extended Kalman filter (EKF). A practical and effective design strategy is provided. Unlike traditional approaches that rely on accurate mathematical models, the combination of inverse optimal control and neural networks does not require an accurate model and is more relevant to real-world engineering scenarios. To enhance the secondary control, the EKF optimization parameters are used in conjunction with the inverse optimal control to achieve more accurate and effective repair results. The synergistic effect improves control performance and ensures superior secondary recovery. Real-time validation is performed through rigorous simulations on the StarSim hardware-in-the-loop experimental platform. • Practical controller design for more real-world engineering: The approach combines inverse optimal control with neural networks, eliminating the need for exact models and providing a more practical and realistic controller design for the complexity of engineering applications. • Enhanced secondary control precision: The integration of EKF optimization with inverse optimal control results in a more accurate and effective secondary restoration. • Real-time validation through hardware-in-loop simulation: To validate the proposed control strategy in real-world scenarios, rigorous simulations are conducted on the StarSim Hardware-in-loop experimental platform. [ABSTRACT FROM AUTHOR]

• We propose a new framework to solve asset allocation/consumption under risk aversion. • This new model relies on the indirect utility as the primitive of the framework. • The indirect utility function is a function of wealth and is then easily recovered from investors' trading positions. • We show the efficiency of our approach through a strategy in the S&P500 market. Articles about asset allocation rely on the direct utility as a primitive to model the risk appetite of investors and to derive optimal asset allocation and consumption. In a simple setting, we show that the same problem can be solved where the indirect utility function is used as a primitive. Our approach offers various advantages. The indirect utility function measures the satisfaction at optimum and perfectly describes the risk appetite of investors. It is a function of wealth whereas the direct utility function depends on consumption. [ABSTRACT FROM AUTHOR]

This paper proposes an impulsive control scheme for a complex network that helps reduce the spread of two epidemic diseases: influenza type A and COVID-19. Both are respiratory infections; thus, they have a similar form of transmission, and it is possible to use the same control scheme in both study cases. The objective of this work is to use neural impulsive inverse optimal pinning control for complex networks to reduce the effects of propagation. The dynamic model is considered unknown, for which we design a neural identifier that, through training using the extended Kalman filter algorithm, provides the appropriate nonlinear model for this complex network. The dynamics of the network nodes are represented by the Susceptible-Infected-Removed (SIR) compartmental model in their discrete form. The results of the simulations are presented and addressed, applying the same control scheme but with different parameter values for each case study.

This article develops a methodology that enables learning an objective function of an optimal control system from incomplete trajectory observations. The objective function is assumed to be a weighted sum of features (or basis functions) with unknown weights, and the observed data is a segment of a trajectory of system states and inputs. The proposed technique introduces the concept of the recovery matrix to establish the relationship between any available segment of the trajectory and the weights of given candidate features. The rank of the recovery matrix indicates whether a subset of relevant features can be found among the candidate features and the corresponding weights can be learned from the segment data. The recovery matrix can be obtained iteratively and its rank non-decreasing property shows that additional observations may contribute to the objective learning. Based on the recovery matrix, a method for using incomplete trajectory observations to learn the weights of selected features is established, and an incremental inverse optimal control algorithm is developed by automatically finding the minimal required observation. The effectiveness of the proposed method is demonstrated on a linear quadratic regulator system and a simulated robot manipulator. [ABSTRACT FROM AUTHOR]

Scheduling therapies is a central paradigm in personalized medicine. Modeling and control theoretical approaches are opening an avenue for linking interactions between drugs and diseases in the host. In this paper, considering an inverse optimal approach, we formulate a framework based on a bounded input impulsive control to schedule therapies. Results for stability with a bounded input are introduced. We show the application of the algorithms with illustrative examples as well as the biomedical application of the long-term impact of therapies for a functional HIV cure. Results highlight that the proposed framework has the potential for tailoring therapies in different diseases such as diabetes, cancer, and infectious diseases. • Impulsive control law using an inverse optimal approach. • Scheduling Therapies in Infectious Diseases. • Bounded Input Impulsive Control. [ABSTRACT FROM AUTHOR]

In this paper it is presented the design of a controller for a reaction wheel pendulum using a discrete-time representation via optimal control from the point of view of passivity-based control analysis. The main advantage of the proposed approach is that it allows to guarantee asymptotic stability convergence using a quadratic candidate Lyapunov function. Numerical simulations show that the proposed inverse optimal control design permits to reach superior numerical performance reported by continuous approaches such as Lyapunov control functions and interconnection, and damping assignment passivity-based controllers. An additional advantage of the proposed inverse optimal control method is its easy implementation since it does not employ additional states. It is only required a basic discretization of the time-domain dynamical model based on the backward representation. All the simulations are carried out in MATLAB/OCTAVE software using a codification on the script environment.

Researchers in biomechanics, neuroscience, human–machine interaction and other fields are interested in inferring human intentions and objectives from observed actions. The problem of inferring objectives from observations has received extensive theoretical and methodological development from both the controls and machine learning communities. In this paper, we provide an integrating view of objective learning from human demonstration data. We differentiate algorithms based on the assumptions made about the objective function structure, how the similarity between the inferred objectives and the observed demonstrations is assessed, the assumptions made about the agent and environment model, and the properties of the observed human demonstrations. We review the application domains and validation approaches of existing works and identify the key open challenges and limitations. The paper concludes with an identification of promising directions for future work. [ABSTRACT FROM AUTHOR]

In this paper it is presented the design of a controller for a reaction wheel pendulum using a discrete-time representation via optimal control from the point of view of passivity-based control analysis. The main advantage of the proposed approach is that it allows to guarantee asymptotic stability convergence using a quadratic candidate Lyapunov function. Numerical simulations show that the proposed inverse optimal control design permits to reach superior numerical performance reported by continuous approaches such as Lyapunov control functions and interconnection, and damping assignment passivity-based controllers. An additional advantage of the proposed inverse optimal control method is its easy implementation since it does not employ additional states. It is only required a basic discretization of the time-domain dynamical model based on the backward representation. All the simulations are carried out in MATLAB/OCTAVE software using a codification on the script environment. [ABSTRACT FROM AUTHOR]

• Providing a historical review on inverse optimal control (IOC) and inverse reinforcement learning (IRL). • Providing a comparative view on IOC and IRL. • Categorizing the methods in IOC and IRL. • Explaining the most important methods of IOC and IRL in similar frameworks. • Discussing the challenges existing in IOC and IRL. Inverse optimal control (IOC) is a powerful theory that addresses the inverse problems in control systems, robotics, Machine Learning (ML) and optimization taking into account the optimal manners. This paper reviews the history of the IOC and Inverse Reinforcement Learning (IRL) approaches and describes the connections and differences between them to cover the research gap in the existing literature. The general formulation of IOC/IRL is described and the related methods are categorized based on a hierarchical approach. For this purpose, IOC methods are categorized under two classes, namely classic and modern approaches. The classic IOC is typically formulated for control systems, while IRL, as a modern approach to IOC, is considered for machine learning problems. Despite the presence of a handful of IOC/IRL methods, a comprehensive categorization of these methods is lacking. In addition to the IOC/IRL problems, this paper elaborates, where necessary, on other relevant concepts such as Learning from Demonstration (LfD), Imitation Learning (IL), and Behavioral Cloning. Some of the challenges encountered in the IOC/IRL problems are further discussed in this work, including ill-posedness, non-convexity, data availability, non-linearity, the curses of complexity and dimensionality, feature selection, and generalizability. [ABSTRACT FROM AUTHOR]

This paper presents an active controller for electric vehicles in which active front steering and torque vectoring are control actions combined to improve the vehicle driving safety. The electric powertrain consists of four independent in–wheel electric motors situated on each corner. The control approach relies on an inverse optimal controller based on a neural network identifier of the vehicle plant. Moreover, to minimize the number of sensors needed for control purposes, the authors present a discrete–time reduced–order state observer for the estimation of vehicle lateral and roll dynamics. The use of a neural network identifier presents some interesting advantages. Notably, unlike standard strategies, the proposed approach avoids the use of tire lateral forces or Pacejka’s tire parameters. In fact, the neural identification provides an input–affine model in which these quantities are absorbed by neural synaptic weights adapted online by an extended Kalman filter. From a practical standpoint, this eliminates the need of additional sensors, model tuning, or estimation stages. In addition, the yaw angle command given by the controller is converted into electric motor torques in order to ensure safe driving conditions. The mathematical models used to describe the electric machines are able to reproduce the dynamic behavior of Elaphe M700 in–wheel electric motors. Finally, quality and performances of the proposed control strategy are discussed in simulation, using a CarSim® full vehicle model running through a double–lane change maneuver.

Optimal operation of hydropower plants (HP) is a crucial task for the control of several variables involved in the power generation process, including hydraulic level and power generation rate. In general, there are three main problems that an optimal operation approach must address: (i) maintaining a hydraulic head level which satisfies the energy demand at a given time, (ii) regulating operation to match with certain established conditions, even in the presence of system’s parametric variations, and (iii) managing external disturbances at the system’s input. To address these problems, in this paper we propose an approach for optimal hydraulic level tracking based on an Inverse Optimal Controller (IOC), devised with the purpose of regulating power generation rates on a specific HP infrastructure. The Closed–Loop System (CLS) has been simulated using data collected from the HP through a whole year of operation as a tracking reference. Furthermore, to combat parametric variations, an accumulative action is incorporated into the control scheme. In addition, a Recurrent Neural Network (RNN) based on Feature Engineering (FE) techniques has been implemented to aid the system in the prediction and management of external perturbations. Besides, a landslide is simulated, causing the system’s response to show a deviation in reference tracking, which is corrected through the control action. Afterward, the RNN is including of the aforementioned system, where the trajectories tracking deviation is not perceptible, at the hand of, a better response with respect to use a single scheme. The results show the robustness of the proposed control scheme despite climatic variations and landslides in the reservoir operation process. This proposed combined scheme shows good performance in presence of parametric variations and external perturbations.

Model predictive control (MPC) is a multi-objective control technique that can handle system constraints. However, the performance of an MPC controller highly relies on a proper prioritization weight for each objective, which highlights the need for a precise weight tuning technique. In this paper, we propose an analytical tuning technique by matching the MPC controller performance with the performance of a linear quadratic regulator (LQR) controller. The proposed methodology derives the transformation of a LQR weighting matrix with a fixed weighting factor using a discrete algebraic Riccati equation (DARE) and designs an MPC controller using the idea of a discrete time linear quadratic tracking problem (LQT) in the presence of constraints. The proposed methodology ensures optimal performance between unconstrained MPC and LQR controllers and provides a sub-optimal solution while the constraints are active during transient operations. The resulting MPC behaves as the discrete time LQR by selecting an appropriate weighting matrix in the MPC control problem and ensures the asymptotic stability of the system. In this paper, the effectiveness of the proposed technique is investigated in the application of a novel vehicle collision avoidance system that is designed in the form of linear inequality constraints within MPC. The simulation results confirm the potency of the proposed MPC control technique in performing a safe, feasible and collision-free path while respecting the inputs, states and collision avoidance constraints.

In this paper, we show how a special class of inverse optimal control problems of elliptic partial differential equations can be solved globally. Using the optimal value function of the underlying parametric optimal control problem, we transfer the overall hierarchical optimization problem into a nonconvex single-level one. Unfortunately, standard regularity conditions like Robinson's CQ are violated at all the feasible points of this surrogate problem. It is, however, shown that locally optimal solutions of the problem solve a Clarke-stationarity-type system. Moreover, we relax the feasible set of the surrogate problem iteratively by approximating the lower level optimal value function from above by piecewise affine functions. This allows us to compute globally optimal solutions of the original inverse optimal control problem. The global convergence of the resulting algorithm is shown theoretically and illustrated by means of a numerical example. [ABSTRACT FROM AUTHOR]

Carrying out firefighting activities at disaster sites is extremely difficult. Therefore, robots that support and enhance these operations are required. In this paper, a crawler robot that tracks the moving path of a firefighter is proposed. It is commonly believed that trained firefighters select the best route; thus, it was assumed that this route is the easiest for the crawler robot as well. Using two 3D light detection and ranging sensors, once the firefighter's coordinates are detected, the coordinates are combined with 3D simultaneous localization and mapping results, then a target path is generated. The crawler robot follows the path using inverse optimal tracking control. The controller has a stability margin that guarantees robustness, which is an ideal property for disaster response robots used in severe conditions. The results of several experiments show that the proposed system is effective and practical for the crawler robot. [ABSTRACT FROM AUTHOR]

Recently, there has been an increasing interest towards understanding human movement behaviors. In this regard, one of the approaches is to retrieve the unknown underlying objective function that the human has to optimize while achieving a certain movement behavior. Existing research of behavioral understanding merely depends on predefined optimality criteria, where the minimum time, minimum variance or/and minimum effort are mainly adopted. These criteria are assumed to be constant, where the human is assumed to have the same preferences during the movement duration. However, in this paper, the optimality criteria underlying the kinematic characteristics of a certain human behavior are assumed to be exponentially discounted to account for the change in the human preferences that could happen while achieving this behavior. A new Inverse Discounted-based Linear Quadratic Regulator (ID-LQR) algorithm is developed in the light of Inverse Optimal Control (IOC) framework to find out the discounted cost function that could reproduce the measured human behavior perfectly. Meanwhile, an Incremental version of the ID-LQR algorithm is proposed to continuously refine the so far learned cost function in the case of sequentially presented demonstrations. The saccadic eye gaze movement is studied as an example to quantify both the proposed ID-LQR and Inverse ID-LQR approaches. Simulation results are encouraging and show that the saccadic trajectories generated by ID-LQR approach match the experimental data in many aspects, including position and velocity profiles of saccades. Moreover, when it is assessed by a subsequent set of scenarios, the incremental ID-LQR algorithm confirms its capability to generalize the so far retrieved cost function for the unseen saccadic demonstrations. [ABSTRACT FROM AUTHOR]

This paper contributes a novel framework to efficiently learn cost-to-go function representations for robotic tasks with latent modes. The proposed approach relies on the principle behind ensemble methods, where improved performance is obtained by aggregating a group of simple models, each of which can be efficiently learnedq. The maximum-entropy approximation is adopted as an effective initialization and the quality of this surrogate is guaranteed by a theoretical bound. Our approach also provides an alternative perspective to view the popular mixture of Gaussians under the framework of inverse optimal control. We further propose to enforce a dynamics on the model ensemble, using Kalman estimation to infer and modulate model modes. This allows robots to exploit the demonstration redundancy and to adapt to human interventions, especially in tasks where sensory observations are non-Markovian. The framework is demonstrated with a synthetic inverted pendulum example and online adaptation tasks, which include robotic handwriting and mail delivery. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the distributed optimal control for multi-agent systems with impulsive effects. We present sufficient condition to guarantee that the distributed protocol can make desired performance index reach minimal value for the impulsive multi-agent systems and the general case, respectively. An example is also presented to illustrate the efficiency of the obtained results. [ABSTRACT FROM AUTHOR]

The variability of renewable energies and their integration into the grid via power electronics demands the design of robust control algorithms. This work incorporates two techniques to ensure the stability of a boost converter through its state equations, implementing the inverse optimal control and the gain-scheduling technique for robust control settings. In such a way that, under a single adjustment, it is capable of damping different changes such as changes in the parameters, changes in the load, the input voltage, and the reference voltage. On the other hand, inverse optimal control is based on a discrete-time control Lyapunov function (CLF), and CLF candidate depends on fixed parameters that are selected to obtain the solution for inverse optimal control. Once these parameters have been found through heuristic or artificial intelligence methods, the new proposed methodology is capable of obtaining a robust optimal control scheme, without having to search for new parameters through other methods, since these are sometimes sensitive changes and many times the process of a new search is delayed. The results of the approach are simulated using Matlab, obtaining good performance of the proposed control under different operation conditions. Such simulations yielded errors of less than 1% based on the voltage reference, given the disturbances caused by changes in the input variables, system parameters, and changes in the reference. Thus, applying the new methodology, the stability of our system was preserved in all cases.

This paper mathematically explains how state derivative space (SDS) system form with state derivative related feedback can supplement standard state space system with state related feedback in control designs. Practically, inverse optimal control is attractive because it can construct a stable closed-loop system while optimal control may not have exact solution. Unlike the previous algorithms which mainly applied state feedback, in this paper inverse optimal control are carried out utilizing state derivative alone in SDS system. The effectiveness of proposed algorithms are verified by design examples of DC motor tracking control without tachometer and very challenging control problem of singular system with impulse mode. Feedback of direct measurement of state derivatives without integrations can simplify implementation and reduce cost. In addition, the proposed design methods in SDS system with state derivative feedback are analogous to those in state space system with state feedback. Furthermore, with state derivative feedback control in SDS system, wider range of problems such as singular system control can be handled effectively. These are main advantages of carrying out control designs in SDS system.

In this tutorial we study a safety analog of the classical zero-sum differential game with positive definite penalties on the state and the two inputs. Consider a nonlinear system affine in two inputs, which are called "offender" and "defender." Let the inputs have the opposing objectives in relation to an infinite-time cost which, in addition to penalizing the inputs of both agents, incorporates a safety index of the system (a barrier function), with the defender aiming to maximize the system safety and the offender aiming to minimize it. If there is a pair of (offender, defender) non-Nash feedback policies of the L g h form with a safe outcome, namely, where the defender maintains safety while the offender fails to violate safety, then there exists an inverse optimal pair of policies that attain a Nash equilibrium relative to the safety minimax objective. In the tutorial we study both deterministic and stochastic offenders. The deterministic offender applies its feedback through its deterministic input value, while the stochastic offender applies its feedback through its incremental covariance. In addition to Nash policies for a minimax offender–defender formulation, we provide feedback laws for the defender, in the scenario where the offender action is unrestricted by optimality, and where the defender ensures input-to-state safety in the deterministic and stochastic senses. This tutorial is derived from our recent article on inverse optimal safety filters, by setting the nominal control to zero and declaring the disturbance to be the offender agent. Among several illustrative examples, one is particularly interesting and unconventional. We consider a safety game played on a unicycle vehicle between its two inputs: the angular velocity and the linear velocity, as the opposing players. We consider two scenarios. In the first, the angular velocity, acting as an offender, attempts to run the vehicle into an obstacle by steering, while the linear velocity, acting as a defender, drives the vehicle forward or in reverse to prevent the vehicle being run into the obstacle. In the second scenario, the linear velocity acts as an offender and angular velocity acts as a defender (in the deterministic case by varying the heading rate; in the stochastic case by varying the variance of a white noise driving the heading rate). A "wind" towards the obstacle advantages the offender in both scenarios. The input policies derived are optimal in the sense of their opposite objectives, under the best possible policy of the opponent, under meaningful costs on their actions. The linear velocity input prevails, whether acting in the role of a defender, in which case the collision with the obstacle is prevented, or in the role of an offender, in which case the collision with the obstacle is achieved. [ABSTRACT FROM AUTHOR]

Robots collaborating naturally with a human partner in a confined workspace need to understand and predict human motions. For understanding, a model-based approach is required as the human motor control system relies on the biomechanical properties to control and execute actions. The model-based control models explain human motions descriptively, which in turn enables predicting and analyzing human movement behaviors. In motor control, reaching motions are framed as an optimization problem. However, different optimality criteria predict disparate motion behavior. Therefore, the inverse problem—finding the optimality criterion from a given arm motion trajectory—is not unique. This paper implements an inverse optimal control (IOC) approach to determine the combination of cost functions that governs a motion execution. The results indicate that reaching motions depend on a trade-off between kinematics and dynamics related cost functions. However, the computational efficiency is not sufficient for online prediction to be utilized for HRI. In order to predict human reaching motions with high efficiency and accuracy, we combine the IOC approach with a probabilistic movement primitives formulation. This hybrid model allows an online-capable prediction while taking into account motor variability and the interpersonal differences. The proposed framework affords a descriptive and a generative model of human reaching motions which can be effectively utilized online for human-in-the-loop robot control and task execution.

Abstract This study considers the cooperative guidance problem for multiple missiles against a stationary target with the field-of-view constraint. A two-stage guidance scheme is proposed to realize salvo attack of multiple missiles without using any information on time-to-go or its estimation. In the first guidance phase, this study integrates the cooperative guidance problem, state tracking problem and field-of-view constraint into a unified optimal control framework. A non-quadratic field-of-view constraint cost is innovatively constructed via an inverse optimal control approach, which leads to an analytical, distributed, and optimal guidance law to provide favored initial conditions for the latter phase. The stability and optimality of the closed-loop system is also proven. With respect to the second guidance phase, missiles disconnect from each other and are governed by the typical pure proportional navigation guidance law to eventually realize the salvo attack mission. Numerical simulations are performed to validate the theoretical results. [ABSTRACT FROM AUTHOR]