Search

This paper suggests a procedure to construct the Pareto frontier and efficiently computes the strong Nash equilibrium for a class of time-discrete ergodic controllable Markov chain games. The procedure finds the strong Nash equilibrium, using the Newton optimization method presenting a potential advantage for ill-conditioned problems. We formulate the solution of the problem based on the Lagrange principle, adding a Tikhonov’s regularization parameter for ensuring both the strict convexity of the Pareto frontier and the existence of a unique strong Nash equilibrium. Then, any welfare optimum arises as a strong Nash equilibrium of the game. We prove the existence and characterization of the strong Nash equilibrium, which is one of the main results of this paper. The method is validated theoretically and illustrated with an application example.

A traditional non-cooperative bargaining situation involves two or more forward-looking players making offers and counteroffers alternately until an agreement is reached, with a penalty according to the time taken by players in the decision-making process. We introduce a game that aids myopic players to reach the equilibrium as if they were forward-looking agents. The key elements of the game are that players are penalized both for their deviation from the previous best-reply strategy and their time taken for the decision-making at each step of the game. It is shown that our game has an equilibrium not only for the traditional processes and utilities used in traditional non-cooperative bargaining literature, but for an expanded and very comprehensive set of stochastic processes (such as Markov processes) and utility functions. Our work not only complements traditional non-cooperative bargaining literature for myopic agents, but also enlarges the class of processes and functions where Rubinstein’s non-cooperative bargaining solutions might be defined and applied.

The aim of this study is to develop robust guidance laws for the control motion of an underwater autonomous vehicle (UAV) in a three-dimensional (3D) space. The control design is based on the use of Averaged Sub-Gradient (ASG) version of a class of dynamic integral sliding mode (ISM) method being sequentially applied to the subsystems of the complete model realizing, the so-called backstepping (or cascade) approach. The mathematical form of the UAV model induces a backstepping formulation for solving the tracking trajectory problem sequentially for the position, translation velocity, angular velocity and actuators (thrusters) dynamics. The solution of the trajectory tracking problem at each stage implements the ASG-version of the ISM method. This problem is treated as the optimization of a suitable convex (not obligatory strongly convex) cost functional, depending on the tracking error and reaching its minimal value at the origin of the error tracking space. This study shows that the minimization of the proposed functional leads to the optimal tracking regime under the presence of uncertainties in the mathematical model description. A numerical example proves the effectiveness of the suggested robust dynamic controller. The comparison between the obtained trajectory tracking results and the outcomes produced by a set of standard proportional integral derivative (PID) controllers, is presented. The proposed controller exhibits a better tracking of the reference trajectory compared with the PID version, showing a smaller mean square estimation for the tracking error.

This paper presents a novel approach for computing the Kalai–Smorodinsky bargaining equilibrium for continuous time and discrete states Markov chains games. To solve the bargaining situation we set the disagreement point as the Nash equilibrium of the problem, then to find the new agreement point we follow the bargaining model presented by Kalai–Smorodinsky employing the utopia point concept. We exemplify the game formulation in terms of nonlinear programming equations implementing the Lagrange principle. The Tikhonov’s regularization method is applied to ensure the convergence of the cost-functions to an equilibrium point. For solving the problem we use a programming method implemented by the extraproximal optimization approach. The proposed method is validated by a numerical example related to the labor market problem for a three-person bargaining problem.

The classical transfer pricing model considers a cooperative approach taking into account that each division purchases goods from an upstream division in the supply chain. However, divisions of a firm can be decentralized and not necessarily act cooperatively. A conflict in decentralization arises. A lack of coordination between the divisions and central management occurs if the divisions have full autonomy: managers might engage in actions that would benefit their divisions to the detriment of maximizing the well-being of the entire firm. This paper presents a model for transfer pricing that is able to maintain divisional decentralization by computing the Nash equilibrium and at the same time encourage divisions to achieve central management optimal results by computing the strong Nash equilibrium. The resulting strong Nash/Nash equilibrium point involves the computation of a cooperative and non-cooperative behavior. The model is able to provide a measure of the divisional profit maximization of the firm. In addition, we consider a bi-level structure of the company’s supply chain and solve the problem employing a Stackelberg equilibrium approach. For solving the problem we represent the transfer pricing game in terms of a coupled nonlinear programming equations implementing the penalty approach. A regularization method is employed to ensure the convergence of the utility-functions to a unique equilibrium point. For computing the equilibrium point we employ a minimization of the Euclidean distance. Next, we transform the penalty problem into a new system of equations in the Euclidean distance format. For minimizing the Euclidean distance we use a gradient method approach. A numerical example validates the effectiveness and usefulness of the proposed model for transfer pricing.

This paper suggests a new controller exploitation-exploration (CEE) reinforcement learning (RL) architecture that attains a near-optimal policy. The proposed architecture consists of three modules: controller, fast-tracked learning and the actor-critic. The strategies are represented by a probability distribution $$c_{ik}$$ . The controller employs a combination (balance) of the exploration or exploitation using the Kullback–Leibler divergence deciding if the new strategies are better than currently employed immediate strategy. The exploitation uses a fast-tracked learning algorithm, which employs a fix strategy and priori knowledge. The method is (only) asked to find estimated values of the transition matrices and utilities. The exploration employs an actor-critic architecture. The actor is responsible for the computation of the strategies using a policy gradient method. The critic determines the acceptance of the proposed strategies. We show the convergence of the proposed algorithms for implementing the architecture. An application example related to inventory shows the effectiveness of the proposed architecture.

This paper considers the subject of penalty regularized expected utilities and investigates the applicability of the method for computing the mean–variance Markowitz customer portfolio optimization problem. We penalize the large values by introducing a penalty term expressed as least-squares in order to avoid an explosive number of solutions. This penalty term is known as the Tikhonov regularization parameter. Tikhonov’s regularization is one of the most popular approaches to solve discrete ill-posed problems and, in our case, it plays a fundamental role in order to ensure the convergence to a unique portfolio solution. In this sense, we first provide the parameter conditions under which the penalty regularized expected utility of a given optimal portfolio admits a unique solution. A crucial problem concerning Tikhonov’s regularization is the proper choice of the regularization parameter because it can modify (sometimes significantly) the shape of the original functional. The main objective of this paper is to derive a method for regularization in an optimal way. For solving the problem, the parameters of the regularized poly-linear optimization problem are balanced simultaneously. Then, we prove that the original Markowitz portfolio optimization problem converges to an exact solution (with the minimal weighted norm). We consider a projection gradient method for finding the extremal points including the proof of convergence of the method. We show how to select the parameters of the algorithm in order to guarantee the convergence of the suggested procedure. Finally, we present a numerical example to illustrate the practical implications of the theoretical issues of a penalty regularized portfolio optimization problem.

This paper presents a novel algorithm for Simultaneous Localization and Mapping (SLAM) of mobile robots. The proposed algorithm, termed as SLAM using on Attractive Ellipsoid Method (AEM) and Luenberger filter-type application with nonlinear models, under the hypothesis that errors affecting all sensor’s measurements and robot’s motions are unknown-but-bounded. We suggest to select the best parameters of the suggested observer providing a minimal size of this attractive ellipsoid. It is shown that this optimization problem may be converted into the trace optimization of this attractive ellipsoid under some specific constraints of a set of LMI’s (Linear Matrix Inequalities). Simulated indoor experiments are used to demonstrate the performance of the proposed algorithm.

This paper proposes a new approach for designing a nonlinear optimal controller with an integral sliding mode component employing a generalization of the saddle point method which consists on controlling a controllable nonlinear system. Based on the initial and final conditions of the dynamical system, we consider an open-loop control such that the state of the system can be moved to a neighborhood of the equilibrium state corresponding to the given final condition. The implementation of the method for solving the problem involves a two-step iterated procedure: (i) The first step consists of a “prediction” which calculates the preliminary position approximation to the steady-state point, and (ii) the second step is designed to find a “basic adjustment” of the previous prediction. We apply the controller to a Cartesian 3D-crane. The formulation of the 3D-crane is in terms of nonlinear programming problems implementing the Lagrange principle. We transform the problem in a system of equations where each equation is itself an optimization problem. For designing the controller we suggest to employ an integral sliding mode method which suppress the model uncertainties consequence of moving the trolley and the bridge, lifting the cargo as well as external forces. As a result, the optimal controller will be simultaneously able to lift the cargo, suppressing the payload vibration, tracking the trolley and moving the bridge. A numerical example involving the simulation of a 3D-crane shows the effectiveness of the controller.

In this paper, we present a multi-objective solution of the Pareto front for a certain class of discrete-time ergodic controllable Markov chains. For solving the problem, we transform the original multi-objective problem into an equivalent nonlinear programming problem implementing the Lagrange principle. We then propose to adopt the Tikhonov’s regularization method to ensure the convergence of the cost functions to a unique point into the Pareto front. We show that Pareto policies are characterized as optimal policies. One of the fundamental problems related to the construction of the Pareto front is the existence and characterization of both, Pareto and strong Pareto policies. By introducing the Tikhonov’s regularizator, we ensure the existence of strong Pareto policies. This paper proposes a real solution to this problem. We formulate the original problem considering several constraints: (a) we employ the c-variable method for the introduction of linear constraints over the nonlinear problem and (b) restrict the cost functions, allowing points in the Pareto front to have a small distance from one another. The constraints imposed by the c-variable method make the problem computationally tractable and the restriction imposed by the small distance change ensures the continuation of the Pareto front. The resulting equation in this nonlinear system is an optimization problem for which the necessary and efficient condition of a minimum is solved using the projected gradient method. Moreover, we also prove the convergence conditions and compute the estimate rate of convergence of variables corresponding to the Lagrange principle and the Tikhonov’s regularization, respectively. We provide all the details needed to implement the proposed method in an efficient way. The usefulness of the method is successfully demonstrated by a numerical example.

In this paper we generalize two models for chemical reactions, based on continuous-time Markov chains, into continuous-time Markov decision process. We propose a mathematical optimization approach for solving an average optimality criterion in state-discrete continuous-time Markov decision process. Our proposal extends the c-variable method used in discrete-time decision process by introducing a new linear constraint for continuous time. The advantage of our approach is that it reduces the continuous-time Markov decision process to a discrete-time Markov decision process where the linear constraints make the problem computationally tractable. The usefulness of the method is illustrated in chemical reactions where the concentration dynamics is modelled as a continuous-time Markov chain. The first application is a single reversible reaction for the formation of the amidogen radical where we found the optimal temperature that minimizes the average expected rate of H formation at steady state. The second is a chemical reaction network for the proton transfer, hydration and tautomeric reaction of anthocyanin pigments, in this case we found an optimal strategy over a set of values of $$\hbox {H}^{+}$$ that minimizes the average expected total number of molecules at steady state.

In this paper, we present a new method for finding a fixed local-optimal policy for computing the customer lifetime value. The method is developed for a class of ergodic controllable finite Markov chains. We propose an approach based on a non-converging state-value function that fluctuates (increases and decreases) between states of the dynamic process. We prove that it is possible to represent that function in a recursive format using a one-step-ahead fixed-optimal policy. Then, we provide an analytical formula for the numerical realization of the fixed local-optimal strategy. We also present a second approach based on linear programming, to solve the same problem, that implement the c-variable method for making the problem computationally tractable. At the end, we show that these two approaches are related: after a finite number of iterations our proposed approach converges to same result as the linear programming method. We also present a non-traditional approach for ergodicity verification. The validity of the proposed methods is successfully demonstrated theoretically and, by simulated credit-card marketing experiments computing the customer lifetime value for both an optimization and a game theory approach.

This article describes the application of the so-called attractive ellipsoidal method to solve the trajectory stabilization problem for a class of genetic network systems modelled by a stochastic model. The genetic network model is described by a stochastic quasi-linear system affected by additive and multiplicative noises simultaneously. The solution of the control design provided in this study is based on a linear feedback structure. In this paper the algorithm to construct a suboptimal gain for adjusting the control design is introduced. The attractive ellipsoidal method is the key stone for designing the so-called suboptimal gain. Moreover, the practical stability of the genetic network trajectories is demonstrated on the mean and in almost sure senses. Some numerical simulations show how a set of stochastic trajectories are stabilized by the controller suggested in this study and how the predicted ellipsoid region is achieved by these trajectories.

This paper deals with time-varying parameter estimation of stochastic systems under dependent noise perturbations. The filter, which generates this dependent noise from a standard "white noise," is assumed to be partially known (a nominal plant plus a bounded deviation). The considered approach consists of two consecutive steps. At the first step, the application of a sliding-mode-type algorithm is suggested, providing a finite-time equivalence of the original stochastic process with unknown parameters to an auxiliary one. Such an "equivalence" does not cancel the noise effects, but allows one to identify the model in the "regression form" for a sufficiently short time and, simultaneously, to transform the dependent noise, keeping bounded uncertainties as an external unmeasured dynamics. At the second step the least squares method with a scalar forgetting factor (LSMFF) is applied to estimate time-varying parameters of the given model. A convergence zone analysis is presented. A numerical example illustrates the effectiveness of the proposed approach.

This paper analyzes the mathematical modeling procedure to describe the decomposition of adsorbed phenanthrene in prototypical and real soil samples (sand and agricultural soil, respectively) by ozone. The modeling scheme considered a set of ordinary differential equations with time varying coefficients. This model used the adsorbed ozone in the soil, the ozone reacting with the contaminant and the phenanthrene concentration in the soil sample. The main parameters involved in the mathematical model included a time varying ozone saturation function (k sat (t)) and reaction constants (k r). These parameters were calculated using the ozone concentration variation at the reactor output, named as ozonogram, and the measurements of phenanthrene decomposition through ozonation. The model was validated using two series of experiments: (1) soil saturated with ozone in the absence of the contaminant and (2) soil artificially contaminated with phenanthrene. In both cases, the proposed parametric identification method yields to validate the mathematical model. This fact was confirmed by the correspondence between numerical simulations and experimental data. In particular, total decomposition of phenanthrene adsorbed in two different systems (ozone-sand and ozone-agricultural soil) was obtained after 15 and 30 min of reaction, respectively. This difference was obtained as a consequence of soil physicochemical characteristics: specific surface area and pore volume. The ozonation reaction rate constants of phenanthrene in the sand and agricultural soil were calculated using the same parameter identification scheme.

To identify a Saccharomyces cerevisiae fed-batch fermentation process, it is suggested that a differential (dynamic in continuous time) neural network (DNN) be implemented. A priori information on the dynamic equations and their parameters is not required to realize this approach. Based on real data, obtained from laboratory experiments, this DNN of a simple structure (six neurons) is shown to have a high capability of identifying this process (to reproduce the input–output relation) after several working hours of learning.