Search

In this paper, we study a kind of optimal control problem for forward-backward stochastic differential equations (FBSDEs for short) of McKean–Vlasov type via the dynamic programming principle (DPP for short) motivated by studying the infinite dimensional Hamilton–Jacobi–Bellman (HJB for short) equation derived from the decoupling field of the FBSDEs posed by Carmona and Delarue (2015) [28]. At the beginning, the value function is defined by the solution to the controlled BSDE alluded to earlier. On one hand, we can prove the value function is deterministic function with respect to the initial random variable; On the other hand, we can show that the value function is law-invariant , i.e., depending on only via its distribution by virtue of BSDE property. Afterward, utilizing the notion of differentiability with respect to probability measures introduced by P.L. Lions [50] , we are able to establish a DPP for the value function in the Wasserstein space of probability measures based on the application of BSDE approach, particularly, employing the notion of stochastic backward semigroups associated with stochastic optimal control problems and Itô formula along a flow property of the conditional law of the controlled forward state process. We prove that the value function is the unique viscosity solutions of the associated generalized HJB equations in some separable Hilbert space. Finally, as an application, we formulate an optimal control problem for linear stochastic differential equations with quadratic cost functionals of McKean-Vlasov type under nonlinear expectation, g -expectation introduced by Peng [43] and derive the optimal feedback control explicitly by means of several groups of Riccati equations. [ABSTRACT FROM AUTHOR]

A feedback control method is proposed to asymptotically stabilize the chain-like multi-degree-of-freedom (MDOF) nonlinear structural system under purely parametric Gaussian white noises. First, the motion equation of a chain-like nonlinear structural system with n coupled elements and an undetermined control law is derived. It is reduced to a one-dimensional partially averaged Itô stochastic differential equation of the controlled Hamiltonian by applying the stochastic averaging method, which bypasses the challenge of calculating multiple Lyapunov exponents. Next, based on the dynamical programming principle, the dynamical programming equation of the averaged system with an undetermined cost function is established and addressed to obtain the optimal control law. Then, the Lyapunov exponent of the completely averaged Itô equation is derived. For the given requirement of stabilizing the system, the Lyapunov exponent is entirely fixed and used to analyze the stability of the originally chain-like MDOF nonlinear structural system. A controlled chain-like 8-DOF nonlinear structural system is taken as an example to illustrate the application and effectiveness of the proposed procedure and the effect of control forces on stabilizing the system. [ABSTRACT FROM AUTHOR]

The collocation method is one of the most powerful and effective techniques to solve nonlinear differential equations. This method reduces the original problem to a system of algebraic equations. Awareness of the nature of the problem and the analytical behavior of the solution has an important influence on the choice of type and number of basis functions required by the collocation method for solving the different classes of differential equations. In general, the polynomial basis functions such as the Legendre polynomials are not always an appropriate choice to approximate the functions that can be expanded based on monomial basis functions with real powers. In such cases, a combination of polynomial basis functions with some maps to replace the natural powers with arbitrary real powers can be a suitable alternative. This paper presents a collocation method based on the shifted fractional-order Legendre functions to solve the linear and nonlinear Hamilton–Jacobi–Bellman PDEs in stochastic optimal control problems. The convergence analysis of the proposed method has also been investigated. Moreover, the method is used to solve some practical problems with different solution structures such as resource extraction and Merton's portfolio selection models. Numerical results for four different examples indicated that the collocation method based on the fractional-order Legendre functions are provided. Due to the non differentiability of the solution of HJB PDEs at x = 0 for resource extraction and Merton's portfolio selection models, the Legendre collocation method cannot produce solutions with the desired accuracy. In fact, it is observed that the collocation method based on the fractional-order Legendre functions can be more efficient than the standard Legendre collocation method. Also, the sample paths of the control and state processes are obtained together with the estimate given by the Monte Carlo simulation. [ABSTRACT FROM AUTHOR]

In this paper, we propose a low rank approximation method for efficiently solving stochastic partial differential equations. Specifically, our method utilizes a novel low rank approximation of the stiffness matrices, which can significantly reduce the computational load and storage requirements associated with matrix inversion without losing accuracy. To demonstrate the versatility and applicability of our method, we apply it to address two crucial uncertainty quantification problems: stochastic elliptic equations and optimal control problems governed by stochastic elliptic PDE constraints. Based on varying dimension reduction ratios, our algorithm exhibits the capability to yield a high-precision numerical solution for stochastic partial differential equations, or provides a rough representation of the exact solutions as a pre-processing phase. Meanwhile, our algorithm for solving stochastic optimal control problems allows a diverse range of gradient-based unconstrained optimization methods, rendering it particularly appealing for computationally intensive large-scale problems. Numerical experiments are conducted and the results provide strong validation of the feasibility and effectiveness of our algorithm. [ABSTRACT FROM AUTHOR]

In control theory, typically a nominal model is assumed based on which an optimal control is designed and then applied to an actual (true) system. This gives rise to the problem of performance loss because of the mismatch between the true and assumed models. A robustness problem in this context is to show that the error because of the mismatch between a true and an assumed model decreases to zero as the assumed model approaches the true model. We study this problem when the state dynamics of the system are governed by controlled diffusion processes. In particular, we discuss continuity and robustness properties of finite and infinite horizon α-discounted/ergodic optimal control problems for a general class of nondegenerate controlled diffusion processes as well as for optimal control up to an exit time. Under a general set of assumptions and a convergence criterion on the models, we first establish that the optimal value of the approximate model converges to the optimal value of the true model. We then establish that the error because of the mismatch that occurs by application of a control policy, designed for an incorrectly estimated model, to a true model decreases to zero as the incorrect model approaches the true model. We see that, compared with related results in the discrete-time setup, the continuous-time theory lets us utilize the strong regularity properties of solutions to optimality (Hamilton–Jacobi–Bellman) equations, via the theory of uniformly elliptic partial differential equations, to arrive at strong continuity and robustness properties. Funding: The research of S. Yüksel was partially supported by the Natural Sciences and Engineering Research Council of Canada (NSERC). [ABSTRACT FROM AUTHOR]

Optimal control theory is a branch of mathematics. It is developed to find optimal ways to control a dynamic system. In 1957, R.Bellman applied dynamic programming to solve optimal control of discrete-time systems. His procedure resulted in closed-loop, generally nonlinear, and feedback schemes. Optimal control problems which will be tackled involve the minimization of a cost function subject to constraints on the state vector and the control. Lagrange multipliers provide a method of converting a constrained minimization problem into an unconstrained minimization problem of higher order. The necessary condition for optimality can be obtained as the solution of the unconstrained optimization problem of the Lagrange function and the bordered Hessian matrix is used for the second-derivative test. A spectral method for solving optimal control problems is presented. The method is based on Bernoulli polynomials approximation. By using the Bernoulli operational matrix of integration and the Lagrangian function, stochastic optimal control is transformed into an optimisation problem, where the unknown Bernoulli coefficients are determined by using Newton’s iterative method. The convergence analysis of the proposed method is given. The simulation results based on the Monte-Carlo technique prove the performance of the proposed method. Some error estimations are provided and illustrative examples are also included to demonstrate the efficiency and applicability of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper we study optimal trading strategies in a financial market in which stock returns depend on a hidden Gaussian mean reverting drift process. Investors obtain information on that drift by observing stock returns. Moreover, expert opinions in the form of signals about the current state of the drift arriving at fixed and known dates are included in the analysis. Drift estimates are based on Kalman filter techniques. They are used to transform a power utility maximization problem under partial information into an optimization problem under full information where the state variable is the filter of the drift. The dynamic programming equation for this problem is studied and closed-form solutions for the value function and the optimal trading strategy of an investor are derived. They allow to quantify the monetary value of information delivered by the expert opinions. We illustrate our theoretical findings by results of extensive numerical experiments. [ABSTRACT FROM AUTHOR]

Uncrewed aircraft systems and advanced air mobility are associated with uncertain conditions, such as wind prediction errors, and they may deviate from dedicated airspaces referred to as corridors. However, further studies on these aspects are required. This study focuses on the covariance control for a stochastic discrete-time linear system under correlated uncertainties. A covariance control problem with chance constraints can be formulated as a convex-programming problem. Numerical simulations of uncrewed aircraft systems path planning can be used to compare two control laws: covariance control laws that consider correlated and noncorrelated noise. Numerical simulations were conducted to verify that the covariance control law considering correlated noise generates a trajectory that appropriately satisfies the constraints. Consequently, the effect of considering correlated noise in the covariance control law was clarified. In addition, covariance control has been demonstrated to generate appropriate trajectories for various state constraints. Moreover, numerical simulations indicated that covariance control facilitated the control of uncertainty, and the effectiveness of the covariance control was evaluated. The insights gained from the results of this study can inform enhanced uncertainty management for improving reliability and safety in real-world applications of uncrewed aircraft systems and advanced air mobility. [ABSTRACT FROM AUTHOR]

This study refines the methodology for solving stochastic optimal control problems with quality criteria that include the sum of the quality functional of the classical formulation and an extremal measure. A two-level optimization solution of these kinds of problems is presented already for the case where the quality functional consists only of the extremal measure. Our study shows the possibility of solving the original time inconsistency problem through solving a two-level optimization problem, where the outer problem is solved by gradient methods since the value function is convex and the inner problem is solved by classical methods. Some experiments were carried out and confirmed the validity of the theory. The results of the study can be applied to the case of portfolio management with quality criteria containing the Conditional Value-at-Risk (CVaR) metric. [ABSTRACT FROM AUTHOR]

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition. [ABSTRACT FROM AUTHOR]

When companies or organizations can evade audits on their harmful incidents, how should the affected entities design their audit and penalty policies? In "Audit and Remediation Strategies in the Presence of Evasion Capabilities" by Wang, de Véricourt, and Sun, the authors find random audits may be needed in the optimal policy. Specifically, the optimal policy alternates between ascending monetary penalties (without any audits) and random audits at a constant rate (when the penalty reach its maximum level). Only when the evasion is ineffective or the self-correction is too costly do deterministic audits become optimal. They tackle the problem in a continuous-time principal-agent framework with both adverse selection and moral hazard. In this paper, we explore how to uncover an adverse issue that may occur in organizations with the capability to evade detection. To that end, we formalize the problem of designing efficient auditing and remedial strategies as a dynamic mechanism design model. In this setup, a principal seeks to uncover and remedy an issue that occurs to an agent at a random point in time and that harms the principal if not addressed promptly. Only the agent observes the issue's occurrence, but the principal may uncover it by auditing the agent at a cost. The agent, however, can exert effort to reduce the audit's effectiveness in discovering the issue. We first establish that this setup reduces to the optimal stochastic control of a piecewise deterministic Markov process. The analysis of this process reveals that the principal should implement a dynamic cyclic auditing and remedial cost-sharing mechanism, which we characterize in closed form. Importantly, we find that the principal should randomly audit the agent unless the agent's evasion capacity is not very effective, and the agent cannot afford to self-correct the issue. In this latter case, the principal should follow predetermined audit schedules. Funding: This work was supported by the Deutsche Forschungsgemeinschaft (German Research Foundation) ["Audit Schedules in the Presence of Concealing Effort"; Grant 387250733]. Supplemental Material: The computer code and data that supports the findings of this study and the online appendix are available within this article's supplemental material at https://doi.org/10.1287/opre.2022.0289. [ABSTRACT FROM AUTHOR]

Problem definition: More than 99% of the new power generation capacity to be installed in the United States from 2023 to 2050 will be powered by wind, solar, and natural gas. Additionally, large-scale battery systems are planned to support power systems. It is paramount for policymakers and electric utilities to deepen the understanding of the operational and investment relations among renewable, flexible (natural gas-powered), and storage capacities. In this paper, we optimize both the joint operations and investment mix of these three types of resources, examining whether they act as investment substitutes or complements. Methodology/results: Using stochastic control theory, we identify and prove the structure of the optimal storage control policy, from which we determine various pairs of charging and discharging operations. We find that whether storage complements or substitutes other resources hinges on the operational pairs involved and whether executing these pairs is constrained by charging or discharging. Through extensive numerical analysis using data from a Florida utility, government agencies, and industry reports, we demonstrate how storage operations drive the investment relations among renewable, flexible, and storage capacities. Managerial implications: Storage and renewables substitute each other in meeting peak demand; storage complements renewables by storing surplus renewable output; renewables complement storage by compressing peak periods, facilitating peak shaving and displacement of flexible capacity. These substitution and complementary effects often coexist, and the dominant effect can alternate as costs change. A thorough understanding of these relations at both operational and investment levels empowers decision makers to optimize energy infrastructure investments and operations, thereby unlocking their full potential. Funding: This research was supported in part by Lilly Endowment, Inc., through its support for the Indiana University Pervasive Technology Institute. This research was also supported by Kelley School of Business, Indiana University, and Haslam College of Business, University of Tennessee. O. Q. Wu thanks Grant Thornton for their generous support. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.0068. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

This article concentrates in analyzing optimal controls for stochastic integrodifferential equation (SIDE) in Hilbert space. Necessary parameters are imposed to demonstrate the system that follows a unique variation of parameter formula using Leray Schauder Alternative. Subsequently, the existence of optimal control is investigated for the considered Lagrange control problem. The theoretical example with the mechanical example of ethanol fuelled engine are discussed to validate the results obtained. [ABSTRACT FROM AUTHOR]

This paper presents a novel optimization model that explores the optimal investment strategies for DC pension plans with return of premium clauses. We have assumed that the financial market consists of a risk-free asset and a risky asset, where the price of the risky asset follows the CEV model. Under the expected utility criterion, the optimal investment strategies were derived by employing stochastic optimal control theory and the Legendre transformation method. Explicit expressions of the optimal investment strategy were provided when the utility function was specified as exponential, power, or logarithmic. Finally, numerical analysis was conducted to examine the impact of factors such as interest rate, return rate, and volatility of the risky asset on the optimal strategies. [ABSTRACT FROM AUTHOR]

The fashion apparel industry is facing an increasingly growing demand, compounded by the short sales lifecycle and strong seasonality of clothing, posing significant challenges to inventory management in the retail sector. Despite some retailers like Uniqlo and Zara implementing inventory management and dynamic pricing strategies, challenges persist due to the dynamic nature of fashion trends and the stochastic factors affecting inventory. To address these issues, we construct a mathematical model based on the mathematical expression of the deterministic fashion level function, where the geometric Brownian motion, widely applied in finance, is initially utilized in the stochastic fashion level function. Drawing on research findings from deteriorating inventory management and stochastic optimization, we investigate the fluctuation of inventory levels, optimal dynamic pricing, optimal production rates, and profits—four crucial indicators—via Pontryagin's maximum principle. Analytical solutions are derived, and the numerical simulation is provided to verify and compare the proposed model with deterministic fashion level function models. The model emphasizes the importance of considering stochastic factors in decision-making processes and provides insights to enhance profitability, inventory management, and sustainable consumption in the fashion product industry. [ABSTRACT FROM AUTHOR]

Broadcast control (BC) is a bio-inspired coordination technique for a swarm of agents in which a single coordinator broadcasts an identical scalar signal to all performing agents without discrimination, and the agents make appropriate moves towards the agents' collective optimal state without communicating with one another. The BC technique aims to accomplish a globally assigned task for which BC utilizes a stochastic optimization algorithm to coordinate a group of agents. However, the challenge intensifies as the system becomes larger: it requires a larger number of agents, which protracts the converging time for a single coordinator-based BC model. This paper proposes a revamped version of BC model, which assimilates distributed multiple coordinators to control a larger multi-agent system efficiently in a pragmatic manner. Precisely, in this hierarchical BC scheme, the distributed multiple sub-coordinators broadcast the identical feedback signal to the agents, which they receive from the global coordinator to accomplish the coverage control task of the ordinary agents. The dual role of sub-coordinators is manipulated by introducing weighted averaging of the gradient estimation under the stochastic optimization mechanism. The potency of the proposed model is analyzed with numerical simulation for a coverage control task, and various performance aspects are compared with the typical BC schemes to demonstrate its practicability and performance improvement. Particularly, the proposed scheme shows the same convergence with about 30% less traveling costs, and the near convergence is reached by only about one-third of iteration steps compared to the typical BC. [ABSTRACT FROM AUTHOR]

We propose logarithmic-barrier decomposition-based interior-point algorithms for solving two-stage stochastic quadratically constrained convex quadratic programming problems in a Hilbert space. We prove the polynomial complexity of the proposed algorithms, and show that this complexity is independent on the choice of the Hilbert space, and hence it coincides with the best-known complexity estimates in the finite-dimensional case. We also apply our results on a concrete example from the stochastic control theory. [ABSTRACT FROM AUTHOR]

Although adaptive cancer therapy shows promise in integrating evolutionary dynamics into treatment scheduling, the stochastic nature of cancer evolution has seldom been taken into account. Various sources of random perturbations can impact the evolution of heterogeneous tumors, making performance metrics of any treatment policy random as well. In this paper, we propose an efficient method for selecting optimal adaptive treatment policies under randomly evolving tumor dynamics. The goal is to improve the cumulative "cost" of treatment, a combination of the total amount of drugs used and the total treatment time. As this cost also becomes random in any stochastic setting, we maximize the probability of reaching the treatment goals (tumor stabilization or eradication) without exceeding a pre-specified cost threshold (or a "budget"). We use a novel Stochastic Optimal Control formulation and Dynamic Programming to find such "threshold-aware" optimal treatment policies. Our approach enables an efficient algorithm to compute these policies for a range of threshold values simultaneously. Compared to treatment plans shown to be optimal in a deterministic setting, the new "threshold-aware" policies significantly improve the chances of the therapy succeeding under the budget, which is correlated with a lower general drug usage. We illustrate this method using two specific examples, but our approach is far more general and provides a new tool for optimizing adaptive therapies based on a broad range of stochastic cancer models. Author summary: Tumor heterogeneities provide an opportunity to improve therapies by leveraging complex (often competitive) interactions of different types of cancer cells. These interactions are usually stochastic due to both individual cell differences and random events affecting the patient as a whole. The new generation of cancer models strive to account for this inherent stochasticity, and adaptive treatment plans need to reflect it as well. In optimizing such treatment, the most common approach is to maximize the probability of eventually stabilizing or eradicating the tumor. In this paper, we consider a more nuanced version of success, maximizing the probability of reaching these therapy goals before the cumulative burden from the disease and treatment exceed a chosen threshold. Importantly, our method allows computing such optimal treatment plans efficiently and for a range of thresholds at once. If used on a high-fidelity personalized model, our general approach could potentially be used by clinicians to choose the most suitable threshold after a detailed discussion of a specific patient's goals (e.g., to include the trade-offs between toxicity and quality of life). [ABSTRACT FROM AUTHOR]

This paper proposes an innovative approach for constrained rendezvous trajectory optimization (CRTO) using stochastic optimal feedback control and unscented transform (UT) uncertainty quantification. The method overcomes limitations of traditional deterministic CRTO solutions that suffer from uncontrolled terminal state error growth and severely deteriorated path constraint satisfaction when initial state uncertainty, dynamics uncertainty, and navigation uncertainty are considered. The approach involves constructing a stochastic optimal feedback control (SOFC) problem with chance constraints and introducing linear feedback control to regulate both the mean and variance of terminal state errors. UT is employed to approximately quantify the state errors and their propagation, transforming the SOFC problem into an unconstrained deterministic optimization (UDO) problem. Differential dynamic programming (DDP) is then used to solve the UDO problem. The obtained stochastic optimal solution provides a robust rendezvous trajectory and a corresponding explicit closed-loop guidance law, improving terminal accuracy and path constraint satisfaction in the presence of various considered uncertainties. The influence of different term weights in the objective function on terminal accuracy and path constraint satisfaction is also studied. The effectiveness of the proposed approach is verified through Monte Carlo simulation, demonstrating the robustness of the closed-loop control strategy. The results highlight the potential of the method in enhancing terminal accuracy and path constraint satisfaction in uncertain rendezvous scenarios. • The uncertainty-resilient rendezvous trajectory optimization algorithm provides not only a nominal trajectory but an explicit optimal state-feedback control policy. • Rendezvous trajectory uncertainty caused by initial state errors, dynamics errors, and navigation errors is propagated using an unscented transformation method. • The proposed method considers common input constraints and path constraints in space rendezvous missions. [ABSTRACT FROM AUTHOR]

A mean-field linear quadratic stochastic (MF-SLQ for short) optimal control problem with hybrid disturbances and cross terms in a finite horizon is concerned. The state equation is a systems driven by the Wiener process and the Poisson random martingale measure disturbed by some stochastic perturbations. The cost functional is also disturbed, which means more general cases could be characterized, especially when extra environment perturbations exist. In this paper, the well-posedness result on the jump diffusion systems is obtained by the fixed point theorem and also the solvability of the MF-SLQ problem. Actually, by virtue of adjoint variables, classic variational calculus, and some dual representation, an optimal condition is derived. Throughout our research, in order to connect the optimal control and the state directly, two Riccati differential equations, a BSDE with random jumps and an ordinary equation (ODE for short) on disturbance terms are obtained by a decoupling technique, which provide an optimal feedback regulator. Meanwhile, the relationship between the two Riccati equations and the so-called mean-field stochastic Hamilton system is established. Consequently, the optimal value is characterized by the initial state, disturbances, and original value of the Riccati equations. Finally, an example is provided to illustrate our theoretic results. [ABSTRACT FROM AUTHOR]

Topology optimization is an appealing technique for optimal layout of active controllers of structures. However, the existing methods are mainly restricted to deterministic excitations, and the optimal control law involved is determined by use of the classical optimal control (COC) method, in which the sensitivities of the gain matrix with respect to the design variables need to be determined by solving the Riccati sensitivity equation numerically. In this study, a reliability-based topology optimization framework is proposed for optimal layout of active controllers of structures under nonstationary random excitations. The optimization problem is formulated as the minimization of the failure probability of the structure subjected to a specified maximum number of controllers. To avoid solving the Riccati equation, an explicit optimal control (EOC) method is first employed to derive the closed-form optimal control law in terms of the position parameters of controllers. The statistical moments of the optimal control forces and structural responses under random excitations are then obtained explicitly by the operation rules of moments, and the first-passage dynamic reliability of the structure can be formulated using the level-crossing theory. On this basis, the sensitivities of the structural failure probability with respect to the position parameters of controllers can be derived analytically by the direct differential method. Finally, the explicit formulations of the response statistics and the relevant sensitivities are incorporated into a gradient-based method of moving asymptotes (MMA) for topology optimization of the layout of controllers in conjunction with the solid isotropic material with penalization (SIMP) technique. Two numerical examples are presented to demonstrate the feasibility of the proposed topology optimization framework. [ABSTRACT FROM AUTHOR]

This paper deals with a class of optimal control problems which arises in advertising models with Volterra Ornstein-Uhlenbeck process representing the product goodwill. Such choice of the model can be regarded as a stochastic modification of the classical Nerlove-Arrow model that allows to incorporate both presence of uncertainty and empirically observed memory effects such as carryover or distributed forgetting. We present an approach to solve such optimal control problems based on an infinite dimensional lift which allows us to recover Markov properties by formulating an optimization problem equivalent to the original one in a Hilbert space. Such technique, however, requires the Volterra kernel from the forward equation to have a representation of a particular form that may be challenging to obtain in practice. We overcome this issue for Hölder continuous kernels by approximating them with Bernstein polynomials, which turn out to enjoy a simple representation of the required type. Then we solve the optimal control problem for the forward process with approximated kernel instead of the original one and study convergence. The approach is illustrated with simulations. [ABSTRACT FROM AUTHOR]

Information dissemination has a significant impact on social development. This paper considers that there are many stochastic factors in the social system, which will result in the phenomena of information cross-dissemination and variation. The dual-system stochastic susceptible-infectious-mutant-recovered model of information cross-dissemination and variation is derived from this problem. Afterward, the existence of the global positive solution is demonstrated, sufficient conditions for the disappearance of information and its stationary distribution are calculated, and the optimal control strategy for the stochastic model is proposed. The numerical simulation supports the results of the theoretical analysis and is compared to the parameter variation of the deterministic model. The results demonstrate that cross-dissemination of information can result in information variation and diffusion. Meanwhile, white noise has a positive effect on information dissemination, which can be improved by adjusting the perturbation parameters. [ABSTRACT FROM AUTHOR]

We present a two-country model (North and South) that describes the phenomenon of offshoring and reshoring. The model is a continuous time-controlled Markov chain with binary states. The main trade-off involves production costs and transaction costs between one country and another. In the first part of this paper, we identify the key parameters of the model: the difference in unit production costs between the two countries considered, the marginal cost of transitioning between countries, and the incentive paid by the North country to all companies that have not relocated at the end of the planning interval. The final goal of our paper is to understand how national tax incentives can influence this process. [ABSTRACT FROM AUTHOR]

In this paper we explore optimal liquidation in a market populated by a number of heterogeneous market makers that have limited inventory-carrying and risk-bearing capacity. We derive a reduced form model for the dynamics of their aggregated inventory considering a proper scaling limit. The resulting price impact profile is shown to depend on the characteristics and relative importance of their inventories. The model is flexible enough to reproduce the empirically documented power law behavior of the price impact function. For any choice of the market makers characteristics, optimal execution within this modeling approach can be recast as a linear-quadratic stochastic control problem. The value function and the associated optimal trading rate can be obtained semi-explicitly subject to solving a differential matrix Riccati equation. Numerical simulations are conducted to illustrate the performance of the resulting optimal liquidation strategy in relation to standard benchmarks. Remarkably, they show that the increase in performance is determined by a substantial reduction of higher order moment risk. [ABSTRACT FROM AUTHOR]

David A. Kendrick, a respected scholar and co-founder of the Journal of Computational Economics, has passed away at the age of 86. Kendrick began his career in politics but later pursued science, earning a Ph.D. from MIT and becoming a professor at Harvard and the University of Texas. He made significant contributions to computational economics, particularly in stochastic control theory and the development of GAMS software. Kendrick's legacy includes numerous books and articles, as well as the establishment of the David A. Kendrick Distinguished Service Award. He will be remembered for his kindness, generosity, and dedication to his students and colleagues. The document provided is a list of publications by Hans Amman, Ruben Mercado, and Berç Rustem. These publications cover a range of topics in economics, such as stochastic control, macroeconomic models, computational economics, and optimal control models. The authors discuss various methods and approaches used in economic analysis and offer insights into the challenges and advancements in the field. The publications span from 1981 to 2020, providing a comprehensive overview of the authors' contributions to economics. [Extracted from the article]

We study the approximate optimal control for a class of fractional stochastic hemivariational inequalities with non-instantaneous impulses driven by Rosenblatt process in a Hilbert space. Firstly, a suitable definition of piecewise continuous mild solution is introduced, and by using stochastic analysis, properties of α -order sine and cosine family and Picard type approximate sequences, we show the existence and uniqueness of approximate mild solutions for the inequality problems of fractional order (1, 2] under the non-Lipschitz conditions. Secondly, we provide the existence conditions of approximate solutions to optimal control problems driven by the presented control systems with the help of a new minimizing sequence method. Finally, an example is provided to illustrate the theory. [ABSTRACT FROM AUTHOR]

This paper is concerned with the stochastic optimal control problem of a 1-dimensional McKean–Vlasov stochastic differential equation (SDE) with reflection, of which the drift coefficient and diffusion coefficient can be both dependent on the state of the solution process along with its law and control. One backward stochastic partial differential equation (BSPDE) with the Neumann boundary condition can represent the value function of this control problem. Existence and uniqueness of the solution to the above equation are obtained. Finally, the optimal feedback control can be constructed by the BSPDE. [ABSTRACT FROM AUTHOR]

The stochastic fluctuation of information induces the dynamic behavior and disease control strategy to change in the HBV epidemic model, but how stochastic information dissemination affects them is vague. Here, we consider an HBV epidemic model with stochastic information intervention. Using the Lyapunov function, we demonstrate that the system has a unique global positive solution, in addition, we also derive the threshold dynamics to obtain sufficient conditions that can ensure the extinction and persistence, as well as stationarity of the system. Moreover, we qualify the impact of stochasticity of the information on the optimal control strategy of the HBV, indicating that while information intervention could effectively reduce the disease peak, the fluctuation will attenuate this effect, i.e., the stable information is better than the noisy one. Finally, various stochastic and optimal control simulations are performed to verify the theoretical results and verify that optimal control can accelerate the extinction of the disease. [ABSTRACT FROM AUTHOR]

Over the years, the research of backward stochastic differential equations (BSDEs) has come a long way. As a extension of the BSDEs, the BSDEs with time delay have played a major role in the stochastic optimal control, financial risk, insurance management, pricing, and hedging. In this paper, we study a class of BSDEs with time-delay generators driven by Caputo fractional derivatives. In contrast to conventional BSDEs, in this class of equations, the generator is also affected by the past values of solutions. Under the Lipschitz condition and some new assumptions, we present a theorem on the existence and uniqueness of solutions. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive framework addressing optimal nonlinear analysis and feedback control synthesis for nonlinear stochastic dynamical systems. The focus lies on establishing connections between stochastic Lyapunov theory and stochastic Hamilton–Jacobi–Bellman theory within a unified perspective. We demonstrate that the closed-loop nonlinear system's asymptotic stability in probability is ensured through a Lyapunov function, identified as the solution to the steady-state form of the stochastic Hamilton–Jacobi–Bellman equation. This dual assurance guarantees both stochastic stability and optimality. Additionally, optimal feedback controllers for affine nonlinear systems are developed using an inverse optimality framework tailored to the stochastic stabilization problem. Furthermore, the paper derives stability margins for optimal and inverse optimal stochastic feedback regulators. Gain, sector, and disk margin guarantees are established for nonlinear stochastic dynamical systems controlled by nonlinear optimal and inverse optimal Hamilton–Jacobi–Bellman controllers. [ABSTRACT FROM AUTHOR]

We study an optimal investment and consumption problem with heterogeneous consumption of basic and luxury goods, together with the choice of time for retirement. The optimal heterogeneous consumption strategies for a class of nonhomothetic utility maximizer are shown to consume only basic goods when the wealth is small, to consume basic goods and make savings when the wealth is intermediate, and to consume almost all in luxury goods when the wealth is large. The optimal retirement policy is shown to be both universal, in the sense that all individuals should retire at the same level of marginal utility that is determined only by income, labor cost, discount factor as well as market parameters, and not universal, in the sense that all individuals can achieve the same marginal utility with different utility and wealth. We also show that individuals prefer to retire as time goes by if the marginal labor cost increases faster than that of income. This paper studies an optimal investment and consumption problem with heterogeneous consumption of basic and luxury goods, together with the choice of time for retirement. The utility for luxury goods is not necessarily a concave function. The optimal heterogeneous consumption strategies for a class of nonhomothetic utility maximizer are shown to consume only basic goods when the wealth is small, to consume basic goods and make savings when the wealth is intermediate, and to consume almost all in luxury goods when the wealth is large. The optimal retirement policy is shown to be both universal, in the sense that all individuals should retire at the same level of marginal utility that is determined only by income, labor cost, discount factor and market parameters, and not universal, in the sense that all individuals can achieve the same marginal utility with different utility and wealth. It is also shown that individuals prefer to retire as time goes by if the marginal labor cost increases faster than that of income. The main tools used in analyzing the problem are from a partial differential equation and stochastic control theory including variational inequality and dual transformation. We finally conduct the simulation analysis for the featured model parameters to investigate practical and economic implications by providing their figures. Funding: This work was supported by Hong Kong Research Grants Council General Research Fund [Grants 15202421 and 15202817], the National Research Foundation of Korea [Grant 2021R1C1C1004647], the PolyU-SDU Joint Research Center on Financial Mathematics, the CAS AMSS-PolyU Joint Laboratory of Applied Mathematics, and Hong Kong Polytechnic University, the National Natural Science Foundation of China [Grant 11971409], and the Engineering and Physical Sciences Research Council (UK) [Grant EP/V008331/1]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/opre.2022.2328. [ABSTRACT FROM AUTHOR]

Defined benefit (DB) pension plans are a primary type of pension schemes with the sponsor assuming most of the risks. Longevity-indexed bonds have been used to hedge or transfer risks in pension plans. Our objective is to study an aggregated DB pension plan's optimal risk management problem focusing on minimizing the solvency risk over a finite time horizon and to investigate the investment strategies in a market, comprising a longevity-indexed bond and a risk-free asset, under stochastic nominal interest rates. Using the dynamic programming technique in the stochastic control problem, we obtain the closed-form optimal investment strategy by solving the corresponding Hamilton–Jacobi–Bellman (HJB) equation. In addition, a comparative analysis implicates that longevity-indexed bonds significantly reduce solvency risk compared to zero-coupon bonds, offering a strategic advantage in pension fund management. Besides the closed-form solution and the comparative study, another novelty of this study is the extension of actuarial liability (AL) and normal cost (NC) definitions, and we introduce the risk neutral valuation of liabilities in DB pension scheme with the consideration of mortality rate. [ABSTRACT FROM AUTHOR]

A stochastic discrete fractional Cournot duopoly game model with a unique interior Nash equilibrium is developed in this study. Some sufficient criteria of the Lyapunov stability in probability for the proposed model at the interior Nash equilibrium are derived using the Lyapunov theory. The proposed model's finite time stability in probability is then investigated using a nonlinear feedback control approach at the interior Nash equilibrium. The stochastic Bellman theory is also used to explore the locally optimum control problem. Furthermore, bifurcation diagrams, time series, and the 0-1 test are used to investigate the chaotic dynamics of this model. Finally, numerical examples are given to illustrate the obtained results. [ABSTRACT FROM AUTHOR]

This paper focuses on the application of meta-reinforcement learning to the robust design of low-thrust interplanetary trajectories in the presence of multiple uncertainties. A closed-loop control policy is used to optimally steer the spacecraft to a final target state despite the considered perturbations. The control policy is approximated by a deep recurrent neural network, trained by policy-gradient reinforcement learning on a collection of environments featuring mixed sources of uncertainty, namely dynamic uncertainty and control execution errors. The recurrent network is able to build an internal representation of the distribution of environments, thus better adapting the control to the different stochastic scenarios. The results in terms of optimality, constraint handling, and robustness on a fuel-optimal low-thrust transfer between Earth and Mars are compared with those obtained via a traditional reinforcement learning approach based on a feed-forward neural network. • In interplanetary space missions, the spacecraft trajectory is affected by multiple uncertainties. • Meta-reinforcement learning can be applied seamlessly to any uncertainty and dynamic model. • A recurrent neural network is used as a history-dependent closed-loop control policy. • The network is trained on a low-thrust transfer featuring dynamic uncertainties and control execution errors. • Meta-reinforcement learning shows improved performance and robustness compared to standard reinforcement learning. [ABSTRACT FROM AUTHOR]

Releasing Wolbachia-infected mosquitoes into the wild to suppress wild mosquito populations is an effective method for mosquito control. This paper investigates the finite-time contraction stability and optimal control problem of a mosquito population suppression model with different release strategies. By taking into account the average duration of one reproductive cycle and the influences of environmental fluctuations on mosquitoes, we consider two cases: one with a time delay and another perturbed by stochastic noises. By employing Lyapunov's method and comparison theorem, the finite-time contraction stabilities of these two cases under a constant release strategy are analyzed. Sufficient conditions dependent on delay and noise for these two systems are provided, respectively. These conditions are related to the prespecified bounds in finite-time stability (FTS) and finite-time contraction stability (FTCS) of the system, and FTCS required stronger conditions than FTS. This also suggests that the specified bounds and the delay (or the noise intensity) play a critical role in the FTCS analysis. And finally, the optimal control for the stochastic mosquito population model under proportional releases is researched. [ABSTRACT FROM AUTHOR]

The problem of decentralized observer-based event-triggered stabilization for an interconnected fractional-order system subject to stochastic cyber-attacks is studied. To address this issue, the decentralized event-triggered mechanism is proposed for the interconnected fractional-order system, where the event-triggered schemes are designed based on the states of fractional-order observers, and the stochastic attacks are considered both on control inputs and observer outputs. By combining decentralized observers and decentralized event-triggered controllers, we aim to achieve decentralized control with reduced amplifying error and use local signals to improve overall system performance. By utilizing the diffusive representation of the fractional-order system, the interconnected fractional-order system is transformed into an equivalent integer-order one to simplify the analysis and control design. Employing the Lyapunov indirect approach, a sufficient condition is obtained to guarantee the stochastic asymptotically stability of the augmented system. Additionally, by the singular value decomposition technique, the approach of simultaneously computing the decentralized observer gains and controller gains is presented. Finally, a simulation example is provided to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

Safety-critical and mission-critical systems are often sensitive to functional degradation at the system or component level. Such degradation dynamics are often dependent on system usage (or control input), and may lead to significant losses and a potential system failure. Therefore, it becomes imperative to develop control designs that are able to ensure system stability and performance whilst mitigating the effects of incipient degradation by modulating the control input appropriately. In this context, this paper proposes a novel approach based on an optimal control theory framework wherein the degradation state of the system is considered in the augmented system model and estimated using sensor measurements. Further, it is incorporated within the optimal control paradigm leading to a control law that results in deceleration of the degradation rate at the cost of system performance whilst ensuring system stability. To that end, the speed of degradation and the state of the system in discrete time are considered to develop a linear quadratic tracker (LQT) and regulator (LQR) over a finite horizon in a mathematically rigorous manner. Simulation studies are performed to assess the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, a computational method based on parameterizing state and control variables is presented for solving Stochastic Optimal Control (SOC) problems. By using Chebyshev wavelets with unknown coefficients, state and control variables are parameterized, and then a stochastic optimal control problem is converted to a stochastic optimization problem. The expected cost functional of the resulting stochastic optimization problem is approximated by sample average approximation thereby the problem can be solved by optimization methods more easily. For facilitating and guaranteeing convergence of the presented method, a new theorem is proved. Finally, the proposed method is implemented based on a newly designed algorithm for solving one of the well-known problems in mathematical finance, the Merton portfolio allocation problem in finite horizon. The simulation results illustrate the improvement of the constructed portfolio return. [ABSTRACT FROM AUTHOR]

Investor sentiment contagion has a profound influence on economic and social development. This paper explores the diverse influences of various investor sentiments in modern society on the economy and society. It also investigates the interference of various uncertain factors on investor sentiments in the modern economy and society. On this basis, the dual-system stochastic SPA2G2R model was constructed, incorporating positive and negative sentiments, as well as a supervision and isolation mechanism. The global existence of positive solutions was established, and sufficient conditions for the disappearance and steady distribution of investor sentiment were calculated. An optimal control strategy for the stochastic model was put forward, with numerical simulation supporting the theoretical analysis results. A comparison with parameter changes in the deterministic model was also conducted. The research reveals a competitive relationship between different investor sentiments. Enhancing societal guidance mechanisms promotes positive investor sentiment contagion. Timely control by the supervisory department effectively curbs the spread of investor sentiment. Additionally, white noise promotes investor sentiment contagion, suggesting effective regulation through control of noise intensity and disturbance parameters. [ABSTRACT FROM AUTHOR]

We compute the social cost of carbon (SCC) when decision makers want robust estimates in the face of deep (or "Knightian") uncertainty. We introduce the notion of full-path accumulated robust preferences from stochastic control theory to an integrated assessment model. Robust preferences are appropriate for analyzing climate-related problems because, given the large uncertainty in climate science, they enable decision makers to attain solutions that are robust to a wide range of climate change scenarios. We solve the resulting model, which includes uncertainty about climate change and the ensuing economic damage and show the existence of optimal solutions and time-consistent optimal deterministic Markov policies. Additionally, we also prove that the standard Hansen–Sargent recursive utility provides an upper bound of this full-path utility. In our baseline model specification, we find that the year 2020's optimal SCC is US$162 per tCO2 with an average annual growth rate of 2.5%, setting the world on a 1.37°C path, which requires full decarbonization by 2068. We introduce the notion of the SCC robustness premium, which we define as the additional SCC price tag for robustness. For a plausible range of preference parameters, the SCC robustness premium in 2020 is between US$1.41 and US$25.89 per tCO2 with US$2.20 per tCO2 in our baseline calibration. Over time, this premium grows significantly. The forecasts of our model facilitate managerial decision making during the world's transition from a carbon- and emission-intensive economy to a regenerative economy. The high estimates for the SCC predict drastic rises in emission costs for high-emission industries. This paper was accepted by Elke Weber, Special Section of Management Science on Business and Climate Change. Funding: Y. Zhao and T. S. Lontzek acknowledge funding by the German Ministry of Research and Education (BMBF), Project CDR: "DAC-Tales", under [Grant 01LS2196A]. Supplemental Material: The data files are available at https://doi.org/10.1287/mnsc.2023.4736. [ABSTRACT FROM AUTHOR]

The problem of finding the optimal open-loop control for discrete-time stochastic dynamical systems is considered. It is assumed that the initial conditions and external influences are random. The average value of the Bolza functional defined on individual trajectories is minimized. It is proposed to solve the problem by means of classical and modified migrating optimization algorithms. The modification of the migrating algorithm consists of cloning the members of the initial population and choosing different strategies of migratory behavior for the main population and for populations formed by clones. At the final stage of the search for an extremum, an intensively clarifying migration cycle is implemented with the participation of three leaders of the populations participating in the search process. Problems of optimal control of bundles of trajectories of deterministic discrete dynamical systems, as well as individual trajectories, are considered as special cases. Seven model examples illustrating the performance of the proposed approach are solved. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the optimal control problems for a class of neutral stochastic integrodifferential equations (NSIDEs) with infinite delay driven by Poisson jumps and the Rosenblat process in Hilbert space involving concrete-fading memory-phase space, in which we define the advanced phase space for infinite delay for the stochastic process. First, we introduce conditions that ensure the existence and uniqueness of mild solutions using stochastic analysis theory, successive approximation, and Grimmer's resolvent operator theory. Next, we prove exponential stability, which includes mean square exponential stability, and this especially includes the exponential stability of solutions and their maps. Following that, we discuss the existence requirements of an optimal pair of systems governed by stochastic partial integrodifferential equations with infinite delay. Then, we explore examples that illustrate the potential of the main result, mainly in the heat equation, filter system, traffic signal light systems, and the biological processes in the human body. We conclude with a numerical simulation of the system studied. This work is a unique combination of the theory with practical examples and a numerical simulation. [ABSTRACT FROM AUTHOR]

This paper addresses the synchronization problem of delayed stochastic neural networks with discontinuous activation functions (DSNNsDF), specifically focusing on fixed/preassigned-time synchronization. The objective is to develop a class of simplified controllers capable of effectively addressing the challenges posed by time delays, discontinuous activation functions, and stochastic perturbations during the synchronization process. In this regard, we propose several controllers with simpler structures to achieve the desired preassigned-time synchronization (PTS) result. To enhance the accuracy of time estimation, stochastic fixed-time control theory is employed. Rigorous numerical simulations are conducted to validate the effectiveness of our approach. The utilization of our proposed results significantly improves the performance of the synchronization controller for DSNNsDF, thereby enabling advancements and diverse applications in the field. [ABSTRACT FROM AUTHOR]

In the theory of portfolio selection, there are few methods that effectively address the combined challenge of insider information and model uncertainty, despite numerous methods proposed for each individually. This paper studies the problem of the robust optimal investment for an insider under model uncertainty. To address this, we extend the Itô formula for forward integrals by Malliavin calculus, and use it to establish an implicit anticipating stochastic differential game model for the robust optimal investment. Since traditional stochastic control theory proves inadequate for solving anticipating control problems, we introduce a new approach. First, we employ the variational method to convert the original problem into a nonanticipative stochastic differential game problem. Then we use the stochastic maximum principle to derive the Hamiltonian system governing the robust optimal investment. In cases where the insider information filtration is of the initial enlargement type, we derive the closed-form expression for the investment by using the white noise theory when the insider is 'small'. When the insider is 'large', we articulate a quadratic backward stochastic differential equation characterization of the investment. We present the numerical result and conduct an economic analysis of the optimal strategy across various scenarios. [ABSTRACT FROM AUTHOR]

Consciousness has been described as acting as a global workspace that integrates perception, imagination, emotion and action programming for adaptive decision making. The mechanisms of this workspace and their relationships to the phenomenology of consciousness need to be further specified. Much research in this area has focused on the neural correlates of consciousness, but, arguably, computational modeling can better be used toward this aim. According to the Projective Consciousness Model (PCM), consciousness is structured as a viewpoint-organized, internal space, relying on 3D projective geometry and governed by the action of the Projective Group as part of a process of active inference. The geometry induces a group-structured subjective perspective on an encoded world model, enabling adaptive perspective taking in agents. Here, we review and discuss the PCM. We emphasize the role of projective mechanisms in perception and the appraisal of affective and epistemic values as tied to the motivation of action, under an optimization process of Free Energy minimization, or more generally stochastic optimal control. We discuss how these mechanisms enable us to model and simulate group-structured drives in the context of social cognition and to understand the mechanisms underpinning empathy, emotion expression and regulation, and approach–avoidance behaviors. We review previous results, drawing on applications in robotics and virtual humans. We briefly discuss future axes of research relating to applications of the model to simulation- and model-based behavioral science, geometrically structured artificial neural networks, the relevance of the approach for explainable AI and human–machine interactions, and the study of the neural correlates of consciousness. [ABSTRACT FROM AUTHOR]

A household consumption and optimal portfolio problem pertinent to life insurance (LI) in a continuous time setting is examined. The family receives a random income before the parents' retirement date. The price of the risky asset is driven by the exponential Ornstein–Uhlenbeck (O-U) process, which can better reflect the state of the financial market. If the parents pass away prior to their retirement time, the children do not have any work income and LI can be purchased to hedge the loss of wealth due to the parents' accidental death. Meanwhile, utility functions (UFs) of the parents and children are individually taken into account in relation to the uncertain lifetime. The purpose of the family is to appropriately maximize the weighted average of the corresponding utilities of the parents and children. The optimal strategies of the problem are achieved using a dynamic programming approach to solve the associated Hamilton–Jacobi–Bellman (HJB) equation by employing the convex dual theory and Legendre transform (LT). Finally, we aim to examine how variations in the weight of the parents' UF and the coefficient of risk aversion affect the optimal policies. [ABSTRACT FROM AUTHOR]

A boundary value problem for a non-linear difference equation of order three is considered. We show that this equation can be interpreted as the equation satisfied by the value function in a stochastic optimal control problem. We thus obtain an expression for the solution of the non-linear difference equation that can be used to find an explicit solution to this equation. An example is presented. [ABSTRACT FROM AUTHOR]