Search

The committor constitutes the primary quantity of interest within chemical kinetics as it is understood to encode the ideal reaction coordinate for a rare reactive event. We show the generative utility of the committor in that it can be used explicitly to produce a reactive trajectory ensemble that exhibits numerically exact statistics as that of the original transition path ensemble. This is done by relating a time-dependent analog of the committor that solves a generalized bridge problem to the splitting probability that solves a boundary value problem under a bistable assumption. By invoking stochastic optimal control and spectral theory, we derive a general form for the optimal controller of a bridge process that connects two metastable states expressed in terms of the splitting probability. This formalism offers an alternative perspective into the role of the committor and its gradients in that they encode force fields that guarantee reactivity, generating trajectories that are statistically identical to the way that a system would react autonomously. [ABSTRACT FROM AUTHOR]

In practice, there is always competition among different pension managers. The competitive pension managers are concerned about their relative performance to make better decisions. In addition, to protect the rights of plan members who die before retirement, most of the defined contribution (DC) pension plans contain return of premiums clauses. In this article, we introduce the return of premiums clauses into two competitive DC pension plans with salary risk. Each pension manager cares about not only his own terminal wealth but also his relative wealth versus his competitor. Both the pension managers are allowed to invest the premiums in a financial market, which consists of one risk-free asset and one risky asset whose price process follows the constant elasticity of variance (CEV) model. We investigate the time-consistent investment strategies of the two pension managers under the mean-variance (MV) criterion. The aim of each pension manager is to maximize the mean and minimize the variance of his relative terminal wealth. The explicit expressions of the time-consistent strategies and the efficient frontiers are obtained via applying the extended stochastic optimal control theory. Some special cases are also discussed in detail. Finally, a numerical simulation demonstrates the results obtained. [ABSTRACT FROM AUTHOR]

In this paper, we study a kind of optimal control problem for forward-backward stochastic differential equations (FBSDEs for short) of McKean–Vlasov type via the dynamic programming principle (DPP for short) motivated by studying the infinite dimensional Hamilton–Jacobi–Bellman (HJB for short) equation derived from the decoupling field of the FBSDEs posed by Carmona and Delarue (2015) [28]. At the beginning, the value function is defined by the solution to the controlled BSDE alluded to earlier. On one hand, we can prove the value function is deterministic function with respect to the initial random variable; On the other hand, we can show that the value function is law-invariant , i.e., depending on only via its distribution by virtue of BSDE property. Afterward, utilizing the notion of differentiability with respect to probability measures introduced by P.L. Lions [50] , we are able to establish a DPP for the value function in the Wasserstein space of probability measures based on the application of BSDE approach, particularly, employing the notion of stochastic backward semigroups associated with stochastic optimal control problems and Itô formula along a flow property of the conditional law of the controlled forward state process. We prove that the value function is the unique viscosity solutions of the associated generalized HJB equations in some separable Hilbert space. Finally, as an application, we formulate an optimal control problem for linear stochastic differential equations with quadratic cost functionals of McKean-Vlasov type under nonlinear expectation, g -expectation introduced by Peng [43] and derive the optimal feedback control explicitly by means of several groups of Riccati equations. [ABSTRACT FROM AUTHOR]

A feedback control method is proposed to asymptotically stabilize the chain-like multi-degree-of-freedom (MDOF) nonlinear structural system under purely parametric Gaussian white noises. First, the motion equation of a chain-like nonlinear structural system with n coupled elements and an undetermined control law is derived. It is reduced to a one-dimensional partially averaged Itô stochastic differential equation of the controlled Hamiltonian by applying the stochastic averaging method, which bypasses the challenge of calculating multiple Lyapunov exponents. Next, based on the dynamical programming principle, the dynamical programming equation of the averaged system with an undetermined cost function is established and addressed to obtain the optimal control law. Then, the Lyapunov exponent of the completely averaged Itô equation is derived. For the given requirement of stabilizing the system, the Lyapunov exponent is entirely fixed and used to analyze the stability of the originally chain-like MDOF nonlinear structural system. A controlled chain-like 8-DOF nonlinear structural system is taken as an example to illustrate the application and effectiveness of the proposed procedure and the effect of control forces on stabilizing the system. [ABSTRACT FROM AUTHOR]

In this article, we study a stochastic optimal control problem in the pathwise sense, as initially proposed by Lions and Souganidis in [C. R. Acad. Sci. Paris Ser. I Math., 327 (1998), pp. 735-741]. The corresponding Hamilton-Jacobi-Bellman (HJB) equation, which turns out to be a non-adapted stochastic partial differential equation, is analyzed. Making use of the viscosity solution framework, we show that the value function of the optimal control problem is the unique solution of the HJB equation. When the optimal drift is defined, we provide its characterization. Finally, we describe the associated conserved quantities, namely the space-time transformations leaving our pathwise action invariant. [ABSTRACT FROM AUTHOR]

We study a stochastic velocity tracking problem for the 2D-Navier–Stokes equations perturbed by a multiplicative Gaussian noise. From a physical point of view, the control acts through a boundary injection/suction device with uncertainty, modeled by stochastic non-homogeneous Navier-slip boundary conditions. We show the existence and uniqueness of the solution to the state equation, and prove the existence of an optimal solution to the control problem. [ABSTRACT FROM AUTHOR]

We consider the stochastic control problem of the shallow lake and continue the work of Kossioris, Loulakis, and Souganidis. First, we generalise the characterisation of the value function as the viscosity solution of a well-posed problem to include more general recycling rates. Then, we prove approximate optimality under bounded controls and we establish quantitative estimates. Finally, we implement a convergent and stable numerical scheme for the computation of the value function to investigate properties of the optimally controlled stochastic shallow lake. This approach permits to derive tails asymptotics for the invariant distribution and to extend results of Grass, Kiseleva and Wagener. ] beyond the small noise limit. [ABSTRACT FROM AUTHOR]

The collocation method is one of the most powerful and effective techniques to solve nonlinear differential equations. This method reduces the original problem to a system of algebraic equations. Awareness of the nature of the problem and the analytical behavior of the solution has an important influence on the choice of type and number of basis functions required by the collocation method for solving the different classes of differential equations. In general, the polynomial basis functions such as the Legendre polynomials are not always an appropriate choice to approximate the functions that can be expanded based on monomial basis functions with real powers. In such cases, a combination of polynomial basis functions with some maps to replace the natural powers with arbitrary real powers can be a suitable alternative. This paper presents a collocation method based on the shifted fractional-order Legendre functions to solve the linear and nonlinear Hamilton–Jacobi–Bellman PDEs in stochastic optimal control problems. The convergence analysis of the proposed method has also been investigated. Moreover, the method is used to solve some practical problems with different solution structures such as resource extraction and Merton's portfolio selection models. Numerical results for four different examples indicated that the collocation method based on the fractional-order Legendre functions are provided. Due to the non differentiability of the solution of HJB PDEs at x = 0 for resource extraction and Merton's portfolio selection models, the Legendre collocation method cannot produce solutions with the desired accuracy. In fact, it is observed that the collocation method based on the fractional-order Legendre functions can be more efficient than the standard Legendre collocation method. Also, the sample paths of the control and state processes are obtained together with the estimate given by the Monte Carlo simulation. [ABSTRACT FROM AUTHOR]

In this paper, we propose a low rank approximation method for efficiently solving stochastic partial differential equations. Specifically, our method utilizes a novel low rank approximation of the stiffness matrices, which can significantly reduce the computational load and storage requirements associated with matrix inversion without losing accuracy. To demonstrate the versatility and applicability of our method, we apply it to address two crucial uncertainty quantification problems: stochastic elliptic equations and optimal control problems governed by stochastic elliptic PDE constraints. Based on varying dimension reduction ratios, our algorithm exhibits the capability to yield a high-precision numerical solution for stochastic partial differential equations, or provides a rough representation of the exact solutions as a pre-processing phase. Meanwhile, our algorithm for solving stochastic optimal control problems allows a diverse range of gradient-based unconstrained optimization methods, rendering it particularly appealing for computationally intensive large-scale problems. Numerical experiments are conducted and the results provide strong validation of the feasibility and effectiveness of our algorithm. [ABSTRACT FROM AUTHOR]

Optimal control theory is a branch of mathematics. It is developed to find optimal ways to control a dynamic system. In 1957, R.Bellman applied dynamic programming to solve optimal control of discrete-time systems. His procedure resulted in closed-loop, generally nonlinear, and feedback schemes. Optimal control problems which will be tackled involve the minimization of a cost function subject to constraints on the state vector and the control. Lagrange multipliers provide a method of converting a constrained minimization problem into an unconstrained minimization problem of higher order. The necessary condition for optimality can be obtained as the solution of the unconstrained optimization problem of the Lagrange function and the bordered Hessian matrix is used for the second-derivative test. A spectral method for solving optimal control problems is presented. The method is based on Bernoulli polynomials approximation. By using the Bernoulli operational matrix of integration and the Lagrangian function, stochastic optimal control is transformed into an optimisation problem, where the unknown Bernoulli coefficients are determined by using Newton’s iterative method. The convergence analysis of the proposed method is given. The simulation results based on the Monte-Carlo technique prove the performance of the proposed method. Some error estimations are provided and illustrative examples are also included to demonstrate the efficiency and applicability of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper we study optimal trading strategies in a financial market in which stock returns depend on a hidden Gaussian mean reverting drift process. Investors obtain information on that drift by observing stock returns. Moreover, expert opinions in the form of signals about the current state of the drift arriving at fixed and known dates are included in the analysis. Drift estimates are based on Kalman filter techniques. They are used to transform a power utility maximization problem under partial information into an optimization problem under full information where the state variable is the filter of the drift. The dynamic programming equation for this problem is studied and closed-form solutions for the value function and the optimal trading strategy of an investor are derived. They allow to quantify the monetary value of information delivered by the expert opinions. We illustrate our theoretical findings by results of extensive numerical experiments. [ABSTRACT FROM AUTHOR]

Uncrewed aircraft systems and advanced air mobility are associated with uncertain conditions, such as wind prediction errors, and they may deviate from dedicated airspaces referred to as corridors. However, further studies on these aspects are required. This study focuses on the covariance control for a stochastic discrete-time linear system under correlated uncertainties. A covariance control problem with chance constraints can be formulated as a convex-programming problem. Numerical simulations of uncrewed aircraft systems path planning can be used to compare two control laws: covariance control laws that consider correlated and noncorrelated noise. Numerical simulations were conducted to verify that the covariance control law considering correlated noise generates a trajectory that appropriately satisfies the constraints. Consequently, the effect of considering correlated noise in the covariance control law was clarified. In addition, covariance control has been demonstrated to generate appropriate trajectories for various state constraints. Moreover, numerical simulations indicated that covariance control facilitated the control of uncertainty, and the effectiveness of the covariance control was evaluated. The insights gained from the results of this study can inform enhanced uncertainty management for improving reliability and safety in real-world applications of uncrewed aircraft systems and advanced air mobility. [ABSTRACT FROM AUTHOR]

This article explores pure-endowment contracts with investments that are simultaneously allocated in risk-free and risky financial markets. Employing the optimal stochastic control method and assuming that the jumps of the risky financial market follow either a finite or infinite activity Lévy process, and that the policyholder's utility function is a Constant Relative Risk Aversion (CRRA) utility function, the article derives an optimal investment strategy and optimal policyholder consumption, with dependency on the mortality rate. Various mortality models and jump parameters are utilized to investigate the sensitivity of our findings. Finally, the article establishes the fair price of such contracts under different circumstances, showcasing practical applications through several examples. [ABSTRACT FROM AUTHOR]

This study refines the methodology for solving stochastic optimal control problems with quality criteria that include the sum of the quality functional of the classical formulation and an extremal measure. A two-level optimization solution of these kinds of problems is presented already for the case where the quality functional consists only of the extremal measure. Our study shows the possibility of solving the original time inconsistency problem through solving a two-level optimization problem, where the outer problem is solved by gradient methods since the value function is convex and the inner problem is solved by classical methods. Some experiments were carried out and confirmed the validity of the theory. The results of the study can be applied to the case of portfolio management with quality criteria containing the Conditional Value-at-Risk (CVaR) metric. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of optimal investment and proportional reinsurance in a dynamic asset model for an insurance company. The objective is to minimize the probability of drawdown, which is defined as the probability of an asset reaching a fixed proportion of its historical maximum. We assume that the debt of the company's wealth process is a deterministic function of its value. We discuss the problem separately according to the two cases that the historical maximum value is less than the safety value and greater than the safety value. By utilizing stochastic control theory, we derive the optimal investment and reinsurance strategy along with the corresponding minimum drawdown probability. We show that when the historical maximum is relatively low (below the safety level), raising the historical maximum to the safety level is the optimal strategy. Finally, we provide examples to illustrate the impact of model parameters on the optimal results. [ABSTRACT FROM AUTHOR]

We study optimal stochastic control problems for fully coupled forward-backward stochastic differential equations driven by anomalous subdiffusion, which have nontrivial mixed features of deterministic and stochastic controls. Both the stochastic maximum principle (SMP) and sufficient SMP are obtained by using a convex variational method. The paper ends with an application of the main results of this paper to a linear quadratic problem in the subdiffusive setting, which is solved explicitly. [ABSTRACT FROM AUTHOR]

We provide a new version of the Tikhonov theorem for both two-scale forward systems and also two-scale forward-backward systems of stochastic differential equations, which also covers the McKean--Vlasov case. Differently from what is usually done in the literature, we prove a type of convergence for the "fast" variable, which allows the limiting process to be discontinuous. This is relevant for the second part of the paper, where we present a new application of this theory to the approximation of the solution of mean field control problems. Towards this aim, we construct a two-scale system whose "fast" component converges to the optimal control process, while the "slow" component converges to the optimal state process. The interest in such a procedure is that it allows one to approximate the solution of the control problem, avoiding the usual step of the minimization of the Hamiltonian. [ABSTRACT FROM AUTHOR]

The authors give a stochastic maximum principle for square-integrable optimal control of linear stochastic systems. The control domain is not necessarily convex and the cost functional can have a quadratic growth. In particular, they give a stochastic maximum principle for the linear quadratic optimal control problem. [ABSTRACT FROM AUTHOR]

This paper focuses on addressing the linear quadratic (LQ) optimal control problem on an infinite time horizon for stochastic systems controlled by impulses. No constraint regarding the sign of the quadratic functional is applied. That is why our first concern is to find conditions which guarantee that the considered optimal control problem is well posed. Then, when the optimal control problem is well posed, it is natural to look for conditions which guarantee the attainability of the optimal control problem that is being evaluated. The main tool involved in the solution of the problems stated before is a backward jump matrix linear differential equation (BJMLDE) with a Riccati-type jumping operator. This is formulated using the matrix coefficients of the controlled system and the weight matrices of the performance criterion. We show that the well posedness of the optimal control problem under investigation is guaranteed by the existence of the maximal and bounded solution of the associated BJMLDE with a Riccati-type jumping operator. Further, we show that when the associated BJMLDE with a Riccati-type jumping operator has a maximal solution which satisfies a suitable sign condition, then the optimal control problem is attainable if and only if it has an optimal control in a state feedback form, or if and only if the maximal solution of the BJMLDE with a Riccati-type jumping operator is a stabilizing solution. In order to make the paper more self-contained, we present a set of conditions that correspond to the existence of the maximal solution of the BJMLDE satisfying the desired sign condition. [ABSTRACT FROM AUTHOR]

In this paper we study a system of decoupled forward-backward stochastic differential equations driven by a G-Brownian motion (G-FBSDEs) with non-degenerate diffusion. Our objective is to establish the existence of a relaxed optimal control for a non-smooth stochastic optimal control problem. The latter is given in terms of a decoupled G-FBSDE. The cost functional is the solution of the backward stochastic differential equation at the initial time. The key idea to establish existence of a relaxed optimal control is to replace the original control problem by a suitably regularised problem with mollified coefficients, prove the existence of a relaxed control, and then pass to the limit. [ABSTRACT FROM AUTHOR]

We study an optimal control problem for a stochastic model of tumour growth with drug application. This model consists of three stochastic hyperbolic equations describing the evolution of tumour cells. It also includes two stochastic parabolic equations describing the diffusions of nutrient and drug concentrations. Since all systems are subject to many uncertainties, we have added stochastic terms to the deterministic model to consider the random perturbations. Then, we have added control variables to the model according to the medical concepts to control the concentrations of drug and nutrient. In the optimal control problem, we have defined the stochastic and deterministic cost functions and we have proved the problems have unique optimal controls. For deriving the necessary conditions for optimal control variables, the stochastic adjoint equations are derived. We have proved the stochastic model of tumour growth and the stochastic adjoint equations have unique solutions. For proving the theoretical results, we have used a change of variable which changes the stochastic model and adjoint equations (a.s.) to deterministic equations. Then we have employed the techniques used for deterministic ones to prove the existence and uniqueness of optimal control. [ABSTRACT FROM AUTHOR]

When companies or organizations can evade audits on their harmful incidents, how should the affected entities design their audit and penalty policies? In "Audit and Remediation Strategies in the Presence of Evasion Capabilities" by Wang, de Véricourt, and Sun, the authors find random audits may be needed in the optimal policy. Specifically, the optimal policy alternates between ascending monetary penalties (without any audits) and random audits at a constant rate (when the penalty reach its maximum level). Only when the evasion is ineffective or the self-correction is too costly do deterministic audits become optimal. They tackle the problem in a continuous-time principal-agent framework with both adverse selection and moral hazard. In this paper, we explore how to uncover an adverse issue that may occur in organizations with the capability to evade detection. To that end, we formalize the problem of designing efficient auditing and remedial strategies as a dynamic mechanism design model. In this setup, a principal seeks to uncover and remedy an issue that occurs to an agent at a random point in time and that harms the principal if not addressed promptly. Only the agent observes the issue's occurrence, but the principal may uncover it by auditing the agent at a cost. The agent, however, can exert effort to reduce the audit's effectiveness in discovering the issue. We first establish that this setup reduces to the optimal stochastic control of a piecewise deterministic Markov process. The analysis of this process reveals that the principal should implement a dynamic cyclic auditing and remedial cost-sharing mechanism, which we characterize in closed form. Importantly, we find that the principal should randomly audit the agent unless the agent's evasion capacity is not very effective, and the agent cannot afford to self-correct the issue. In this latter case, the principal should follow predetermined audit schedules. Funding: This work was supported by the Deutsche Forschungsgemeinschaft (German Research Foundation) ["Audit Schedules in the Presence of Concealing Effort"; Grant 387250733]. Supplemental Material: The computer code and data that supports the findings of this study and the online appendix are available within this article's supplemental material at https://doi.org/10.1287/opre.2022.0289. [ABSTRACT FROM AUTHOR]

Problem definition: More than 99% of the new power generation capacity to be installed in the United States from 2023 to 2050 will be powered by wind, solar, and natural gas. Additionally, large-scale battery systems are planned to support power systems. It is paramount for policymakers and electric utilities to deepen the understanding of the operational and investment relations among renewable, flexible (natural gas-powered), and storage capacities. In this paper, we optimize both the joint operations and investment mix of these three types of resources, examining whether they act as investment substitutes or complements. Methodology/results: Using stochastic control theory, we identify and prove the structure of the optimal storage control policy, from which we determine various pairs of charging and discharging operations. We find that whether storage complements or substitutes other resources hinges on the operational pairs involved and whether executing these pairs is constrained by charging or discharging. Through extensive numerical analysis using data from a Florida utility, government agencies, and industry reports, we demonstrate how storage operations drive the investment relations among renewable, flexible, and storage capacities. Managerial implications: Storage and renewables substitute each other in meeting peak demand; storage complements renewables by storing surplus renewable output; renewables complement storage by compressing peak periods, facilitating peak shaving and displacement of flexible capacity. These substitution and complementary effects often coexist, and the dominant effect can alternate as costs change. A thorough understanding of these relations at both operational and investment levels empowers decision makers to optimize energy infrastructure investments and operations, thereby unlocking their full potential. Funding: This research was supported in part by Lilly Endowment, Inc., through its support for the Indiana University Pervasive Technology Institute. This research was also supported by Kelley School of Business, Indiana University, and Haslam College of Business, University of Tennessee. O. Q. Wu thanks Grant Thornton for their generous support. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.0068. [ABSTRACT FROM AUTHOR]

In this article, we study the optimal control of stochastic differential equations with random impulses. We optimize the performance index and add the influence of random impulses to the performance index with a random compensation function. Using the idea of stochastic analysis and dynamic programming principle, a new Hamilton–Jacobi–Bellman (HJB) equation is obtained, and the existence and uniqueness of its viscosity solution are proved. [ABSTRACT FROM AUTHOR]

This paper is concerned with the turnpike property for a class of stochastic linear-quadratic (LQ, for short) optimal control problems with periodic coefficients. The stability and stabilizability of the control system are studied, followed by the discussion of the existence and uniqueness of periodic solutions. A deterministic periodic LQ problem is introduced and solved, whose optimal pair, together with a pair of correction processes, serves as the turnpike limit of the stochastic problem. It is shown that the turnpike limit is periodic in the distribution sense. In the special case of constant coefficients, the turnpike limit turns out to have stationary distributions, with the expectation being the solution to a static optimization problem. [ABSTRACT FROM AUTHOR]

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle-type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

We consider a method for solving the problem of optimal control of a composite dynamic system with stochastic resource constraints. The method is based on the method of hierarchical optimization, which includes iterative solution of local problems and the coordination problem. To solve local problems, we apply the model predictive control. Taking into account stochastic resource constraints leads to the solution of one-stage problems of stochastic programming. [ABSTRACT FROM AUTHOR]

In this paper the connection between stochastic optimal control and reinforcement learning is investigated. Our main motivation is to apply importance sampling to sampling rare events which can be reformulated as an optimal control problem. By using a parameterised approach the optimal control problem becomes a stochastic optimization problem which still raises some open questions regarding how to tackle the scalability to high-dimensional problems and how to deal with the intrinsic metastability of the system. To explore new methods we link the optimal control problem to reinforcement learning since both share the same underlying framework, namely a Markov Decision Process (MDP). For the optimal control problem we show how the MDP can be formulated. In addition we discuss how the stochastic optimal control problem can be interpreted in the framework of reinforcement learning. At the end of the article we present the application of two different reinforcement learning algorithms to the optimal control problem and a comparison of the advantages and disadvantages of the two algorithms. [ABSTRACT FROM AUTHOR]

Summary: In this paper, we consider the optimal control of unknown stochastic dynamical system for both the finite‐horizon and infinite‐horizon cases. The objective of this paper is to find an optimal controller to minimize the expected value of a function which depends on the random disturbance. Throughout this paper, it is assumed that the mean vector and covariance matrix of the disturbance distribution is unknown. An uncertainty set in the space of mean vector and the covariance matrix is introduced. For the finite‐horizon case, we derive a closed‐form expression of the unique optimal policy and the opponents policy that generates the worst‐case distribution. For the infinite‐horizon case, we simplify the Riccati equation obtained in the finite‐hozion setting to an algebraic Riccati equation, which can guarantee the existence of the solution of the Riccati equation. It is shown that the resulting optimal policies obtained in these two cases can stabilize the expected value of the system state under the worst‐case distribution. Furthermore, the unknown system matrices can also be explicitly computed using the adaptive dynamic programming technique, which can help compute the optimal control policy by solving the algebraic Riccati equation. Finally, a simulation example is presented to demonstrate the effectiveness of our theoretical results. [ABSTRACT FROM AUTHOR]

This article concentrates in analyzing optimal controls for stochastic integrodifferential equation (SIDE) in Hilbert space. Necessary parameters are imposed to demonstrate the system that follows a unique variation of parameter formula using Leray Schauder Alternative. Subsequently, the existence of optimal control is investigated for the considered Lagrange control problem. The theoretical example with the mechanical example of ethanol fuelled engine are discussed to validate the results obtained. [ABSTRACT FROM AUTHOR]

This paper presents a novel optimization model that explores the optimal investment strategies for DC pension plans with return of premium clauses. We have assumed that the financial market consists of a risk-free asset and a risky asset, where the price of the risky asset follows the CEV model. Under the expected utility criterion, the optimal investment strategies were derived by employing stochastic optimal control theory and the Legendre transformation method. Explicit expressions of the optimal investment strategy were provided when the utility function was specified as exponential, power, or logarithmic. Finally, numerical analysis was conducted to examine the impact of factors such as interest rate, return rate, and volatility of the risky asset on the optimal strategies. [ABSTRACT FROM AUTHOR]

This paper deals with finite‐horizon minimum‐variance and covariance steering problems subject to constraints. The goal of the minimum variance problem is to steer the state mean of an uncertain system to a prescribed vector while minimizing the trace of its terminal state covariance whereas the goal in the covariance steering problem is to steer the covariance matrix of the terminal state to a prescribed positive definite matrix. The paper proposes a solution approach that relies on a stochastic version of the affine disturbance feedback control parametrization. In this control policy parametrization, the control input at each stage is expressed as an affine function of the history of disturbances that have acted upon the system. It is shown that this particular parametrization reduces the stochastic optimal control problems considered in this paper into tractable convex programs or difference of convex functions programs with essentially the same decision variables. In addition, the paper proposes a variation of this control parametrization that relies on truncated histories of past disturbances, which allows for sub‐optimal controllers to be designed that strike a balance between performance and computational cost. The suboptimality of the truncated policies is formally analyzed and closed form expressions are provided for the performance loss due to the use of the truncation scheme. Finally, the paper concludes with a comparative analysis of the truncated versions of the proposed policy parametrization and other standard policy parametrizations through numerical simulations. [ABSTRACT FROM AUTHOR]

In this paper, a stochastic brucellosis model with nonlocal transmission and spatial diffusion is established. Existence, uniqueness and positivity of mild solution to the model are obtained by adopting a truncation method. To ensure the Mean Square Exponential Stability (MSES) of the solution, sufficient conditions are given by employing the inequality techniques and the stability implies that the brucellosis die out in a short period of time. Moreover, we introduce elimination of infected animals and disinfection of brucella to the model as control strategies. Necessary conditions are given to obtain the optimal control which best balance the outcomes and costs of the control by applying Pontryagin’s maximum principle. Finally, the theoretical results are demonstrated by the numerical simulations. [ABSTRACT FROM AUTHOR]

In deterministic and stochastic control theory, relaxed or randomized control policies allow for versatile mathematical analysis (on continuity, compactness, convexity, and approximations) to be applicable with no artificial restrictions on the classes of control policies considered, leading to very general existence results on optimal measurable policies under various setups and information structures. On relaxed controls, two studied topologies are the Young and Borkar (weak\ast) topologies on spaces of functions from a state/measurement space to the space of probability measures on control action spaces; the former via a weak convergence topology on probability measures on a product space with a fixed marginal on the input (state) space, and the latter via a weak\ast topology on randomized policies viewed as maps from states/measurements to the space of signed measures with bounded variation. We establish implication and equivalence conditions between the Young and Borkar topologies on control policies. We then show that, under some conditions, for a controlled Markov chain with standard Borel spaces the invariant measure is weakly continuous on the space of stationary control policies defined by either of these topologies. An implication is near-optimality of quantized stationary policies in state and actions or continuous stationary and deterministic policies for average cost control under two sets of continuity conditions (with either weak continuity in the state-action pair or strong continuity in the action for each state) on transition kernels. [ABSTRACT FROM AUTHOR]

We consider a singularly perturbed system of stochastic differential equations proposed by Chaudhari et al. (Res. Math. Sci. 2018) to approximate the entropic gradient descent in the optimization of deep neural networks via homogenization. We embed it in a much larger class of two-scale stochastic control problems and rely on convergence results for Hamilton--Jacobi--Bellman equations with unbounded data proved recently by ourselves (ESAIM Control Optim. Calc. Var. 2023). We show that the limit of the value functions is itself the value function of an effective control problem with extended controls and that the trajectories of the perturbed system converge in a suitable sense to the trajectories of the limiting effective control system. These rigorous results improve the understanding of the convergence of the algorithms used by Chaudhari et al., as well as of their possible extensions where some tuning parameters are modeled as dynamic controls. [ABSTRACT FROM AUTHOR]

The fashion apparel industry is facing an increasingly growing demand, compounded by the short sales lifecycle and strong seasonality of clothing, posing significant challenges to inventory management in the retail sector. Despite some retailers like Uniqlo and Zara implementing inventory management and dynamic pricing strategies, challenges persist due to the dynamic nature of fashion trends and the stochastic factors affecting inventory. To address these issues, we construct a mathematical model based on the mathematical expression of the deterministic fashion level function, where the geometric Brownian motion, widely applied in finance, is initially utilized in the stochastic fashion level function. Drawing on research findings from deteriorating inventory management and stochastic optimization, we investigate the fluctuation of inventory levels, optimal dynamic pricing, optimal production rates, and profits—four crucial indicators—via Pontryagin's maximum principle. Analytical solutions are derived, and the numerical simulation is provided to verify and compare the proposed model with deterministic fashion level function models. The model emphasizes the importance of considering stochastic factors in decision-making processes and provides insights to enhance profitability, inventory management, and sustainable consumption in the fashion product industry. [ABSTRACT FROM AUTHOR]

This special issue of the Journal of Optimization Theory & Applications focuses on the work of Professors Cornelis Roos and Florian A. Potra in the field of optimization. The issue includes a biographical sketch of both professors, highlighting their achievements. The papers in the special issue cover various topics such as conic optimization, interior point methods, proximal methods, convex optimization, and stochastic optimal control. The authors of the papers have close connections to Professors Roos and Potra, either as students, visitors, or co-authors. The document is a compilation of research papers on interior-point methods (IPMs) and their applications in optimization and conic programming. The papers cover theoretical developments of IPMs, algorithms for linear complementarity problems (LCPs), weighted LCPs, and sufficient linear complementarity problems. There are also papers on new computational approaches for IPMs, applications of IPMs in different fields such as mechanics and support vector machines, and the theory and application of conic optimization. The document also includes papers on proximal methods for convex and non-convex problems, as well as papers on convexity and convex optimization. The volume concludes with acknowledgments to the authors, reviewers, and editor-in-chief. [Extracted from the article]

In this work, we obtain weak order-2 conditions of Runge–Kutta method for the optimal control of stochastic differential equations which occurs in many areas of economics and finance and recently in cognitive sciences and neuroscience. We get the order conditions that a stochastic Runge–Kutta technique must meet to have weak order two by comparing the stochastic expansion of the approximation with the associated Taylor scheme. Moreover, we present numerical examples which verify the theoretical results. We conclude our paper by a summary and an outlook to future research and application. [ABSTRACT FROM AUTHOR]

Broadcast control (BC) is a bio-inspired coordination technique for a swarm of agents in which a single coordinator broadcasts an identical scalar signal to all performing agents without discrimination, and the agents make appropriate moves towards the agents' collective optimal state without communicating with one another. The BC technique aims to accomplish a globally assigned task for which BC utilizes a stochastic optimization algorithm to coordinate a group of agents. However, the challenge intensifies as the system becomes larger: it requires a larger number of agents, which protracts the converging time for a single coordinator-based BC model. This paper proposes a revamped version of BC model, which assimilates distributed multiple coordinators to control a larger multi-agent system efficiently in a pragmatic manner. Precisely, in this hierarchical BC scheme, the distributed multiple sub-coordinators broadcast the identical feedback signal to the agents, which they receive from the global coordinator to accomplish the coverage control task of the ordinary agents. The dual role of sub-coordinators is manipulated by introducing weighted averaging of the gradient estimation under the stochastic optimization mechanism. The potency of the proposed model is analyzed with numerical simulation for a coverage control task, and various performance aspects are compared with the typical BC schemes to demonstrate its practicability and performance improvement. Particularly, the proposed scheme shows the same convergence with about 30% less traveling costs, and the near convergence is reached by only about one-third of iteration steps compared to the typical BC. [ABSTRACT FROM AUTHOR]

We propose logarithmic-barrier decomposition-based interior-point algorithms for solving two-stage stochastic quadratically constrained convex quadratic programming problems in a Hilbert space. We prove the polynomial complexity of the proposed algorithms, and show that this complexity is independent on the choice of the Hilbert space, and hence it coincides with the best-known complexity estimates in the finite-dimensional case. We also apply our results on a concrete example from the stochastic control theory. [ABSTRACT FROM AUTHOR]

Although adaptive cancer therapy shows promise in integrating evolutionary dynamics into treatment scheduling, the stochastic nature of cancer evolution has seldom been taken into account. Various sources of random perturbations can impact the evolution of heterogeneous tumors, making performance metrics of any treatment policy random as well. In this paper, we propose an efficient method for selecting optimal adaptive treatment policies under randomly evolving tumor dynamics. The goal is to improve the cumulative "cost" of treatment, a combination of the total amount of drugs used and the total treatment time. As this cost also becomes random in any stochastic setting, we maximize the probability of reaching the treatment goals (tumor stabilization or eradication) without exceeding a pre-specified cost threshold (or a "budget"). We use a novel Stochastic Optimal Control formulation and Dynamic Programming to find such "threshold-aware" optimal treatment policies. Our approach enables an efficient algorithm to compute these policies for a range of threshold values simultaneously. Compared to treatment plans shown to be optimal in a deterministic setting, the new "threshold-aware" policies significantly improve the chances of the therapy succeeding under the budget, which is correlated with a lower general drug usage. We illustrate this method using two specific examples, but our approach is far more general and provides a new tool for optimizing adaptive therapies based on a broad range of stochastic cancer models. Author summary: Tumor heterogeneities provide an opportunity to improve therapies by leveraging complex (often competitive) interactions of different types of cancer cells. These interactions are usually stochastic due to both individual cell differences and random events affecting the patient as a whole. The new generation of cancer models strive to account for this inherent stochasticity, and adaptive treatment plans need to reflect it as well. In optimizing such treatment, the most common approach is to maximize the probability of eventually stabilizing or eradicating the tumor. In this paper, we consider a more nuanced version of success, maximizing the probability of reaching these therapy goals before the cumulative burden from the disease and treatment exceed a chosen threshold. Importantly, our method allows computing such optimal treatment plans efficiently and for a range of thresholds at once. If used on a high-fidelity personalized model, our general approach could potentially be used by clinicians to choose the most suitable threshold after a detailed discussion of a specific patient's goals (e.g., to include the trade-offs between toxicity and quality of life). [ABSTRACT FROM AUTHOR]

Information drives trading and hence affects price dynamics. We study how an informed trader optimally trades, how multiple traders compete with each other, and how their competition affects price dynamics as well as their inventories in the linear price impact model framework. We formulate the informed trading problem as a stochastic optimal control problem and a stochastic differential game in different cases. By virtue of different methods, problems are solved explicitly in all cases. In the case of a single trader, the risk-averse informed trader trades less in the beginning and then speeds up to reduce exposure to the volatility of dynamic information. In addition, the risk-averse trader reveals information more slowly and thus contributes less to price efficiency. When multiple risk-neutral traders compete with information, regardless of the competition structure, they tend to trade more rapidly in the early stage to fill orders at a good price. Competition and the presence of a leader help promote price efficiency, and competition also leads to the empirically observed mean-reverting behavior of price dynamics and order flows. Furthermore, among risk-averse informed traders, inventory management triggers predatory trading and active information leakage during informed trading, and the positions of traders with the same level of risk aversion tend to be identical and mean-reverting to zero under multiple rounds of competition. [ABSTRACT FROM AUTHOR]

This paper proposes an innovative approach for constrained rendezvous trajectory optimization (CRTO) using stochastic optimal feedback control and unscented transform (UT) uncertainty quantification. The method overcomes limitations of traditional deterministic CRTO solutions that suffer from uncontrolled terminal state error growth and severely deteriorated path constraint satisfaction when initial state uncertainty, dynamics uncertainty, and navigation uncertainty are considered. The approach involves constructing a stochastic optimal feedback control (SOFC) problem with chance constraints and introducing linear feedback control to regulate both the mean and variance of terminal state errors. UT is employed to approximately quantify the state errors and their propagation, transforming the SOFC problem into an unconstrained deterministic optimization (UDO) problem. Differential dynamic programming (DDP) is then used to solve the UDO problem. The obtained stochastic optimal solution provides a robust rendezvous trajectory and a corresponding explicit closed-loop guidance law, improving terminal accuracy and path constraint satisfaction in the presence of various considered uncertainties. The influence of different term weights in the objective function on terminal accuracy and path constraint satisfaction is also studied. The effectiveness of the proposed approach is verified through Monte Carlo simulation, demonstrating the robustness of the closed-loop control strategy. The results highlight the potential of the method in enhancing terminal accuracy and path constraint satisfaction in uncertain rendezvous scenarios. • The uncertainty-resilient rendezvous trajectory optimization algorithm provides not only a nominal trajectory but an explicit optimal state-feedback control policy. • Rendezvous trajectory uncertainty caused by initial state errors, dynamics errors, and navigation errors is propagated using an unscented transformation method. • The proposed method considers common input constraints and path constraints in space rendezvous missions. [ABSTRACT FROM AUTHOR]

A mean-field linear quadratic stochastic (MF-SLQ for short) optimal control problem with hybrid disturbances and cross terms in a finite horizon is concerned. The state equation is a systems driven by the Wiener process and the Poisson random martingale measure disturbed by some stochastic perturbations. The cost functional is also disturbed, which means more general cases could be characterized, especially when extra environment perturbations exist. In this paper, the well-posedness result on the jump diffusion systems is obtained by the fixed point theorem and also the solvability of the MF-SLQ problem. Actually, by virtue of adjoint variables, classic variational calculus, and some dual representation, an optimal condition is derived. Throughout our research, in order to connect the optimal control and the state directly, two Riccati differential equations, a BSDE with random jumps and an ordinary equation (ODE for short) on disturbance terms are obtained by a decoupling technique, which provide an optimal feedback regulator. Meanwhile, the relationship between the two Riccati equations and the so-called mean-field stochastic Hamilton system is established. Consequently, the optimal value is characterized by the initial state, disturbances, and original value of the Riccati equations. Finally, an example is provided to illustrate our theoretic results. [ABSTRACT FROM AUTHOR]

Topology optimization is an appealing technique for optimal layout of active controllers of structures. However, the existing methods are mainly restricted to deterministic excitations, and the optimal control law involved is determined by use of the classical optimal control (COC) method, in which the sensitivities of the gain matrix with respect to the design variables need to be determined by solving the Riccati sensitivity equation numerically. In this study, a reliability-based topology optimization framework is proposed for optimal layout of active controllers of structures under nonstationary random excitations. The optimization problem is formulated as the minimization of the failure probability of the structure subjected to a specified maximum number of controllers. To avoid solving the Riccati equation, an explicit optimal control (EOC) method is first employed to derive the closed-form optimal control law in terms of the position parameters of controllers. The statistical moments of the optimal control forces and structural responses under random excitations are then obtained explicitly by the operation rules of moments, and the first-passage dynamic reliability of the structure can be formulated using the level-crossing theory. On this basis, the sensitivities of the structural failure probability with respect to the position parameters of controllers can be derived analytically by the direct differential method. Finally, the explicit formulations of the response statistics and the relevant sensitivities are incorporated into a gradient-based method of moving asymptotes (MMA) for topology optimization of the layout of controllers in conjunction with the solid isotropic material with penalization (SIMP) technique. Two numerical examples are presented to demonstrate the feasibility of the proposed topology optimization framework. [ABSTRACT FROM AUTHOR]

Based on the mean-variance criterion, this paper studies the robust optimal reinsurance problem under the competition between an insurance company and a reinsurance company. The insurance company operates n kinds of dependent insurance businesses, and it buys reinsurance for each insurance business to reduce the claim risk. Through relative performance, this paper quantifies the competition between the insurance company and the reinsurance company. The insurance company's goal is to choose an optimal reinsurance strategy to minimize the risk when the mean of terminal wealth is given in the worst market situation. By using the theory of stochastic control and stochastic dynamic programming, this paper establishes the Hamilton-Jacob-Bellman-Isaacs (HJBI) equation. Furthermore, by solving HJBI equation and using Lagrange duality theory, this paper obtains the explicit solution for the robust optimal reinsurance strategy. Finally, the inuence of model parameters on the robust optimal reinsurance strategy and efficient frontier is explained by numerical experiments. The research results can guide insurance companies to adopt optimal reinsurance strategies to minimize the risks that they face when operating a variety of insurance businesses. [ABSTRACT FROM AUTHOR]

In this paper, we study one kind of stochastic recursive optimal control problem with Markovian Switching. In this problem, the cost functional is described by the solution of backward stochastic differential equations with Markov chains. We prove the dynamic programming principle for this kind of optimal control problem and show that the value function is the unique viscosity solution of the corresponding Hamilton-Jacobi-Bellman equation. [ABSTRACT FROM AUTHOR]

This paper's focus is on finding the optimal strategies for a trader who invests in stock, a money market account and an inflation-linked index bond. The stock follows a jump diffusion process and the bond is linked to inflation making the two risky. The optimal strategies are determined on two generations of the life of an investor, that is before the investor dies and after the investor dies. We applied the concept of change of probability measures considering Girsanov's and the Radon–Nikodym theorems. We found the generator of the Backward Stochastic differential equations defined and employed the Hamilton–Jacobi–Bellman (HJB) dynamic programming in finding the stochastic optimal controls of interest. [ABSTRACT FROM AUTHOR]

This paper deals with a class of optimal control problems which arises in advertising models with Volterra Ornstein-Uhlenbeck process representing the product goodwill. Such choice of the model can be regarded as a stochastic modification of the classical Nerlove-Arrow model that allows to incorporate both presence of uncertainty and empirically observed memory effects such as carryover or distributed forgetting. We present an approach to solve such optimal control problems based on an infinite dimensional lift which allows us to recover Markov properties by formulating an optimization problem equivalent to the original one in a Hilbert space. Such technique, however, requires the Volterra kernel from the forward equation to have a representation of a particular form that may be challenging to obtain in practice. We overcome this issue for Hölder continuous kernels by approximating them with Bernstein polynomials, which turn out to enjoy a simple representation of the required type. Then we solve the optimal control problem for the forward process with approximated kernel instead of the original one and study convergence. The approach is illustrated with simulations. [ABSTRACT FROM AUTHOR]

In this paper, we study the maximum principle of mean field type control problems when the volatility function depends on the state and its measure and also the control, by using our recently developed method in [Bensoussan, A., Huang, Z. and Yam, S. C. P. [2023] Control theory on Wasserstein space: A new approach to optimality conditions, Ann. Math. Sci. Appl.; Bensoussan, A., Tai, H. M. and Yam, S. C. P. [2023] Mean field type control problems, some Hilbert-space-valued FBSDEs, and related equations, preprint (2023), arXiv:2305.04019; Bensoussan, A. and Yam, S. C. P. [2019] Control problem on space of random variables and master equation, ESAIM Control Optim. Calc. Var. 25, 10]. Our method is to embed the mean field type control problem into a Hilbert space to bypass the evolution in the Wasserstein space. We here give a necessary condition and a sufficient condition for these control problems in Hilbert spaces, and we also derive a system of forward–backward stochastic differential equations. [ABSTRACT FROM AUTHOR]