Search

In this paper, we propose a sensitivity-based analysis to study the nonlinear behavior under nonexpected utility with probability distortions (or 'distorted utility' for short). We first discover the 'monolinearity' of distorted utility, which means that after properly changing the underlying probability measure, distorted utility becomes locally linear in probabilities, and the derivative of distorted utility is simply an expectation of the sample path derivative under the new measure. From the monolinearity property, simulation algorithms for estimating the derivative of distorted utility can be developed, leading to gradient-based search algorithms for the optimum of distorted utility. We then apply the sensitivity-based approach to the portfolio selection problem under distorted utility with complete and incomplete markets. For the complete markets case, the first-order condition is derived and optimal wealth deduced. For the incomplete markets case, a dual characterization of optimal policies is provided; a solvable incomplete market example with unhedgeable interest rate risk is also presented. We expect this sensitivity-based approach to be generally applicable to optimization problems involving probability distortions. [ABSTRACT FROM AUTHOR]

Abstract: We introduce a sensitivity-based view to the area of learning and optimization of stochastic dynamic systems. We show that this sensitivity-based view provides a unified framework for many different disciplines in this area, including perturbation analysis, Markov decision processes, reinforcement learning, identification and adaptive control, and singular stochastic control; and that this unified framework applies to both the discrete event dynamic systems and continuous-time continuous-state systems. Many results in these disciplines can be simply derived and intuitively explained by using two performance sensitivity formulas. In addition, we show that this sensitivity-based view leads to new results and opens up new directions for future research. For example, the n th bias optimality of Markov processes has been established and the event-based optimization may be developed; this approach has computational and other advantages over the state-based approaches. [Copyright &y& Elsevier]

We propose a unified framework to Markov decision problems and performance sensitivity analysis for multichain Markov processes with both discounted and average-cost performance criteria. With the fundamental concept of performance potentials, we derive both performance-gradient and performance-difference formulas, which play the central role in performance optimization. The standard policy iteration algorithms for both discounted- and average-reward MDPs can be established using the performance-difference formulas in a simple and intuitive way; and the performance-gradient formulas together with stochastic approximation may lead to new optimization schemes. This sensitivity-based point of view of performance optimization provides some insights that link perturbation analysis, Markov decision processes, and reinforcement learning together. The research is an extension of the previous work on ergodic Markov chains (Cao, Automatica 36 (2000) 771). [Copyright &y& Elsevier]

Abstract: We consider the optimization of queueing systems with service rates depending on system states. The optimization criterion is the long-run customer-average performance, which is an important performance metric, different from the traditional time-average performance. We first establish, with perturbation analysis, a difference equation of the customer-average performance in closed networks with exponentially distributed service times and state-dependent service rates. Then we propose a policy iteration optimization algorithm based on this difference equation. This algorithm can be implemented on-line with a single sample path and does not require knowing the routing probabilities of queueing systems. Finally, we give numerical experiments which demonstrate the efficiency of our algorithm. This paper gives a new direction to efficiently optimize the “customer-centric” performance in queueing systems. [Copyright &y& Elsevier]

Financial institutions are interconnected by holding debt claims against each other. A default bank may cause its creditors to default, and the risk may be further propagated to up-stream institutes (risk contagion). Such interconnection is a key contributing factor to the past worldwide financial crisis. We show that a good mechanism of default liquidation may improve the total wealth of the financial system and therefore may curb the risk contagion. We formulate this problem as a nonlinear optimization problem with constraints and propose an optimal liquidation policy to minimize the system’s loss. We show that the problem resembles a Markov decision problem (MDP) and therefore we can apply the direct-comparison based optimization approach to solve this problem. Higher order directional derivatives and some optimality properties are obtained. Furthermore, we derive an iterative algorithm which combines both the policy iteration and the gradient based approach to find a local optimal policy, and under some conditions, a global optimal policy. Our work provides a new direction in curbing the risk contagion in financial networks; and it illustrates the advantages of the direct-comparison based approach, originated in the field of discrete event dynamic system, in nonlinear optimization problems. [ABSTRACT FROM AUTHOR]