Search

We study continuous-time reinforcement learning (RL) for stochastic control in which system dynamics are governed by jump-diffusion processes. We formulate an entropy-regularized exploratory control problem with stochastic policies to capture the exploration--exploitation balance essential for RL. Unlike the pure diffusion case initially studied by Wang et al. (2020), the derivation of the exploratory dynamics under jump-diffusions calls for a careful formulation of the jump part. Through a theoretical analysis, we find that one can simply use the same policy evaluation and $q$-learning algorithms in Jia and Zhou (2022a, 2023), originally developed for controlled diffusions, without needing to check a priori whether the underlying data come from a pure diffusion or a jump-diffusion. However, we show that the presence of jumps ought to affect parameterizations of actors and critics in general. We investigate as an application the mean--variance portfolio selection problem with stock price modelled as a jump-diffusion, and show that both RL algorithms and parameterizations are invariant with respect to jumps. Finally, we present a detailed study on applying the general theory to option hedging.

We propose a very efficient method for pricing various types of lookback options under Markov models. We utilize the model-free representations of lookback option prices as integrals of first passage probabilities. We combine efficient numerical quadrature with continuous-time Markov chain approximation for the first passage problem to price lookbacks. Our method is applicable to a variety of models, including one-dimensional time-homogeneous and time-inhomogeneous Markov processes, regime-switching models and stochastic local volatility models. We demonstrate the efficiency of our method through various numerical examples.

We develop deep learning models to learn the hedge ratio for S&P500 index options directly from options data. We compare different combinations of features and show that a feedforward neural network model with time to maturity, Black-Scholes delta and a sentiment variable (VIX for calls and index return for puts) as input features performs the best in the out-of-sample test. This model significantly outperforms the standard hedging practice that uses the Black-Scholes delta and a recent data-driven model. Our results demonstrate the importance of market sentiment for hedging efficiency, a factor previously ignored in developing hedging strategies.

We propose a method based on continuous time Markov chain approximation to compute the distribution of Parisian stopping times and price Parisian options under general one-dimensional Markov processes. We prove the convergence of the method under a general setting and obtain sharp estimate of the convergence rate for diffusion models. Our theoretical analysis reveals how to design the grid of the CTMC to achieve faster convergence. Numerical experiments are conducted to demonstrate the accuracy and efficiency of our method for both diffusion and jump models. To show the versality of our approach, we develop extensions for multi-sided Parisian stopping times, the joint distribution of Parisian stopping times and first passage times, Parisian bonds and for more sophisticated models like regime-switching and stochastic volatility models.

We develop a new simulation method for multidimensional diffusions with sticky boundaries. The challenge comes from simulating the sticky boundary behavior, for which standard methods like the Euler scheme fail. We approximate the sticky diffusion process by a multidimensional continuous time Markov chain (CTMC), for which we can simulate easily. We develop two ways of constructing the CTMC: approximating the infinitesimal generator of the sticky diffusion by finite difference using standard coordinate directions, and matching the local moments using the drift and the eigenvectors of the covariance matrix as transition directions. The first approach does not always guarantee a valid Markov chain whereas the second one can. We show that both construction methods yield a first order simulation scheme, which can capture the sticky behavior and it is free from the curse of dimensionality. We apply our method to two applications: a multidimensional Brownian motion with all dimensions sticky which arises as the limit of a queuing system with exceptional service policy, and a multi-factor short rate model for low interest rate environment in which the stochastic factors are unbounded but the short rate is sticky at zero.

We propose a two-step framework for predicting the implied volatility surface over time without static arbitrage. In the first step, we select features to represent the surface and predict them over time. In the second step, we use the predicted features to construct the implied volatility surface using a deep neural network (DNN) model by incorporating constraints that prevent static arbitrage. We consider three methods to extract features from the implied volatility data: principal component analysis, variational autoencoder and sampling the surface, and we predict these features using LSTM. Using a long time series of implied volatility data for S\&P500 index options to train our models, we find two feature construction methods, sampling the surface and variational autoencoders combined with DNN for surface construction, are the best performers in out-of-sample prediction. In particular, they outperform a classical method substantially. Furthermore, the DNN model for surface construction not only removes static arbitrage, but also significantly reduces the prediction error compared with a standard interpolation method. Our framework can also be used to simulate the dynamics of the implied volatility surface without static arbitrage.

Semiconducting nanowires offer many opportunities for electronic and optoelectronic device applications due to their special geometries and unique physical properties. However, it has been challenging to synthesize semiconducting nanowires directly on SiO2/Si substrate due to lattice mismatch. Here, we developed a catalysis-free approach to achieve direct synthesis of long and straight InSe nanowires on SiO2/Si substrate through edge-homoepitaxial growth. We further achieved parallel InSe nanowires on SiO2/Si substrate through controlling growth conditions. We attributed the underlying growth mechanism to selenium self-driven vapor-liquid-solid process, which is distinct from conventional metal-catalytic vapor-liquid-solid method widely used for growing Si and III-V nanowires. Furthermore, we demonstrated that the as-grown InSe nanowire-based visible light photodetector simultaneously possesses an extraordinary photoresponsivity of 271 A/W, ultrahigh detectivity of 1.57*10^14 Jones and a fast response speed of microsecond scale. The excellent performance of the photodetector indicates that as-grown InSe nanowires are promising in future optoelectronic applications. More importantly, the proposed edge-homoepitaxial approach may open up a novel avenue for direct synthesis of semiconducting nanowire arrays on SiO2/Si substrate., Comment: 19 pages, 4 figures, published in Small

We develop continuous time Markov chain (CTMC) approximation of one-dimensional diffusions with a lower sticky boundary. Approximate solutions to the action of the Feynman-Kac operator associated with a sticky diffusion and first passage probabilities are obtained using matrix exponentials. We show how to compute matrix exponentials efficiently and prove that a carefully designed scheme achieves second order convergence. We also propose a scheme based on CTMC approximation for the simulation of sticky diffusions, for which the Euler scheme may completely fail. The efficiency of our method and its advantages over alternative approaches are illustrated in the context of bond pricing in a sticky short rate model for low interest environment.