Search

Motivated by applications in queueing theory, we consider a class of singular stochastic control problems whose state space is the d-dimensional positive orthant. The original problem is approximated by a drift control problem, to which we apply a recently developed computational method that is feasible for dimensions up to d=30 or more. To show that nearly optimal solutions are obtainable using this method, we present computational results for a variety of examples, including queueing network examples that have appeared previously in the literature.

This paper studies a diffusion control problem motivated by challenges faced by public health agencies who run clinics to serve the public. A key challenge for these agencies is to motivate individuals to participate in the services provided. They must manage the flow of (voluntary) participants so that the clinic capacity is highly utilized, but not overwhelmed. The organization can deploy costly promotion activities to increase the inflow of participants. Ideally, the system manager would like to have enough participants waiting in a queue to serve as many individuals as possible and efficiently use clinic capacity. However, if too many participants sign up, resulting in a long wait, participants may become irritated and hesitate to participate again in the future. We develop a diffusion model of managing participant inflow mechanisms. Each mechanism corresponds to choosing a particular drift rate parameter for the diffusion model. The system manager seeks to balance three different costs optimally: i) a linear holding cost that captures the congestion concerns; ii) an idleness penalty corresponding to wasted clinic capacity and negative impact on public health, and iii) costs of promotion activities. We show that a nested-threshold policy for deployment of participant inflow mechanisms is optimal under the long-run average cost criterion. In this policy, the system manager progressively deploys mechanisms in increasing order of cost, as the number of participants in the queue decreases. We derive explicit formulas for the queue length thresholds that trigger each promotion activity, providing the system manager with guidance on when to use each mechanism.

Non-profit organizations that provide food, shelter, and other services to people in need, rely on volunteers to deliver their services. Unlike paid labor, non-profit organizations have less control over unpaid volunteers' schedules, efforts, and reliability. However, these organizations can invest in volunteer engagement activities to ensure a steady and adequate supply of volunteer labor. We study a key operational question of how a non-profit organization can manage its volunteer workforce capacity to ensure consistent provision of services. In particular, we formulate a multiclass queueing network model to characterize the optimal engagement activities for the non-profit organization to minimize the costs of enhancing volunteer engagement, while maximizing productive work done by volunteers. Because this problem appears intractable, we formulate an approximating Brownian control problem in the heavy traffic limit and study the dynamic control of that system. Our solution is a nested threshold policy with explicit congestion thresholds that indicate when the non-profit should optimally pursue various types of volunteer engagement activities. A numerical example calibrated using data from a large food bank shows that our dynamic policy for deploying engagement activities can significantly reduce the food bank's total annual cost of its volunteer operations while still maintaining almost the same level of social impact. This improvement in performance does not require any additional resources - it only requires that the food bank strategically deploy its engagement activities based on the number of volunteers signed up to work volunteer shifts.

Motivated by applications in queueing theory, we consider a stochastic control problem whose state space is the $d$-dimensional positive orthant. The controlled process $Z$ evolves as a reflected Brownian motion whose covariance matrix is exogenously specified, as are its directions of reflection from the orthant's boundary surfaces. A system manager chooses a drift vector $\theta(t)$ at each time $t$ based on the history of $Z$, and the cost rate at time $t$ depends on both $Z(t)$ and $\theta(t)$. In our initial problem formulation, the objective is to minimize expected discounted cost over an infinite planning horizon, after which we treat the corresponding ergodic control problem. Extending earlier work by Han et al. (Proceedings of the National Academy of Sciences, 2018, 8505-8510), we develop and illustrate a simulation-based computational method that relies heavily on deep neural network technology. For test problems studied thus far, our method is accurate to within a fraction of one percent, and is computationally feasible in dimensions up to at least $d=30$.

A system manager makes dynamic pricing and dispatch control decisions in a queueing network model motivated by ride-hailing applications. A novel feature of the model is that it incorporates travel times. Unfortunately, this renders the exact analysis of the problem intractable. Therefore, we study this problem in the heavy traffic regime. Under the assumptions of complete resource pooling and common travel distribution, we solve the problem in closed form by analyzing the corresponding Bellman equation. Using this solution, we propose a policy for the queueing system and illustrate its effectiveness in a simulation study.

We consider a network of agents. Associated with each agent are her covariate and outcome. Agents influence each other's outcomes according to a certain connection/influence structure. A subset of the agents participate on a platform, and hence, are observable to it. The rest are not observable to the platform and are called the latent agents. The platform does not know the influence structure of the observable or the latent parts of the network. It only observes the data on past covariates and decisions of the observable agents. Observable agents influence each other both directly and indirectly through the influence they exert on the latent agents. We investigate how the platform can estimate the dependence of the observable agents' outcomes on their covariates, taking the latent agents into account. First, we show that this relationship can be succinctly captured by a matrix and provide an algorithm for estimating it under a suitable approximate sparsity condition using historical data of covariates and outcomes for the observable agents. We also obtain convergence rates for the proposed estimator despite the high dimensionality that allows more agents than observations. Second, we show that the approximate sparsity condition holds under the standard conditions used in the literature. Hence, our results apply to a large class of networks. Finally, we apply our results to two practical settings: targeted advertising and promotional pricing. We show that by using the available historical data with our estimator, it is possible to obtain asymptotically optimal advertising/pricing decisions, despite the presence of latent agents., Comment: 106 pages

We consider optimal control of a multi-class queue in the Halfin--Whitt regime, and revisit the notion of asymptotic optimality and the associated optimality gaps. The existing results in the literature for such systems provide asymptotically optimal controls with optimality gaps of $o(\sqrt{n})$ where $n$ is the system size, for example, the number of servers. We construct a sequence of asymptotically optimal controls where the optimality gap grows logarithmically with the system size. Our analysis relies on a sequence of Brownian control problems, whose refined structure helps us achieve the improved optimality gaps., Comment: Published in at http://dx.doi.org/10.1214/11-AAP777 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

We consider data transmission through a time-selective, correlated (first-order Markov) Rayleigh fading channel subject to an average power constraint. The channel is estimated at the receiver with a pilot signal, and the estimate is fed back to the transmitter. The estimate is used for coherent demodulation, and to adapt the data and pilot powers. We explicitly determine the optimal pilot and data power control policies in a continuous-time limit where the channel state evolves as an Ornstein-Uhlenbeck diffusion process, and is estimated by a Kalman filter at the receiver. The optimal pilot policy switches between zero and the maximum (peak-constrained) value (``bang-bang'' control), and approximates the optimal discrete-time policy at low Signal-to-Noise Ratios (equivalently, large bandwidths). The switching boundary is defined in terms of the system state (estimated channel mean and associated error variance), and can be explicitly computed. Under the optimal policy, the transmitter conserves power by decreasing the training power when the channel is faded, thereby increasing the data rate. Numerical results show a significant increase in achievable rate due to the adaptive training scheme with feedback, relative to constant (non-adaptive) training, which does not require feedback. The gain is more pronounced at relatively low SNRs and with fast fading. Results are further verified through Monte Carlo simulations., Comment: 39 pages. Submitted to IEEE Transactions on Information Theory (Aug 2009)

We consider a class of open stochastic processing networks, with feedback routing and overlapping server capabilities, in heavy traffic. The networks we consider satisfy the so-called complete resource pooling condition and therefore have one-dimensional approximating Brownian control problems. We propose a simple discrete review policy for controlling such networks. Assuming 2+\epsilon moments on the interarrival times and processing times, we provide a conceptually simple proof of asymptotic optimality of the proposed policy., Comment: Published at http://dx.doi.org/10.1214/105051604000000495 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)