Search

In this paper, we consider contextual stochastic optimization using Nadaraya-Watson kernel regression, which is one of the most common approaches in nonparametric regression. Recent studies have explored the asymptotic convergence behavior of using Nadaraya-Watson kernel regression in contextual stochastic optimization; however, the performance guarantee under finite samples remains an open question. This paper derives a finite-sample generalization bound of the Nadaraya-Watson estimator with a spherical kernel under a generic loss function. Based on the generalization bound, we further establish a suboptimality bound for the solution of the Nadaraya-Watson approximation problem relative to the optimal solution. Finally, we derive the optimal kernel bandwidth and provide a sample complexity analysis of the Nadaraya-Watson approximation problem.

In this paper, we propose a general distributionally robust framework for performative optimization, where the selected decision can influence the probabilistic distribution of uncertain parameters. Our framework facilitates safe decision-making in scenarios with incomplete information about the underlying decision-dependent distributions, relying instead on accessible reference distributions. To tackle the challenge of decision-dependent uncertainty, we introduce an algorithm named repeated robust risk minimization. This algorithm decouples the decision variables associated with the ambiguity set from the expected loss, optimizing the latter at each iteration while keeping the former fixed to the previous decision. By leveraging the strong connection between distributionally robust optimization and regularization, we establish a linear convergence rate to a performatively stable point and provide a suboptimality performance guarantee for the proposed algorithm. Finally, we examine the performance of our proposed model through an experimental study in strategic classification.

We study two-stage distributionally robust optimization (DRO) problems with decision-dependent information discovery (DDID) wherein (a portion of) the uncertain parameters are revealed only if an (often costly) investment is made in the first stage. This class of problems finds many important applications in selection problems (e.g., in hiring, project portfolio optimization, or optimal sensor location). Despite the problem's wide applicability, it has not been previously studied. We propose a framework for modeling and approximately solving DRO problems with DDID. We formulate the problem as a min-max-min-max problem and adopt the popular K-adaptability approximation scheme, which chooses K candidate recourse actions here-and-now and implements the best of those actions after the uncertain parameters that were chosen to be observed are revealed. We then present a decomposition algorithm that solves the K-adaptable formulation exactly. In particular, we devise a cutting plane algorithm which iteratively solves a relaxed version of the problem, evaluates the true objective value of the corresponding solution, generates valid cuts, and imposes them in the relaxed problem. For the evaluation problem, we develop a branch-and-cut algorithm that provably converges to an optimal solution. We showcase the effectiveness of our framework on the R&D project portfolio optimization problem and the best box problem.

A traditional stochastic program under a finite population typically seeks to optimize efficiency by maximizing the expected profits or minimizing the expected costs, subject to a set of constraints. However, implementing such optimization-based decisions can have varying impacts on individuals, and when assessed using the individuals' utility functions, these impacts may differ substantially across demographic groups delineated by sensitive attributes, such as gender, race, age, and socioeconomic status. As each group comprises multiple individuals, a common remedy is to enforce group fairness, which necessitates the measurement of disparities in the distributions of utilities across different groups. This paper introduces the concept of Distributionally Fair Stochastic Optimization (DFSO) based on the Wasserstein fairness measure. The DFSO aims to minimize distributional disparities among groups, quantified by the Wasserstein distance, while adhering to an acceptable level of inefficiency. Our analysis reveals that: (i) the Wasserstein fairness measure recovers the demographic parity fairness prevalent in binary classification literature; (ii) this measure can approximate the well-known Kolmogorov-Smirnov fairness measure with considerable accuracy; and (iii) despite DFSO's biconvex nature, the epigraph of the Wasserstein fairness measure is generally Mixed-Integer Convex Programming Representable (MICP-R). Additionally, we introduce two distinct lower bounds for the Wasserstein fairness measure: the Jensen bound, applicable to the general Wasserstein fairness measure, and the Gelbrich bound, specific to the type-2 Wasserstein fairness measure. We establish the exactness of the Gelbrich bound and quantify the theoretical difference between the Wasserstein fairness measure and the Gelbrich bound., Comment: No more revision

We consider the problem of learning fair policies for multi-stage selection problems from observational data. This problem arises in several high-stakes domains such as company hiring, loan approval, or bail decisions where outcomes (e.g., career success, loan repayment, recidivism) are only observed for those selected. We propose a multi-stage framework that can be augmented with various fairness constraints, such as demographic parity or equal opportunity. This problem is a highly intractable infinite chance-constrained program involving the unknown joint distribution of covariates and outcomes. Motivated by the potential impact of selection decisions on people's lives and livelihoods, we propose to focus on interpretable linear selection rules. Leveraging tools from causal inference and sample average approximation, we obtain an asymptotically consistent solution to this selection problem by solving a mixed binary conic optimization problem, which can be solved using standard off-the-shelf solvers. We conduct extensive computational experiments on a variety of datasets adapted from the UCI repository on which we show that our proposed approaches can achieve an 11.6% improvement in precision and a 38% reduction in the measure of unfairness compared to the existing selection policy., Comment: 38th Annual AAAI Conference on Artificial Intelligence, 2024

Consider an M/M/$s$ queue with the additional feature that the arrival rate is a random variable of which only the mean, variance, and range are known. Using semi-infinite linear programming and duality theory for moment problems, we establish for this setting tight bounds for the expected waiting time. These bounds correspond to an arrival rate that takes only two values. The proofs crucially depend on the fact that the expected waiting time, as function of the arrival rate, has a convex derivative. We apply the novel tight bounds to a rational queueing model, where arriving individuals decide to join or balk based on expected utility and only have partial knowledge about the market size., Comment: 28 pages, 3 figures

We consider a continuous-time continuous-space stochastic optimal control problem, where the controller lacks exact knowledge of the underlying diffusion process, relying instead on a finite set of historical disturbance trajectories. In situations where data collection is limited, the controller synthesized from empirical data may exhibit poor performance. To address this issue, we introduce a novel approach named Distributionally Robust Path Integral (DRPI). The proposed method employs distributionally robust optimization (DRO) to robustify the resulting policy against the unknown diffusion process. Notably, the DRPI scheme shows similarities with risk-sensitive control, which enables us to utilize the path integral control (PIC) framework as an efficient solution scheme. We derive theoretical performance guarantees for the DRPI scheme, which closely aligns with selecting a risk parameter in risk-sensitive control. We validate the efficacy of our scheme and showcase its superiority when compared to risk-neutral PIC policies in the absence of the true diffusion process.

This paper presents an extension of Naor's analysis on the join-or-balk problem in observable M/M/1 queues. While all other Markovian assumptions still hold, we explore this problem assuming uncertain arrival rates under the distributionally robust settings. We first study the problem with the classical moment ambiguity set, where the support, mean, and mean-absolute deviation of the underlying distribution are known. Next, we extend the model to the data-driven setting, where decision makers only have access to a finite set of samples. We develop three optimal joining threshold strategies from the perspective of an individual customer, a social optimizer, and a revenue maximizer, such that their respective worst-case expected benefit rates are maximized. Finally, we compare our findings with Naor's original results and the traditional sample average approximation scheme.

We consider generic stochastic optimization problems in the presence of side information which enables a more insightful decision. The side information constitutes observable exogenous covariates that alter the conditional probability distribution of the random problem parameters. A decision maker who adapts her decisions according to the observed side information solves an optimization problem where the objective function is specified by the conditional expectation of the random cost. If the joint probability distribution is unknown, then the conditional expectation can be approximated in a data-driven manner using the Nadaraya-Watson (NW) kernel regression. While the emerging approximation scheme has found successful applications in diverse decision problems under uncertainty, it is largely unknown whether the scheme can provide any reasonable out-of-sample performance guarantees. In this paper, we establish guarantees for the generic problems by leveraging techniques from moderate deviations theory. Our analysis motivates the use of a variance-based regularization scheme which, in general, leads to a non-convex optimization problem. We adopt ideas from distributionally robust optimization to obtain tractable formulations. We present numerical experiments for newsvendor and wind energy commitment problems to highlight the effectiveness of our regularization scheme.

We study two-stage stochastic optimization problems with random recourse, where the adaptive decisions are multiplied with the uncertain parameters in both the objective function and the constraints. To mitigate the computational intractability of infinite-dimensional optimization, we propose a scalable approximation scheme via piecewise linear and piecewise quadratic decision rules. We then develop a data-driven distributionally robust framework with two layers of robustness to address distributionally uncertainty. The emerging optimization problem can be reformulated as an exact copositive program, which admits tractable approximations in semidefinite programming. We design a decomposition algorithm where smaller-size semidefinite programs can be solved in parallel, which further reduces the runtime. Lastly, we establish the performance guarantees of the proposed scheme and demonstrate its effectiveness through numerical examples., Comment: 37 pages, 5 figures, 6 tables

Increasing integration of renewable generation poses significant challenges to ensure robustness guarantees in real-time energy system decision-making. This work aims to develop a robust optimal transmission switching (OTS) framework that can effectively relieve grid congestion and mitigate renewable curtailment. We formulate a two-stage distributionally robust chance-constrained (DRCC) problem that assures limited constraint violations for any uncertainty distribution within an ambiguity set. Here, the second-stage recourse variables are represented as linear functions of uncertainty, yielding an equivalent reformulation involving linear constraints only. We utilize moment-based (mean-mean absolute deviation) and distance-based (infinity-Wasserstein distance) ambiguity sets that lead to scalable mixed-integer linear program (MILP) formulations. Numerical experiments on the IEEE 14-bus and 118-bus systems have demonstrated the performance improvements of the proposed DRCC-OTS approaches in terms of guaranteed constraint violations and reduced renewable curtailment. In particular, the computational efficiency of the moment-based MILP approach, which is scenario-free with fixed problem dimensions, has been confirmed, making it suitable for real-time grid operations.

We propose a distributionally robust classification model with a fairness constraint that encourages the classifier to be fair in view of the equality of opportunity criterion. We use a type-$\infty$ Wasserstein ambiguity set centered at the empirical distribution to model distributional uncertainty and derive a conservative reformulation for the worst-case equal opportunity unfairness measure. We establish that the model is equivalent to a mixed binary optimization problem, which can be solved by standard off-the-shelf solvers. To improve scalability, we further propose a convex, hinge-loss-based model for large problem instances whose reformulation does not incur any binary variables. Moreover, we also consider the distributionally robust learning problem with a generic ground transportation cost to hedge against the uncertainties in the label and sensitive attribute. Finally, we numerically demonstrate that our proposed approaches improve fairness with negligible loss of predictive accuracy.

We consider the problem of clustering datasets in the presence of arbitrary outliers. Traditional clustering algorithms such as k-means and spectral clustering are known to perform poorly for datasets contaminated with even a small number of outliers. In this paper, we develop a provably robust spectral clustering algorithm that applies a simple rounding scheme to denoise a Gaussian kernel matrix built from the data points and uses vanilla spectral clustering to recover the cluster labels of data points. We analyze the performance of our algorithm under the assumption that the "good" data points are generated from a mixture of sub-gaussians (we term these "inliers"), while the outlier points can come from any arbitrary probability distribution. For this general class of models, we show that the misclassification error decays at an exponential rate in the signal-to-noise ratio, provided the number of outliers is a small fraction of the inlier points. Surprisingly, this derived error bound matches with the best-known bound for semidefinite programs (SDPs) under the same setting without outliers. We conduct extensive experiments on a variety of simulated and real-world datasets to demonstrate that our algorithm is less sensitive to outliers compared to other state-of-the-art algorithms proposed in the literature., Comment: 54 pages, 8 figures

Increasing penetration of wind and renewable generation poses significant challenges to the power system operations and reliability. This paper considers the real-time optimal transmission switching (OTS) problem for determining the generation dispatch and network topology that can account for uncertain energy resources. To efficiently solve the resultant two-stage stochastic program, we propose a tractable linear decision rule (LDR) based approximation solution that can eliminate the uncertainty variables and lead to fixed number of constraints. The proposed LDR approach can guarantee feasibility, and significantly reduces the computational complexity of existing approaches that grows with the number of randomly generated samples of uncertainty. Numerical studies on IEEE test cases demonstrate the high approximation accuracy of the proposed LDR solution and its computational efficiency for real-time OTS implementations.

We study decision rule approximations for generic multi-stage robust linear optimization problems. We consider linear decision rules for the case when the objective coefficients, the recourse matrices, and the right-hand sides are uncertain, and consider quadratic decision rules for the case when only the right-hand sides are uncertain. The resulting optimization problems are NP-hard but amenable to copositive programming reformulations that give rise to tight conservative approximations. We further enhance these approximations through new piecewise decision rule schemes. Finally, we prove that our proposed approximations are tighter than the state-of-the-art schemes and demonstrate their superiority through numerical experiments., Comment: 38 pages, 6 tables

We study the problem of finding the Lowner-John ellipsoid, i.e., an ellipsoid with minimum volume that contains a given convex set. We reformulate the problem as a generalized copositive program, and use that reformulation to derive tractable semidefinite programming approximations for instances where the set is defined by affine and quadratic inequalities. We prove that, when the underlying set is a polytope, our method never provides an ellipsoid of higher volume than the one obtained by scaling the maximum volume inscribed ellipsoid. We empirically demonstrate that our proposed method generates high-quality solutions faster than solving the problem to optimality. Furthermore, we outperform the existing approximation schemes in terms of solution time and quality. We present applications of our method to obtain piecewise-linear decision rule approximations for dynamic distributionally robust problems with random recourse, and to generate ellipsoidal approximations for the set of reachable states in a linear dynamical system when the set of allowed controls is a polytope.

In this paper, we show that the popular K-means clustering problem can equivalently be reformulated as a conic program of polynomial size. The arising convex optimization problem is NP-hard, but amenable to a tractable semidefinite programming (SDP) relaxation that is tighter than the current SDP relaxation schemes in the literature. In contrast to the existing schemes, our proposed SDP formulation gives rise to solutions that can be leveraged to identify the clusters. We devise a new approximation algorithm for K-means clustering that utilizes the improved formulation and empirically illustrate its superiority over the state-of-the-art solution schemes.

We study robust convex quadratic programs where the uncertain problem parameters can contain both continuous and integer components. Under the natural boundedness assumption on the uncertainty set, we show that the generic problems are amenable to exact copositive programming reformulations of polynomial size. These convex optimization problems are NP-hard but admit a conservative semidefinite programming (SDP) approximation that can be solved efficiently. We prove that the popular approximate S-lemma method --- which is valid only in the case of continuous uncertainty --- is weaker than our approximation. We also show that all results can be extended to the two-stage robust quadratic optimization setting if the problem has complete recourse. We assess the effectiveness of our proposed SDP reformulations and demonstrate their superiority over the state-of-the-art solution schemes on instances of least squares, project management, and multi-item newsvendor problems.

Adaptive robust optimization problems are usually solved approximately by restricting the adaptive decisions to simple parametric decision rules. However, the corresponding approximation error can be substantial. In this paper we show that two-stage robust and distributionally robust linear programs can often be reformulated exactly as conic programs that scale polynomially with the problem dimensions. Specifically, when the ambiguity set constitutes a 2-Wasserstein ball centered at a discrete distribution, then the distributionally robust linear program is equivalent to a copositive program (if the problem has complete recourse) or can be approximated arbitrarily closely by a sequence of copositive programs (if the problem has sufficiently expensive recourse). These results directly extend to the classical robust setting and motivate strong tractable approximations of two-stage problems based on semidefinite approximations of the copositive cone. We also demonstrate that the two-stage distributionally robust optimization problem is equivalent to a tractable linear program when the ambiguity set constitutes a 1-Wasserstein ball centered at a discrete distribution and there are no support constraints.

In data-driven inverse optimization an observer aims to learn the preferences of an agent who solves a parametric optimization problem depending on an exogenous signal. Thus, the observer seeks the agent's objective function that best explains a historical sequence of signals and corresponding optimal actions. We focus here on situations where the observer has imperfect information, that is, where the agent's true objective function is not contained in the search space of candidate objectives, where the agent suffers from bounded rationality or implementation errors, or where the observed signal-response pairs are corrupted by measurement noise. We formalize this inverse optimization problem as a distributionally robust program minimizing the worst-case risk that the {\em predicted} decision ({\em i.e.}, the decision implied by a particular candidate objective) differs from the agent's {\em actual} response to a random signal. We show that our framework offers rigorous out-of-sample guarantees for different loss functions used to measure prediction errors and that the emerging inverse optimization problems can be exactly reformulated as (or safely approximated by) tractable convex programs when a new suboptimality loss function is used. We show through extensive numerical tests that the proposed distributionally robust approach to inverse optimization attains often better out-of-sample performance than the state-of-the-art approaches.