Search

Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend to become numerically intractable as the size of the model grows fast. Recent works use machine learning techniques to overcome this crucial issue, but with no convergence guarantee. This note provides a brief overview of literature on solving large Markov decision processes, and exploiting them to solve important combinatorial optimization problems.

In time series analysis, when fitting an autoregressive model, one must solve a Toeplitz ordinary least squares problem numerous times to find an appropriate model, which can severely affect computational times with large data sets. Two recent algorithms (LSAR and Repeated Halving) have applied randomized numerical linear algebra (RandNLA) techniques to fitting an autoregressive model to big time-series data. We investigate and compare the quality of these two approximation algorithms on large-scale synthetic and real-world data. While both algorithms display comparable results for synthetic datasets, the LSAR algorithm appears to be more robust when applied to real-world time series data. We conclude that RandNLA is effective in the context of big-data time series., 28 pages, 11 figures

We develop a new efficient algorithm for the analysis of large-scale time series data. We firstly define rolling averages, derive their analytical properties, and establish their asymptotic distribution. These theoretical results are subsequently exploited to develop an efficient algorithm, called Rollage, for fitting an appropriate AR model to big time series data. When used in conjunction with the Durbin's algorithm, we show that the Rollage algorithm can be used as a criterion to optimally fit ARMA models to big time series data. Empirical experiments on large-scale synthetic time series data support the theoretical results and reveal the efficacy of this new approach, especially when compared to existing methodology.

Reinforcement learning is important part of artificial intelligence. In this paper, we review model-free reinforcement learning that utilizes the average reward optimality criterion in the infinite horizon setting. Motivated by the solo survey by Mahadevan (1996a), we provide an updated review of work in this area and extend it to cover policy-iteration and function approximation methods (in addition to the value-iteration and tabular counterparts). We present a comprehensive literature mapping. We also identify and discuss opportunities for future work., 36 pages, refined prelim and politer sections

We apply methods from randomized numerical linear algebra (RandNLA) to develop improved algorithms for the analysis of large-scale time series data. We first develop a new fast algorithm to estimate the leverage scores of an autoregressive (AR) model in big data regimes. We show that the accuracy of approximations lies within $(1+\bigO{\varepsilon})$ of the true leverage scores with high probability. These theoretical results are subsequently exploited to develop an efficient algorithm, called LSAR, for fitting an appropriate AR model to big time series data. Our proposed algorithm is guaranteed, with high probability, to find the maximum likelihood estimates of the parameters of the underlying true AR model and has a worst case running time that significantly improves those of the state-of-the-art alternatives in big data regimes. Empirical results on large-scale synthetic as well as real data highly support the theoretical results and reveal the efficacy of this new approach., 38 pages, 8 figures

Reliable demand forecasts are critical for the effective supply chain management. Several endogenous and exogenous variables can influence the dynamics of demand, and hence a single statistical model that only consists of historical sales data is often insufficient to produce accurate forecasts. In practice, the forecasts generated by baseline statistical models are often judgmentally adjusted by forecasters to incorporate factors and information that are not incorporated in the baseline models. There are however systematic events whose effect can be effectively quantified and modeled to help minimize human intervention in adjusting the baseline forecasts. In this paper, we develop and test a novel regime-switching approach to quantify systematic information/events and objectively incorporate them into the baseline statistical model. Our simple yet practical and effective model can help limit forecast adjustments to only focus on the impact of less systematic events such as sudden climate change or dynamic market activities. The proposed model and approach is validated empirically using sales and promotional data from two Australian companies. Discussions focus on a thorough analysis of the forecasting and benchmarking results. Our analysis indicates that the proposed model can successfully improve the forecast accuracy when compared to the current industry practice which heavily relies on human judgment to factor in all types of information/events.

In this paper, we investigate the possibility of improving the performance of multi-objective optimization solution approaches using machine learning techniques. Specifically, we focus on multi-objective binary linear programs and employ one of the most effective and recently developed criterion space search algorithms, the so-called KSA, during our study. This algorithm computes all nondominated points of a problem with p objectives by searching on a projected criterion space, i.e., a (p-1)-dimensional criterion apace. We present an effective and fast learning approach to identify on which projected space the KSA should work. We also present several generic features/variables that can be used in machine learning techniques for identifying the best projected space. Finally, we present an effective bi-objective optimization based heuristic for selecting the best subset of the features to overcome the issue of overfitting in learning. Through an extensive computational study over 2000 instances of tri-objective Knapsack and Assignment problems, we demonstrate that an improvement of up to 12% in time can be achieved by the proposed learning method compared to a random selection of the projected space.

We study the problem of choosing the best subset of p features in linear regression given n observations. This problem naturally contains two objective functions including minimizing the amount of bias and minimizing the number of predictors. The existing approaches transform the problem into a single-objective optimization problem. We explain the main weaknesses of existing approaches, and to overcome their drawbacks, we propose a bi-objective mixed integer linear programming approach. A computational study shows the efficacy of the proposed approach., 13 pages, 4 figures, 1 table

In this paper, we propose an adaptation of the Cross Entropy (CE) method called Projection-Adapted CE (PACE) to solve a transmission expansion problem that arises in management of national and provincial electricity grids. The aim of the problem is to find an expansion policy that is both economical and operational from the technical perspective. Often, the transmission network expansion problem is mathematically formulated as a mixed integer nonlinear program that is very challenging algorithmically. The challenge originates from the fact that a global optimum should be found despite the presence, of possibly a huge number, of local optima. The PACE method shows promise in solving global optimization problems regardless of continuity or other assumptions. In our approach, we sample the integer variables using the CE mechanism, and solve LPs to obtain matching continuous variables. Numerical results, on selected test systems, demonstrate the potential of this approach. Refereed/Peer-reviewed

We develop a new, random walk-based, algorithm for the Hamiltonian cycle problem. The random walk is on pairs of extreme points of two suitably constructed polytopes. The latter are derived from geometric properties of the space of discounted occupational measures corresponding to a given graph. The algorithm searches for a measure that corresponds to a common extreme point in these two specially constructed polyhedral subsets of that space. We prove that if a given undirected graph is Hamiltonian, then with probability one this random walk algorithm detects its Hamiltonicity in a finite number of iterations. We support these theoretical results by numerical experiments that demonstrate a surprisingly slow growth in the required number of iterations with the size of the given graph. Refereed/Peer-reviewed