Search

The integration of the variable renewable energies makes the operation conditions of the power system ever-changeable. Consequently, the power system operational reliability evaluation is increasingly important. This paper introduces the concept of decision-dependent uncertainty (DDU) in the operational reliability evaluation. Unlike the exogenous uncertainties, DDU reveals that the decisions of the system operation could significantly affect the resolution of the uncertainties which influence the reliability metrics. In this paper, the proposed DDU modeling method links the device reliability indices, i.e., the forced outage rate, and the operational-decision variables. The impacts of DDU on operational reliability are analyzed based on a reliability-constrained stochastic unit commitment (UC) model. An adaptive reliability improvement UC (ARIUC) algorithm is proposed to efficiently solve the problem. Case studies underline the necessity of considering DDU in power system operational reliability evaluations. [ABSTRACT FROM AUTHOR]

The Area under the ROC curve (AUC) is a well-known ranking metric for problems such as imbalanced learning and recommender systems. The vast majority of existing AUC-optimization-based machine learning methods only focus on binary-class cases, while leaving the multiclass cases unconsidered. In this paper, we start an early trial to consider the problem of learning multiclass scoring functions via optimizing multiclass AUC metrics. Our foundation is based on the M metric, which is a well-known multiclass extension of AUC. We first pay a revisit to this metric, showing that it could eliminate the imbalance issue from the minority class pairs. Motivated by this, we propose an empirical surrogate risk minimization framework to approximately optimize the M metric. Theoretically, we show that: (i) optimizing most of the popular differentiable surrogate losses suffices to reach the Bayes optimal scoring function asymptotically; (ii) the training framework enjoys an imbalance-aware generalization error bound, which pays more attention to the bottleneck samples of minority classes compared with the traditional $O(\sqrt{1/N})$ O (1 / N) result. Practically, to deal with the low scalability of the computational operations, we propose acceleration methods for three popular surrogate loss functions, including the exponential loss, squared loss, and hinge loss, to speed up loss and gradient evaluations. Finally, experimental results on 11 real-world datasets demonstrate the effectiveness of our proposed framework. The code is now available at https://github.com/joshuaas/Learning-with-Multiclass-AUC-Theory-and-Algorithms. [ABSTRACT FROM AUTHOR]

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

Compressive covariance estimation has arisen as a class of techniques whose aim is to obtain second-order statistics of stochastic processes from compressive measurements. Recently, these methods have been used in various image processing and communications applications, including denoising, spectrum sensing, and compression. Notice that estimating the covariance matrix from compressive samples leads to ill-posed minimizations with severe performance loss at high compression rates. In this regard, a regularization term is typically aggregated to the cost function to consider prior information about a particular property of the covariance matrix. Hence, this paper proposes an algorithm based on the projected gradient method to recover low-rank or Toeplitz approximations of the covariance matrix from compressive measurements. The proposed algorithm divides the compressive measurements into data subsets projected onto different subspaces and accurately estimates the covariance matrix by solving a single optimization problem assuming that each data subset contains an approximation of the signal statistics. Furthermore, gradient filtering is included at every iteration of the proposed algorithm to minimize the estimation error. The error induced by the proposed splitting approach is analytically derived along with the convergence guarantees of the proposed method. The proposed algorithm estimates the covariance matrix of hyperspectral images from synthetic and real compressive samples. Extensive simulations show that the proposed algorithm can effectively recover the covariance matrix of hyperspectral images from compressive measurements with high compression ratios ($8-15\%$ approx) in noisy scenarios. Moreover, simulations and theoretical results show that the filtering step reduces the recovery error up to twice the number of eigenvectors. Finally, an optical implementation is proposed, and real measurements are used to validate the theoretical findings. [ABSTRACT FROM AUTHOR]

While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems. Based on a proximal primal-dual approach, this paper presents a new (stochastic) distributed algorithm with Nesterov momentum for accelerated optimization of non-convex and non-smooth problems. Theoretically, we show that the proposed algorithm can achieve an $\epsilon$ -stationary solution under a constant step size with $\mathcal {O}(1/\epsilon ^2)$ computation complexity and $\mathcal {O}(1/\epsilon)$ communication complexity when the epigraph of the non-smooth term is a polyhedral set. When compared to the existing gradient tracking based methods, the proposed algorithm has the same order of computation complexity but lower order of communication complexity. To the best of our knowledge, the presented result is the first stochastic algorithm with the $\mathcal {O}(1/\epsilon)$ communication complexity for non-convex and non-smooth problems. Numerical experiments for a distributed non-convex regression problem and a deep neural network based classification problem are presented to illustrate the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Industrial sensor data provides significant insights into the failure risks of microgrid generation assets. In traditional applications, these sensor-driven risks are used to generate alerts that initiate maintenance actions without considering their impact on operational aspects. The focus of this paper is to propose a framework that i) builds a seamless integration between sensor data and operational & maintenance drivers and ii) demonstrates the value of this integration for improving multiple aspects of microgrid operations. The proposed framework offers an integrated stochastic optimization model that jointly optimizes operations and maintenance in a multi-microgrid setting. Maintenance decisions identify optimal crew routing, opportunistic maintenance, and repair schedules as a function of dynamically evolving sensor-driven predictions on asset life. Operational decisions identify commitment and generation from a fleet of distributed energy resources, storage, load management, as well as power transactions with the main grid and neighboring microgrids. Operational uncertainty from renewable generation, demand, and market prices are explicitly modeled through scenarios in the optimization model. We use the structure of the model to develop a decomposition-based solution algorithm to ensure computational scalability. The proposed model provides significant improvements in reliability and enhances a range of operational outcomes, including costs, renewables, generation availability, and resilience. [ABSTRACT FROM AUTHOR]

Stochastic unit commitment is an efficient method for grid operation in the presence of significant uncertainties. An example is an operation during a predicted hurricane with uncertain line out-ages. However, the solution quality comes at the cost of substantial computational burden, which makes its adoption challenging. This paper evaluates some possible ways that machine learning can be used to reduce this computational burden. First, a set of feasibility studies is conducted. Results suggest that using machine learning as an assistant to the stochastic unit commitment solver is more advantageous than using it as a standalone solver. In particular, the machine learning model is trained to facilitate solving the problem by determining the unnecessary constraints that can be removed from the original problem without affecting the final accuracy. The variables that can be used as input features/predictors or outputs for the machine learning model are determined through feasibility studies. Then, an algorithm to train and utilize a machine learning model is proposed. The method is tested on a 500-bus synthetic South Carolina system. Various test cases show an average reduction in solution time by more than 90% by using the trained machine learning model to assist the stochastic unit commitment solver. [ABSTRACT FROM AUTHOR]

In this paper a statistical modeling, based on stochastic differential equations (SDEs), is proposed for retinal Optical Coherence Tomography (OCT) images. In this method, pixel intensities of image are considered as discrete realizations of a Levy stable process. This process has independent increments and can be expressed as response of SDE to a white symmetric alpha stable (s $\boldsymbol {\alpha }\text{s}$) noise. Based on this assumption, applying appropriate differential operator makes intensities statistically independent. Mentioned white stable noise can be regenerated by applying fractional Laplacian operator to image intensities. In this way, we modeled OCT images as s $\boldsymbol {\alpha }\text{s}$ distribution. We applied fractional Laplacian operator to image and fitted s $\boldsymbol {\alpha }\text{s}$ to its histogram. Statistical tests were used to evaluate goodness of fit of stable distribution and its heavy tailed and stability characteristics. We used modeled s $\boldsymbol {\alpha }\text{s}$ distribution as prior information in maximum a posteriori (MAP) estimator in order to reduce the speckle noise of OCT images. Such a statistically independent prior distribution simplified denoising optimization problem to a regularization algorithm with an adjustable shrinkage operator for each image. Alternating Direction Method of Multipliers (ADMM) algorithm was utilized to solve the denoising problem. We presented visual and quantitative evaluation results of the performance of this modeling and denoising methods for normal and abnormal images. Applying parameters of model in classification task as well as indicating effect of denoising in layer segmentation improvement illustrates that the proposed method describes OCT data more accurately than other models that do not remove statistical dependencies between pixel intensities. [ABSTRACT FROM AUTHOR]

In this paper, a fuel-based distributed generator (DG) allocation strategy is proposed to enhance the distribution system resilience against extreme weather. The long-term planning problem is formulated as a two-stage stochastic mixed-integer programming (SMIP). The first stage is to make decisions of DG siting and sizing under the given budget constraint. In the second stage, a post-extreme-event-restoration (PEER) is employed to minimize the operating cost in an uncertain fault scenario. In particular, this study proposes a method to select the most representative scenarios for the SMIP. First, a Monte Carlo Simulation (MCS) is introduced to generate sufficient scenarios considering random fault locations and load profiles. Then, the number of scenarios is reduced by the K-means clustering algorithm. The advantage of scenario reduction is to make a trade-off between accuracy and computational efficiency. Finally, the SMIP is solved by the progressive hedging algorithm. The case studies of the IEEE 33-bus and 123-bus test systems demonstrate the effectiveness of the proposed algorithm in reducing the expected energy not served (EENS), which is a critical criterion of resilience. [ABSTRACT FROM AUTHOR]

As the training dataset size and the model size of machine learning increase rapidly, more computing resources are consumed to speedup the training process. However, the scalability and performance reproducibility of parallel machine learning training, which mainly uses stochastic optimization algorithms, are limited. In this paper, we demonstrate that the sample difference in the dataset plays a prominent role in the scalability of parallel machine learning algorithms. We propose to use statistical properties of dataset to measure sample differences. These properties include the variance of sample features, sample sparsity, sample diversity, and similarity in sampling sequences. We choose four types of parallel training algorithms as our research objects: (1) the asynchronous parallel SGD algorithm (Hogwild! algorithm), (2) the parallel model average SGD algorithm (minibatch SGD algorithm), (3) the decentralization optimization algorithm, and (4) the dual coordinate optimization (DADM algorithm). Our results show that the statistical properties of training datasets determine the scalability upper bound of these parallel training algorithms. [ABSTRACT FROM AUTHOR]

With the prosperity of vehicular networks and intelligent transport systems, vast amount of data can be easily collected by vehicular devices from their users and widely spread in the vehicular networks for the purpose of solving large-scale machine learning problems. Hence how to preserve the data privacy of users during the learning process has become a public concern. To address this concern, under the celebrated framework of differential privacy (DP), we present in this paper a decentralized parallel stochastic gradient descent (D-PSGD) algorithm, called DP2-SGD, which can offer protection for privacy of users in vehicular networks. With thorough analysis we show that DP2-SGD satisfies (ε, δ)-DP while the learning efficiency is the same as D-PSGD without privacy preservation. We also propose a refined algorithm called EC-SGD by introducing an error-compensate strategy. Extensive experiments show that EC-SGD can further improve the convergence efficiency over DP2-SGD in reality. [ABSTRACT FROM AUTHOR]

This paper proposes an algorithm for evaluating the performance of a closed-loop spare machines supply system when the time between failures as well as the repair times is generally distributed. The closed-loop system with finite number of machines and the generic time distribution assumption well represents real-life spare machines supply systems but at the cost of higher complexity. The increased complexity prohibits the use of existing models, which evaluate the performance of closed-loop systems assuming exponentially distributed failure and repair times. The Extended Bottleneck (EBOTT) algorithm in the literature estimates the performance measures of a closed-loop system with generally distributed repair service times. However, as our analysis indicates, the EBOTT algorithm may fall into an infinite loop even in assessing the performance of a simple spare machines supply system. The Modified EBOTT algorithm presented in this paper overcomes the shortcomings of the EBOTT algorithm and exhibits superior performance. Finally, through tradeoff analysis between spare machines inventory level and repair shop capacities, insights into the performance of such systems are presented. [ABSTRACT FROM AUTHOR]

The paper addresses a sequential changepoint detection problem for a general stochastic model, assuming that the observed data may be non-i.i.d. (i.e., dependent and non-identically distributed) and prior distribution of the change point is arbitrary. Tartakovsky and Veeravalli (2005), Baron and Tartakovsky (2006), and, more recently, Tartakovsky (2017) developed a general asymptotic theory of changepoint detection for non-i.i.d. stochastic models, assuming certain stability of the log-likelihood ratio process, in the case of simple hypotheses when both pre-change and post-change models are completely specified. However, in most applications, the post-change distribution is not completely known. In the present paper, we generalize previous results to the case of parametric uncertainty, assuming that the parameter of the post-change distribution is unknown. We introduce two detection rules based on mixtures–the mixture Shiryaev rule and the mixture Shiryaev–Roberts rule–and study their asymptotic properties in the Bayesian context. In particular, we provide sufficient conditions under which these rules are first-order asymptotically optimal, minimizing moments of the delay to detection as the probability of false alarm approaches zero. [ABSTRACT FROM AUTHOR]

High-dimensional and sparse (HiDS) matrices generated by recommender systems contain rich knowledge regarding various desired patterns like users’ potential preferences and community tendency. Latent factor (LF) analysis proves to be highly efficient in extracting such knowledge from an HiDS matrix efficiently. Stochastic gradient descent (SGD) is a highly efficient algorithm for building an LF model. However, current LF models mostly adopt a standard SGD algorithm. Can SGD be extended from various aspects in order to improve the resultant models’ convergence rate and prediction accuracy for missing data? Are such SGD extensions compatible with an LF model? To answer them, this paper carefully investigates eight extended SGD algorithms to propose eight novel LF models. Experimental results on two HiDS matrices generated by real recommender systems show that compared with an LF model with a standard SGD algorithm, an LF model with extended ones can achieve: 1) higher prediction accuracy for missing data; 2) faster convergence rate; and 3) model diversity. [ABSTRACT FROM AUTHOR]

Graphical data is comprised of a graph with marks on its edges and vertices. The mark indicates the value of some attribute associated to the respective edge or vertex. Examples of such data arise in social networks, molecular and systems biology, and web graphs, as well as in several other application areas. Our goal is to design schemes that can efficiently compress such graphical data without making assumptions about its stochastic properties. Namely, we wish to develop a universal compression algorithm for graphical data sources. To formalize this goal, we employ the framework of local weak convergence, also called the objective method, which provides a technique to think of a marked graph as a kind of stationary stochastic processes, stationary with respect to movement between vertices of the graph. In recent work, we have generalized a notion of entropy for unmarked graphs in this framework, due to Bordenave and Caputo, to the case of marked graphs. We use this notion to evaluate the efficiency of a compression scheme. The lossless compression scheme we propose in this paper is then proved to be universally optimal in a precise technical sense. It is also capable of performing local data queries in the compressed form. [ABSTRACT FROM AUTHOR]

In this paper, a novel policy iteration adaptive dynamic programming (ADP) algorithm is presented which is called “local policy iteration ADP algorithm” to obtain the optimal control for discrete stochastic processes. In the proposed local policy iteration ADP algorithm, the iterative decision rules are updated in a local space of the whole state space. Hence, we can significantly reduce the computational burden for the CPU in comparison with the conventional policy iteration algorithm. By analyzing the convergence properties of the proposed algorithm, it is shown that the iterative value functions are monotonically nonincreasing. Besides, the iterative value functions can converge to the optimum in a local policy space. In addition, this local policy space will be described in detail for the first time. Under a few weak constraints, it is also shown that the iterative value function will converge to the optimal performance index function of the global policy space. Finally, a simulation example is presented to validate the effectiveness of the developed method. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of selecting a Pareto set from among finite alternatives, where each alternative has multiple performance measures evaluated by stochastic simulations. Under limited simulation resources, we propose an efficient algorithm for solving this problem based on a statistical hypothesis test. Using the test, the proposed algorithm evaluates the uncertainty of each design based on the observed simulation results to identify whether the selected Pareto set is accurate. Based on the evaluated uncertainty, the algorithm assigns additional resources to the designs to maximize the accuracy of the selected Pareto set. Applying the sequential procedure, the algorithm increases the precision of the observed information selectively and gradually. Several experiments, including a practical case study, demonstrated its improved efficiency compared to the existing algorithms in the literature. This improved efficiency, along with low complexity and high robustness to noise, allows the proposed algorithm to be effectively applied to practical system designs. [ABSTRACT FROM AUTHOR]

The long-term operation planning problem is generally solved by a chain of computational models based on Stochastic Dual Dynamic Programming (SDDP) considering a period of 5–10 years ahead. A recent issue that has arisen concerning this problem is to define a convergence criterion when using conditional value-at-risk (CVaR) with SDDP due to the lack of good upper bound estimators. The main contribution of this paper is to propose a convergence criterion in which including CVaR does not hinder convergence analysis. Also, the proposed method does not increase the computational burden. Moreover, this paper provides a method that allows forward paths to be individually assessed, which can then either be discarded to reduce computational time or even replaced in an alternative resampling scheme in the SDDP. Based on aggregate reservoir representation, the proposed method of convergence was applied on long-term operation planning problems related to the Brazilian Power System. Results showed improvements in both the SDDP technique and the effectiveness of the proposed convergence criterion when CVaR was used. [ABSTRACT FROM AUTHOR]

Grid condition information extraction is very important for converter control systems in order to perform special functionalities, especially grid support during low voltage ride through (LVRT). As a part of grid condition detection, the voltage sag detection provides information about the start of a fault and obtains indices that determine the behavior of the whole control system of the converter during a fault. Estimation of the magnitude of voltage sag (MoVS) is essential for obtaining proper grid support. This paper presents an algorithm for MoVS (as defined in IEEE Std 1564-2014) prediction. The existence of correlation between magnitudes of a set of low-order harmonics during the transient of voltage PQ events with MoVS is for the first time determined and statistically proved. Correlation is mathematically formulated and the prediction function is obtained on the basis of real grid measurements. Complete algorithm for magnitude prediction is explained, and its possible application in distributed generation control is prese-nted. It is shown that significant time gain is achieved, which may be used as an additional benefit in LVRT operation. [ABSTRACT FROM AUTHOR]

Recently, several high-resolution parameter estimation algorithms have been developed to exploit the structure of strictly second-order (SO) non-circular (NC) signals. They achieve a higher estimation accuracy and can resolve up to twice as many signal sources compared to the traditional methods for arbitrary signals. As a benchmark for these NC methods, we derive the closed-form deterministic $R$ -D NC Cramér-Rao bound (NC CRB) for the multi-dimensional parameter estimation of strictly non-circular (rectilinear) signal sources in this paper. Assuming a separable centro-symmetric $R$-D array, we show that in some special cases, the deterministic $R$-D NC CRB reduces to the existing deterministic $R$-D CRB for arbitrary signals. This suggests that no gain from strictly non-circular sources (NC gain) can be achieved under the deterministic data assumption in these cases. For more general scenarios, finding an analytical expression of the NC gain for an arbitrary number of sources is very challenging. Thus, in this paper, we simplify the derived NC CRB and the existing CRB for the special case of two closely-spaced strictly non-circular sources captured by a uniform linear array (ULA). Subsequently, we use these simplified CRB expressions to analytically compute the maximum achievable asymptotic NC gain for the considered two source case. The resulting expression only depends on the various physical parameters and we find the conditions that provide the largest NC gain. Our analysis is supported by extensive simulation results. [ABSTRACT FROM PUBLISHER]

Limited range and charging infrastructure leads to range anxiety of electric vehicle drivers. Current range estimation algorithms are deemed unreliable and large safety margins are reserved to prevent the risk of stranding. Range estimation in general depends on two factors: current battery energy content and the energy consumption forecast on the route to destination. This paper aims at improving the latter by enhancing the forecast with a notion of uncertainty. The prediction algorithm itself learns from driver and traffic data in a training set to generate accurate, driver-individual energy consumption forecasts. Thereby, a central part of the algorithm is the explicit evaluation of the traffic situation by classifying the traffic phases. With the help of this methodology, individual forecasts can be made more precise since they are highly dependent on surrounding traffic. To demonstrate the validity of the algorithms, the performance is evaluated using real test drive data comprising multiple drivers. On the basis of the performance evaluation, both the superiority of stochastic algorithms over deterministic predictions and the improvement of predictive performance by evaluating explicit traffic phases can be shown. Implementing the proposed methodology in modern day electric vehicles could reduce range anxiety and ultimately increase acceptance of electric mobility worldwide. [ABSTRACT FROM AUTHOR]

Estimating the probability of failures or accidents with aerospace systems is often necessary when new concepts or designs are introduced, as it is being done for autonomous aircraft. If the design is safe, as it is supposed to be, accident cases are hard to find. Such analysis needs some variance reduction technique and several algorithms exist for that; however, specific model features may cause difficulties in practice, such as the case of system models where independent agents have to autonomously accomplish missions within finite time, and likely with the presence of human agents. For handling these scenarios, this paper presents a novel estimation approach, based on the combination of the well-established variation reduction technique of interacting particles system with the long-standing optimization algorithm denominated DIviding RECTangles. When combined, these two techniques yield statistically significant results for extremely low probabilities. In addition, this novel approach allows the identification of intermediate events and simplifies the evaluation of sensitivity of the estimated probabilities to certain system parameters. [ABSTRACT FROM AUTHOR]

Due to emerging mobile applications and an Internet of Things, data traffic carried by wireless networks has increased dramatically. As a result, maximizing the utilization of the limited network resource is a top priority. To date, there exists a large body of work on the stochastic network utility maximization (NUM) problem. On the one hand, the stochastic NUM problem and its associated solutions have been mainly investigated under the light-tailed (LT) data condition. On the other hand, empirical results show that data delivered by wireless networks is not only large in volume but also highly variable, which can be characterized by heavy-tailed (HT) distributions. Such HT traffic exhibits high burstiness and strong temporal correlations, which are fundamentally different from the behavior of LT traffic. In this paper, first it is shown that the classic stochastic gradient algorithms (SGAs), as effective solutions for stochastic NUM problems, fail to prevent HT traffic flows from aggressively competing with LT flows for limited resources. This leads to unbounded queueing delay for LT flows, which results in network instability. To counter such a challenge, the time-average stochastic gradient routing algorithm is proposed, which prevents competition among HT and LT traffic flows so that utility optimality and network stability can be achieved simultaneously. Moreover, because HT data have unbounded moments, such as mean and variance, it is not feasible to apply the conventional convergence analysis, which assumes the moment boundedness of traffic arrivals. To address this problem, the ordinary differential equation method is adopted to prove that the proposed algorithm still converges even with highly variable HT traffic. Comprehensive simulation results are shown to verify the derived theoretical results. [ABSTRACT FROM AUTHOR]

Influence Maximization (IM), which selects a set of $k$ users (called seed set) from a social network to maximize the expected number of influenced users (called influence spread), is a key algorithmic problem in social influence analysis. Due to its immense application potential and enormous technical challenges, IM has been extensively studied in the past decade. In this paper, we survey and synthesize a wide spectrum of existing studies on IM from an algorithmic perspective, with a special focus on the following key aspects: (1) a review of well-accepted diffusion models that capture the information diffusion process and build the foundation of the IM problem, (2) a fine-grained taxonomy to classify existing IM algorithms based on their design objectives, (3) a rigorous theoretical comparison of existing IM algorithms, and (4) a comprehensive study on the applications of IM techniques in combining with novel context features of social networks such as topic, location, and time. Based on this analysis, we then outline the key challenges and research directions to expand the boundary of IM research. [ABSTRACT FROM AUTHOR]

This paper studies the stochastic behavior of the LMS and NLMS algorithms in a system identification framework for a cyclostationary white input without assuming a Gaussian distribution for the input. The input cyclostationary signal is modeled by a white random process with periodically time-varying power. The system parameters vary according to a random-walk. Mathematical models are derived for the mean and mean-square-deviation behavior of the adaptive weights as a function of the input cyclostationarity. Analytical models are first derived for the LMS and NLMS algorithms for cyclostationary white inputs. These models show the dependence of the two algorithms upon the kurtosis of the input. Significant differences are found between the behaviors of the two algorithms when the analysis is applied to non-Gaussian cases. Monte Carlo simulations provide strong support for the theory. [ABSTRACT FROM AUTHOR]

This paper proposes a new distance-based distributionally robust unit commitment (DB-DRUC) model via Kullback–Leibler (KL) divergence, considering volatile wind power generation. The objective function of the DB-DRUC model is to minimize the expected cost under the worst case wind distributions restricted in an ambiguity set. The ambiguity set is a family of distributions within a fixed distance from a nominal distribution. The distance between two distributions is measured by KL divergence. The DB-DRUC model is a “min-max-min” programming model; thus, it is intractable to solve. Applying reformulation methods and stochastic programming technologies, we reformulate this “min-max-min” DB-DRUC model into a one-level model, referred to as the reformulated DB-DRUC (RDB-DRUC) model. Using the generalized Benders decomposition, we then propose a two-level decomposition method and an iterative algorithm to address the RDB-DRUC model. The iterative algorithm for the RDB-DRUC model guarantees global convergence within finite iterations. Case studies are carried out to demonstrate the effectiveness, global optimality, and finite convergence of a proposed solution strategy. [ABSTRACT FROM AUTHOR]

The history of infections and epidemics holds famous examples where understanding, containing, and ultimately treating an outbreak began with understanding its mode of spread. Influenza, HIV, and most computer viruses spread person to person, device to device, and through contact networks; Cholera, Cancer, and seasonal allergies, on the other hand, do not. In this paper, we study two fundamental questions of detection. First, given a snapshot view of a (perhaps vanishingly small) fraction of those infected, under what conditions is an epidemic spreading via contact (e.g., Influenza), distinguishable from a random illness operating independently of any contact network (e.g., seasonal allergies)? Second, if we do have an epidemic, under what conditions is it possible to determine which network of interactions is the main cause of the spread—the causative network—without any knowledge of the epidemic, other than the identity of a minuscule subsample of infected nodes? The core, therefore, of this paper, is to obtain an understanding of the diagnostic power of network information. We derive sufficient conditions that networks must satisfy for these problems to be identifiable, and produce efficient, highly scalable algorithms that solve these problems. We show that the identifiability condition we give is fairly mild, and in particular, is satisfied by two common graph topologies: the $d$ -dimensional grid, and the Erdös-Renyi graphs. [ABSTRACT FROM AUTHOR]

As net-load becomes less predictable there is a lot of pressure in changing decision models for power markets such that they account explicitly for future scenarios in making commitment decisions. This paper proposes to make commitment decisions with multiple gate closures. Our proposed model also leverages a state-space formulation for the commitment variables, through which the operational constraints of generation units participating in the market are respected. We also study the problem of constructing scenario tree approximations for stochastic processes and evaluate our algorithms on scenario tree libraries derived from real net-load data. [ABSTRACT FROM AUTHOR]

In a pricing-based spectrum trading market, an interesting issue is to study how secondary users make their selections in a rational manner when there are available spectrum of different qualities in the market. In this case, primary systems inherently require a precise pricing strategy to maximize their profits especially when the secondary users’ selection preferences are not uniform nor predictable during the dynamic access. In this paper, we propose a novel spectrum pricing mechanism for primary systems by taking into account the selection preferences of heterogeneous secondary users and quality diversity of leased spectrum caused by different interference levels and channel fading characteristics. Our work mainly focuses on exploring the optimal pricing solution in the case that the selection preference of secondary buyers is stochastic. In the proposed system model, the spectrum to be leased is divided into a series of uniform subchannels of various qualities for spectrum trading. We analyze the impact of secondary users’ preference on spectrum trading and propose an iterative algorithm for spectrum pricing by introducing the Hotelling model to describe the relationship between the spectrum diversity and the primary system's utility. Proofs of the integrability of the utility function and existence of Nash equilibrium used in the proposed algorithm are provided. Numerical results demonstrate the effectiveness of the proposed pricing method in improving the primary system's profits. [ABSTRACT FROM AUTHOR]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

This paper presents a new general method for evaluating the reliability function and the mean residual life of degrading systems subject to condition monitoring and random failure. In the proposed method, the degradation process of the system is characterized by a continuous-time Markov chain, which is then incorporated into the proportional hazards model as a stochastic covariate process to describe the hazard rate of the time to system failure. Unlike the conventional method based on conditioning, which is applicable only for a small number of degradation states, the proposed method is capable of tackling the case with a general number of degradation states. Using the developed approximation techniques, closed-form formulas for related reliability characteristics are obtained in terms of the appropriate transition probability matrix. The proposed evaluation algorithm is computationally efficient and embeddable to support real-time reliability assessment of the system subject to condition monitoring for developing the optimal maintenance policy. The effectiveness and the accuracy of the method are validated by a numerical study and compared with the conventional method. A general case where the degradation path can be discretized up to ten states is also studied to illustrate the appealing general features. [ABSTRACT FROM PUBLISHER]

This paper examines the mean-square error performance of diffusion stochastic algorithms under a generalized coordinate-descent scheme. In this setting, the adaptation step by each agent is limited to a random subset of the coordinates of its stochastic gradient vector. The selection of coordinates varies randomly from iteration to iteration and from agent to agent across the network. Such schemes are useful in reducing computational complexity at each iteration in power-intensive large data applications. They are also useful in modeling situations where some partial gradient information may be missing at random. Interestingly, the results show that the steady-state performance of the learning strategy is not always degraded, while the convergence rate suffers some degradation. The results provide yet another indication of the resilience and robustness of adaptive distributed strategies. [ABSTRACT FROM AUTHOR]

The current state-of-the-art method used for medium- and long-term planning studies of hydrothermal power system operation is the stochastic dual dynamic programming (SDDP) algorithm. The computational savings provided by this method notwithstanding, it still relies on major system simplifications to achieve acceptable performances in practical applications. In contrast with its actual implementation, simplifications in the planning stage may induce time-inconsistent policies, and consequently, a suboptimality gap. In this paper, we extend the concept of time inconsistency to measure the effects of modeling simplifications in the SDDP framework for hydrothermal operation planning. Case studies involving simplifications in transmission lines modeling and in security criteria indicate that these source of time inconsistency may result in unexpected reservoir depletion and spikes in energy market spot prices. [ABSTRACT FROM PUBLISHER]

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g., a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both nonadaptive and adaptive filter identification problems. [ABSTRACT FROM PUBLISHER]

Distributed estimation of the average value over a Wireless Sensor Network has recently received a lot of attention. Most papers consider single variable sensors and communications with feedback (e.g., peer-to-peer communications). However, in order to use efficiently the broadcast nature of the wireless channel, communications without feedback are advocated. To ensure the convergence in this feedback-free case, the recently-introduced Sum-Weight-like algorithms which rely on two variables at each sensor are a promising solution. In this paper, the convergence towards the consensus over the average of the initial values is analyzed in depth. Furthermore, it is shown that the squared error decreases exponentially with the time. In addition, a powerful algorithm relying on the Sum-Weight structure and taking into account the broadcast nature of the channel is proposed. [ABSTRACT FROM PUBLISHER]

The low-rank matrix completion problem is fundamental to a number of tasks in data mining, machine learning, and signal processing. This paper considers the problem of adaptive matrix completion in time-varying scenarios. Given a sequence of incomplete and noise-corrupted matrices, the goal is to recover and track the underlying low rank matrices. Motivated from the classical least-mean square (LMS) algorithms for adaptive filtering, three LMS-like algorithms are proposed for estimating and tracking low-rank matrices. Performance of the proposed algorithms is provided in form of nonasymptotic bounds on the tracking mean-square error. Tracking performance of the algorithms is also studied via detailed simulations over real-world datasets. [ABSTRACT FROM PUBLISHER]

In this paper, we study Stochastic Composite Optimization (SCO) for sparse learning that aims to learn a sparse solution from a composite function. Most of the recent SCO algorithms have already reached the optimal expected convergence rate \mathcal O(1/\lambda T) , but they often fail to deliver sparse solutions at the end either due to the limited sparsity regularization during stochastic optimization (SO) or due to the limitation in online-to-batch conversion. Even when the objective function is strongly convex, their high probability bounds can only attain \mathcal O(\sqrt\log (1/\delta)/T) with $\delta$ is the failure probability, which is much worse than the expected convergence rate. To address these limitations, we propose a simple yet effective two-phase Stochastic Composite Optimization scheme by adding a novel powerful sparse online-to-batch conversion to the general Stochastic Optimization algorithms. We further develop three concrete algorithms, OptimalSL, LastSL and AverageSL, directly under our scheme to prove the effectiveness of the proposed scheme. Both the theoretical analysis and the experiment results show that our methods can really outperform the existing methods at the ability of sparse learning and at the meantime we can improve the high probability bound to approximately \mathcal {O}(\log (\log (T)/\delta)/\lambda T) . [ABSTRACT FROM AUTHOR]

For increased efficiency and profitability, many processes have multiple modes of operation. Switching between different operating modes is performed according to the standard operating procedures. These procedures are set by considering safety and operating limitations of various subsystems and equipment, and thus put restrictions on the switching of the process modes. In this paper, a hidden Markov model based monitoring method is proposed that can not only handle the multimodality of process data but can also capture the mode switching restrictions. A two-step Viterbi algorithm is proposed for effective mode detection in the event of faults, and a reconstruction-based fault isolation algorithm is developed to build the contribution plots. Application examples demonstrate the effectiveness of the proposed monitoring method. [ABSTRACT FROM PUBLISHER]

In this paper, we present a new effective methodology to build stochastic macromodels for the time-domain analysis of generic linear multiport systems. The proposed technique allows one to calculate a stable and passive polynomial chaos (PC)-based macromodel of a system under stochastic variations. The Galerkin projections method and a PC-based model of the system scattering parameters are used along with the Vector Fitting algorithm, leading to an accurate and efficient description of the system variability features. The proposed technique results to be very versatile and, thus, is well suited to be applied to many different and complex modern electrical systems (e.g., interconnections and filters). The accuracy and computational efficiency of the proposed technique are verified through comparison with the standard Monte Carlo analysis for two pertinent numerical examples, showing a maximum simulation speedup of 765 times. [ABSTRACT FROM AUTHOR]

This paper studies the stochastic behavior of the signed variants of the LMS algorithm for a system identification framework when the input signal is a cyclostationary white Gaussian process. Three algorithms are studied: the signed regressor, the signed error, and the sign–sign algorithms. The input cyclostationary signal is modeled by a white Gaussian random process with periodically time-varying power. The system parameters vary according to a random-walk. Mathematical models are derived for the mean and mean-square-deviation behavior of the adaptive weights with the input cyclostationarity. These models are used to derive new results concerning the performance of the algorithms. Some of these results are surprising. Monte Carlo simulations of the three algorithms provide strong support for the theory. [ABSTRACT FROM PUBLISHER]