Search

In this paper, a two-layer recurrent neural network is proposed to solve the nonsmooth convex optimization problem subject to convex inequality and linear equality constraints. Compared with existing neural network models, the proposed neural network has a low model complexity and avoids penalty parameters. It is proved that from any initial point, the state of the proposed neural network reaches the equality feasible region in finite time and stays there thereafter. Moreover, the state is unique if the initial point lies in the equality feasible region. The equilibrium point set of the proposed neural network is proved to be equivalent to the Karush–Kuhn–Tucker optimality set of the original optimization problem. It is further proved that the equilibrium point of the proposed neural network is stable in the sense of Lyapunov. Moreover, from any initial point, the state is proved to be convergent to an equilibrium point of the proposed neural network. Finally, as applications, the proposed neural network is used to solve nonlinear convex programming with linear constraints and L1 -norm minimization problems. [ABSTRACT FROM AUTHOR]

Purpose: Heuristic algorithms have been widely used in different types of optimization problems. Their unique features in terms of running time and flexibility have made them superior to deterministic algorithms. To accurately compare different heuristic algorithms in solving optimization problems, the final optimal solution needs to be known. Existing deterministic methods such as Exhaustive Search and Integer Linear Programming can provide the final global optimal solution for small-scale optimization problems. However, as the system grows the number of calculations and required memory size incredibly increases, so applying existing deterministic methods is no longer possible for medium and large-scale systems. The purpose of this paper is to introduce a novel deterministic method with short running time and small memory size requirement for optimal placement of Micro Phasor Measurement Units (µPMUs) in radial electricity distribution systems to make the system completely observable. Design/methodology/approach: First, the principle of the method is explained and the observability of the system is analyzed. Then, the algorithm's running time and memory usage when applying on some of the modified versions of the Institute of Electrical and Electronics Engineers 123-node test feeder are obtained and compared with those of its deterministic counterparts. Findings: Because of the innovative method of step-by-step placement of µPMUs, a unique method is developed. Simulation results elucidate that the proposed method has unique features of short running time and small memory size requirements. Originality/value: While the mathematical background of the observability study of electricity distribution systems is very well-presented in the referenced papers, the proposed step-by-step placement method of µPMUs, which shrinks unobservable parts of the system in each step, is not discussed yet. The presented paper is directly applicable to typical problems in the field of power systems. [ABSTRACT FROM AUTHOR]

Purpose: This paper aims to consider both the greenhouse gas (GHG) emissions and behavioural response in the optimal sizing of solar photovoltaic systems (PV modules and batteries) for energy communities. The objective is to achieve a high self-sufficiency rate whilst taking into account the grid carbon intensity and the global warming potential of system components. Design/methodology/approach: Operation and sizing of energy communities leads to optimization problems spanning across multiple timescales. To compute the optimisation in a reasonable time, the authors first apply a simulation periods reduction using a clustering approach, before solving a linear programming problem. Findings: The results show that the minimum GHG emissions is achieved for self-sufficiency rates of 19% in France and 50% in Germany. Research limitations/implications: The analysis is restricted to specific residential profiles: further work will focus on exploring different types of consumption profiles. Practical implications: This paper provides relevant self-sufficiency orders of magnitude for energy communities. Originality/value: This paper combines various approaches in a single use case: environmental considerations, behavioural response as well as multi-year energy system sizing. [ABSTRACT FROM AUTHOR]

Purpose: In this study, the authors investigate a different routing and scheduling problem in the field of home health care (HHC) management system. The purpose of this paper is to route and schedule the workday of health workers, assign the patients to suitable health workers, make accurate decisions to minimize costs, provide timely services and, in general, enhance the efficiency of HHC centers. Design/methodology/approach: A mixed-integer linear programming model is developed to assign health workers to patients. The model considers health professionals with different skills, namely nurses and physicians. Additionally, three groups of patients are considered: patients who need a nurse, patients who need a physician and patients who need both. In the third group, the nurse must be present at the patient's home following the physician's visit in order to perform the required tasks. Findings: The results of this study show a reduction in costs which results from the fewer health workers employed and dispatched in comparison with traditional approaches. With the help of our solution approach and model, HHC centers may not only successfully reduce their costs but also manage to meet their patients' demands by assigning suitable nurses and physicians. Originality/value: Previous studies have often focused on problems involving only one group of health professionals and rarely address problems involving multiple groups. The authors consider this a shortcoming, because in many cases, patients should be visited several times and by various health professionals. [ABSTRACT FROM AUTHOR]

We study distributed optimization to minimize a sum of smooth and strongly-convex functions. Recent work on this problem uses gradient tracking to achieve linear convergence to the exact global minimizer. However, a connection among different approaches has been unclear. In this paper, we first show that many of the existing first-order algorithms are related with a simple state transformation, at the heart of which lies a recently introduced algorithm known as  $\mathcal {AB}$. We then present distributed heavy-ball, denoted as  $\mathcal {AB}m$ , that combines  $\mathcal {AB}$ with a momentum term and uses nonidentical local step-sizes. By simultaneously implementing both row- and column-stochastic weights,  $\mathcal {AB}m$ removes the conservatism in the related work due to doubly stochastic weights or eigenvector estimation.  $\mathcal {AB}m$ thus naturally leads to optimization and average consensus over both undirected and directed graphs. We show that  $\mathcal {AB}m$ has a global $R$ -linear rate when the largest step-size and momentum parameter are positive and sufficiently small. We numerically show that  $\mathcal {AB}m$ achieves acceleration, particularly when the objective functions are ill-conditioned. [ABSTRACT FROM AUTHOR]

This paper presents the stability and criticality analysis of integer linear programs with respect to perturbations in stochastic data given as Markov chains. These perturbations affect the initial distribution, the transition matrix, or the stationary distribution of Markov chains. Stability analysis is concerned with obtaining the set of all perturbations for which a solution remains optimal. This paper gives expressions for stability regions for perturbations in the initial distribution, the transition matrix, the stationary distribution, and the product of elements of the transition matrix and the stationary distribution. Furthermore, criticality measures that describe the sensitivity of the objective function with respect to an element of the problem data are derived. Stability regions that preserve the stochasticity of the problem data are given. Finally, stability regions for perturbations of elements of the transition matrix, given that the problem is not linear in the initial distribution or the transition matrix, are obtained using a small perturbation analysis. The results are applied to sensor placement problems and numerical examples are given. [ABSTRACT FROM PUBLISHER]

The paper considers normalized gradient descent (NGD), a natural modification of classical gradient descent (GD) in optimization problems. It is shown that, contrary to GD, NGD escapes saddle points “quickly.” A serious shortcoming of GD in nonconvex problems is that it can take arbitrarily long to escape from the neighborhood of a saddle point. In practice, this issue can significantly slow the convergence of GD, particularly in high-dimensional nonconvex problems. The paper focuses on continuous-time dynamics. It is shown that 1) NGD “almost never” converges to saddle points and 2) the time required for NGD to escape from a ball of radius $r$ about a saddle point $x^*$ is at most $5\sqrt{\kappa }r$ , where $\kappa$ is the condition number of the Hessian of $f$ at $x^*$. As a simple application of these results, a global convergence-time bound is established for NGD under mild assumptions. [ABSTRACT FROM AUTHOR]

This paper considers the design of digital linear-phase finite-length impulse response (FIR) filters that have adjustable bandwidth(s) whereas the phase response is fixed. For this purpose, a structure is employed in which the overall transfer function is a weighted linear combination of fixed subfilters and where the weights are directly determined by the bandwidth(s). Minimax design techniques are introduced which generate globally optimal overall filters in the minimax (Chebyshev) sense over a whole set of filter specifications. The paper also introduces a new structure for bandstop and bandpass filters with individually adjustable upper and lower band edges, and with a substantially lower arithmetic complexity compared to structures that make use of two separate adjustable-bandwidth low-pass and high-pass filters in cascade or in parallel. Design examples are included in the paper. [ABSTRACT FROM AUTHOR]

Bounded component analysis (BCA) is a framework that can be considered as a more general framework than independent component analysis (ICA) under the boundedness constraint on sources. Using this framework, it is possible to separate dependent as well as independent components from their mixtures. In this paper, as an extension of a recently introduced instantaneous BCA approach, we introduce a family of convolutive BCA criteria and corresponding algorithms. We prove that the global optima of the proposed criteria, under generic BCA assumptions, are equivalent to a set of perfect separators. The algorithms introduced in this paper are capable of separating not only the independent sources but also the sources that are dependent/correlated in both component (space) and sample (time) dimensions. Therefore, under the condition that the sources are bounded, they can be considered as extended convolutive ICA algorithms with additional dependent/correlated source separation capability. Furthermore, they have potential to provide improvement in separation performance, especially for short data records. This paper offers examples to illustrate the space-time correlated source separation capability through a copula distribution-based example. In addition, a frequency-selective Multiple Input Multiple Output equalization example demonstrates the clear performance advantage of the proposed BCA approach over the state-of-the-art ICA-based approaches in setups involving convolutive mixtures of digital communication sources. [ABSTRACT FROM PUBLISHER]

Sparse representation has shown an attractive performance in a number of applications. However, the available sparse representation methods still suffer from some problems, and it is necessary to design more efficient methods. Particularly, to design a computationally inexpensive, easily solvable, and robust sparse representation method is a significant task. In this paper, we explore the issue of designing the simple, robust, and powerfully efficient sparse representation methods for image classification. The contributions of this paper are as follows. First, a novel discriminative sparse representation method is proposed and its noticeable performance in image classification is demonstrated by the experimental results. More importantly, the proposed method outperforms the existing state-of-the-art sparse representation methods. Second, the proposed method is not only very computationally efficient but also has an intuitive and easily understandable idea. It exploits a simple algorithm to obtain a closed-form solution and discriminative representation of the test sample. Third, the feasibility, computational efficiency, and remarkable classification accuracy of the proposed l2 regularization-based representation are comprehensively shown by extensive experiments and analysis. The code of the proposed method is available at http://www.yongxu.org/lunwen.html. [ABSTRACT FROM AUTHOR]

The existing, semisupervised, spectral clustering approaches have two major drawbacks, i.e., either they cannot cope with multiple categories of supervision or they sometimes exhibit unstable effectiveness. To address these issues, two normalized affinity and penalty jointly constrained spectral clustering frameworks as well as their corresponding algorithms, referred to as type-I affinity and penalty jointly constrained spectral clustering (TI-APJCSC) and type-II affinity and penalty jointly constrained spectral clustering (TII-APJCSC), respectively, are proposed in this paper. TI refers to type-I and TII to type-II. The significance of this paper is fourfold. First, benefiting from the distinctive affinity and penalty jointly constrained strategies, both TI-APJCSC and TII-APJCSC are substantially more effective than the existing methods. Second, both TI-APJCSC and TII-APJCSC are fully compatible with the three well-known categories of supervision, i.e., class labels, pairwise constraints, and grouping information. Third, owing to the delicate framework normalization, both TI-APJCSC and TII-APJCSC are quite flexible. With a simple tradeoff factor varying in the small fixed interval (0, 1], they can self-adapt to any semisupervised scenario. Finally, both TI-APJCSC and TII-APJCSC demonstrate strong robustness, not only to the number of pairwise constraints but also to the parameter for affinity measurement. As such, the novel TI-APJCSC and TII-APJCSC algorithms are very practical for medium- and small-scale semisupervised data sets. The experimental studies thoroughly evaluated and demonstrated these advantages on both synthetic and real-life semisupervised data sets. [ABSTRACT FROM AUTHOR]

This paper presents an adaptive event-triggered filter of positive Markovian jump systems based on disturbance observer. A new adaptive event-triggering mechanism is constructed for the systems. A positive disturbance observer is designed for the systems to estimate the disturbance. A distributed output model of each subsystem of positive Markovian jump systems is introduced. Then, an adaptive event-triggering distributed filter is designed by employing stochastic copositive Lyapunov functions. All presented conditions are solvable in terms of linear programming. Under the designed disturbance observer and the distributed filter, the corresponding error system is stochastically stable. The filter design approach is also developed for discrete-time positive Markovian jump systems. The contribution of the paper lies in that: (i) A new adaptive event-triggering mechanism is established for positive systems, (ii) A positive disturbance observer is designed for the disturbance of positive Markovian jump systems, and (iii) The designed distributed filter can guarantee the stochastic stability of the error while existing filters in literature only achieve the stochastic gain stability of the error. Finally, two examples are given to illustrate the effectiveness of the proposed design. [ABSTRACT FROM AUTHOR]

This paper determines the weight decay parameter value of a deep convolutional neural network (CNN) that yields a good generalization. To obtain such a CNN in practice, numerical trials with different weight decay values are needed. However, the larger the CNN architecture is, the higher is the computational cost of the trials. To address this problem, this paper formulates an analytical solution for the decay parameter through a proposed objective function in conjunction with Bayesian probability distributions. For computational efficiency, a novel method to approximate this solution is suggested. This method uses a small amount of information in the Hessian matrix. Theoretically, the approximate solution is guaranteed by a provable bound and is obtained by a proposed algorithm, where its time complexity is linear in terms of both the depth and width of the CNN. The bound provides a consistent result for the proposed learning scheme. By reducing the computational cost of determining the decay value, the approximation allows for the fast investigation of a deep CNN (DCNN) which yields a small generalization error. Experimental results show that our assumption verified with different DCNNs is suitable for real-world image data sets. In addition, the proposed method significantly reduces the time cost of learning with setting the weight decay parameter while achieving good classification performances. [ABSTRACT FROM AUTHOR]

This paper is concerned with the power system state estimation (PSSE) problem that aims to find the unknown operating point of a power network based on a given set of measurements. We first study the power flow (PF) problem as an important special case of PSSE. PF is known to be nonconvex and NP-hard in the worst case. To this end, we propose a set of semidefinite programs (SDPs) with the property that they all solve the PF problem as long as the voltage angles are relatively small. Associated with each SDP, we explicitly characterize the set of all the complex voltages that can be recovered via that convex problem. As a generalization, the design of an SDP problem that recovers multiple nominal points and a neighborhood around each point is also cast as a convex program. The results are, then, extended to the PSSE problem, where the measurements used in the PF problem are subject to noise. A two-term objective function is employed for each convex program developed for the PSSE problem: 1) the first term accounting for the nonconvexity of the PF equations and 2) other one for estimating the noise levels. An upper bound on the estimation error is derived with respect to the noise level, and the proposed techniques are demonstrated on multiple test systems, including a 9241-bus European network. Although the focus of this paper is on power networks, yet the developed results apply to every arbitrary state estimation problem with quadratic measurement equations. [ABSTRACT FROM AUTHOR]

Traffic congestion often incurs environmental problems. One of the most effective ways to mitigate this is carpooling transportation, which substantially reduces automobile demands. Due to the popularization of smartphones and mobile applications, a carpool service can be conveniently accessed via the intelligent carpool system. In this system, the service optimization required to intelligently and adaptively distribute the carpool participant resources is called the carpool service problem (CSP). Several previous studies have examined viable and preliminary solutions to the CSP by using exact and metaheuristic optimization approaches. For CSP-solving, evolutionary computation (e.g., metaheuristics) is a more promising option in comparison to exact-type approaches. However, all the previous state-of-the-art approaches use pure optimization to solve the CSP. In this paper, we employ the framework of neuroevolution to propose the self-organizing map-based neuroevolution (SOMNE) solver by which the SOM-like network represents the abstract CSP solution and is well-trained by using neural learning and evolutionary mechanism. The experimental section of this paper investigates the comparisons and analyses of two objective functions of the CSP and demonstrates that the proposed SOMNE solver achieves superior results when compared against those the other approaches produce, especially in regard to the optimization of the primary objective functions of the CSP. Finally, the visual results of the SOM are illustrated to show the effectiveness and efficiency of the evolutionary neural learning process. [ABSTRACT FROM AUTHOR]

Conservation principles, such as conservation of charge, energy, or mass, provide a natural way to couple and constrain spatially separated variables. In this paper, we propose a dynamical system model that exploits these constraints for solving nonconvex and discrete global optimization problems. Unlike the traditional simulated annealing or quantum annealing-based global optimization techniques, the proposed method optimizes a target objective function by continuously evolving a driver functional over a conservation manifold, using a generalized variant of growth transformations. As a result, the driver functional asymptotically converges toward a Dirac-delta function that is centered at the global optimum of the target objective function. In this paper, we provide an outline of the proof of convergence for the dynamical system model and investigate different properties of the model using a benchmark nonlinear optimization problem. Also, we demonstrate how a discrete variant of the proposed dynamical system can be used for implementing decentralized optimization algorithms, where an ensemble of spatially separated entities (for example, biological cells or simple computational units) can collectively implement specific functions, such as winner-take-all and ranking, by exchanging signals only with its immediate substrate or environment. The proposed dynamical system model could potentially be used to implement continuous-time optimizers, annealers, and neural networks. [ABSTRACT FROM AUTHOR]

This paper provides performance results using the SPERRY UNIVAC 1100 Series linear programming product FMPS to solve a set of 16 real-world linear programming problems. As such, this paper provides a data point for the actual performance of a commercial simplex algorithm on real-world linear programming problems and shows that the simplex algorithm is a linear time algorithm in actual performance. Correlations and performance relationships not previously available are also provided. [ABSTRACT FROM AUTHOR]

In this paper, we consider the l\infty gain characterizations of linear switched systems (LSS) and present various relevant results on their exact computation and optimization. Depending on the role of the switching sequence, we study two broad cases: first, when the switching sequence attempts to maximize, and second, when it attempts to minimize the l\infty gain. The first, named as worst-case throughout the paper, can be related to robustness of the system to uncontrolled switching; the second relates to situations when the switching can be part to the overall decision making. Although, in general, the exact computation of l\infty gains is difficult, we provide specific classes, the input-output switching systems, for which it is shown that linear programming can be used to obtain the worst-case l\infty gain. This is a sufficiently rich class of systems as any stable LSS can be approximated by one. Certain applications to robust control design are provided where we show that a switched compensation independently of the plant has no advantage over a linear time invariant (LTI) compensation, and further, if the plant is strictly causal, even a switched compensation which has a matched switching with the plant does not provide a better performance over an LTI compensation. Also, we present a new necessary and sufficient condition to check the stability of LSS in form of a model matching problem. On the other hand, if one is interested in minimizing the l\infty gain over the switching sequences, we show that, for finite impulse response (FIR) switching systems the minimizing switching sequence can be chosen to be periodic. For input-only or output-only switching an exact, readily computable, characterization of the minimal l\infty gain is provided, and it is shown that the minimizing switching sequence is constant, which, as also shown, is not true for input-output switching. [ABSTRACT FROM AUTHOR]

In this paper, we apply efficient implementations of integer linear programming to the problem of image segmentation. The image is first grouped into superpixels and then local information is extracted for each pair of spatially adjacent superpixels. Given local scores on a map of several hundred superpixels, we use correlation clustering to find the global segmentation that is most consistent with the local evidence. We show that, although correlation clustering is known to be NP-hard, finding the exact global solution is still feasible by breaking the segmentation problem down into subproblems. Each such sub-problem can be viewed as an automatically detected image part. We can further accelerate the process by using the cutting-plane method, which provides a hierarchical structure of the segmentations. The efficiency and improved performance of the proposed method is compared to several state-of-the-art methods and demonstrated on several standard segmentation data sets. [ABSTRACT FROM PUBLISHER]

This paper addresses cooperative environmental monitoring for Pan-Tilt-Zoom (PTZ) visual sensor networks. In particular, we investigate the optimal monitoring problem whose objective function value is intertwined with the uncertain state of the physical world. In addition, due to the large volume of vision data, it is desired for each sensor to execute processing through local computation and communication. To address these issues, we present a distributed solution to the problem based on game theoretic cooperative control and payoff-based learning. At the first stage, a utility function is designed so that the resulting game constitutes a potential game with potential function equal to the group objective function, where the designed utility is shown to be computable through local image processing and communication. Then, we present a payoff-based learning algorithm so that the sensors are led to the global objective function maximizers without using any prior information on the environmental state. Finally, we run experiments to demonstrate the effectiveness of the present approach. [ABSTRACT FROM PUBLISHER]

This paper studies the problem of minimizing a sum of (possible nonsmooth) convex functions that are corresponding to multiple interacting nodes, subject to a convex state constraint set. Time-varying directed network is considered here. Two types of computational constraints are investigated in this paper: one where the information of gradients is not available and the other where the projection steps can only be calculated approximately. We devise a distributed zeroth-order method, the implementation of which needs only functional evaluations and approximate projection. In particular, we show that the proposed method generates expected function value sequences that converge to the optimal value, provided that the projection errors decrease at appropriate rates. [ABSTRACT FROM AUTHOR]

This paper presents a simple linear programming (LP) technique for designing high-accuracy low-complexity finite-impulse-response (FIR) variable fractional-delay (VFD) digital filters in the minimax error sense. The objective of the minimax design is to minimize the maximum absolute error of the variable frequency response (VFR) of an FIR VFD filter, which is a nonlinear problem and difficult to solve. This paper shows that the minimax design can be approximately decomposed into a pair of separate LP subproblems by decoupling the minimization of the real-part VFR error from that of the imaginary-part error. As a result, the original nonlinear minimax design problem can be easily solved by solving the two LP subproblems separately. To reduce the VFD filter complexity, we also propose a one-by-one increase scheme for optimizing the subfilter orders in the Farrow structure such that a given design specification (maximum absolute error of VFR) can be exactly satisfied. Both even-order and odd-order design examples are given to illustrate that the decoupling minimax method is not only simple, but also can achieve excellent high-accuracy low-complexity FIR VFD filters. [ABSTRACT FROM PUBLISHER]

This paper proposes noniterative and iterative linear programming (LP) procedures for designing low-complexity allpass variable fractional-delay (VFD) digital filters in the minimax sense. Expressing each coefficient of an allpass VFD filter as a polynomial in the VFD parameter p, we show that the frequency response error of an allpass VFD filter can be written as a pure imaginary part divided by its denominator. Thus, the minimax design can be approximately formulated as an LP problem through neglecting the denominator, which leads to a noniterative minimax design. To improve the minimax design accuracy, we propose an iterative LP procedure that utilizes the denominator from the preceding iteration as a known. The iterative LP minimization is repeated until it converges to the minimax solution. Moreover, we also present a two-stage algorithm for optimizing the optimal variable range p \in [pMin, pMax] of the VFD parameter p and successively reducing the subfilter orders. Design examples are given to show that both noniterative and iterative LP methods can achieve much better minimax designs (smaller peak errors) than the existing iterative weighted-least-squares (WLS) approaches, which aim to minimize the peak errors of VFD response and variable phase response, respectively. Moreover, the resulting allpass VFD filters have lower complexities than those from the iterative WLS approaches. [ABSTRACT FROM PUBLISHER]

Continuous-time and discrete-time forms of Zhang dynamics (ZD) for time-varying nonlinear optimization have been developed recently. In this paper, a novel discrete-time ZD (DTZD) algorithm is proposed and investigated based on the previous research. Specifically, the DTZD algorithm for time-varying nonlinear optimization is developed by adopting a new Taylor-type difference rule. This algorithm is a five-step iteration process, and thus, is referred to as the five-step DTZD algorithm in this paper. Theoretical analysis and results of the proposed five-step DTZD algorithm are presented to highlight its excellent computational performance. The geometric representation of the proposed algorithm for time-varying nonlinear optimization is also provided. Comparative numerical results are illustrated with four examples to substantiate the efficacy and superiority of the proposed five-step DTZD algorithm for time-varying nonlinear optimization compared with the previous DTZD algorithms. [ABSTRACT FROM AUTHOR]

The vast majority of dimensionality reduction (DR) techniques rely on the second-order statistics to define their optimization objective. Even though this provides adequate results in most cases, it comes with several shortcomings. The methods require carefully designed regularizers and they are usually prone to outliers. In this paper, a new DR framework that can directly model the target distribution using the notion of similarity instead of distance is introduced. The proposed framework, called similarity embedding framework (SEF), can overcome the aforementioned limitations and provides a conceptually simpler way to express optimization targets similar to existing DR techniques. Deriving a new DR technique using the SEF becomes simply a matter of choosing an appropriate target similarity matrix. A variety of classical tasks, such as performing supervised DR and providing out-of-sample extensions, as well as, new novel techniques, such as providing fast linear embeddings for complex techniques, are demonstrated in this paper using the proposed framework. Six data sets from a diverse range of domains are used to evaluate the proposed method and it is demonstrated that it can outperform many existing DR techniques. [ABSTRACT FROM AUTHOR]

In this paper, we analyze the 1-periodic schedule of a class of time interval models under the form of a polyhedron which can describe Timed Event Graphs and P-time Event Graphs. Using the duality and Stiemke's theorem, the main contribution is the determination of conditions where the extremum cycle times are finite and characteristic of a class of models. [ABSTRACT FROM AUTHOR]

Hashing is emerging as a powerful tool for building highly efficient indices in large-scale search systems. In this paper, we study spectral hashing (SH), which is a classical method of unsupervised hashing. In general, SH solves for the hash codes by minimizing an objective function that tries to preserve the similarity structure of the data given. Although computationally simple, very often SH performs unsatisfactorily and lags distinctly behind the state-of-the-art methods. We observe that the inferior performance of SH is mainly due to its imperfect formulation; that is, the optimization of the minimization problem in SH actually cannot ensure that the similarity structure of the high-dimensional data is really preserved in the low-dimensional hash code space. In this paper, we, therefore, introduce reversed SH (ReSH), which is SH with its input and output interchanged. Unlike SH, which estimates the similarity structure from the given high-dimensional data, our ReSH defines the similarities between data points according to the unknown low-dimensional hash codes. Equipped with such a reversal mechanism, ReSH can seamlessly overcome the drawback of SH. More precisely, the minimization problem in our ReSH can be optimized if and only if similar data points are mapped to adjacent hash codes, and mostly important, dissimilar data points are considerably separated from each other in the code space. Finally, we solve the minimization problem in ReSH by multilayer neural networks and obtain state-of-the-art retrieval results on three benchmark data sets. [ABSTRACT FROM AUTHOR]

In this paper, we consider the regularized version of the Jacobi algorithm, a block coordinate descent method for convex optimization with an objective function consisting of the sum of a differentiable function and a block-separable function. Under certain regularity assumptions on the objective function, this algorithm has been shown to satisfy the so-called sufficient decrease condition, and consequently, to converge in objective function value. In this paper, we revisit the convergence analysis of the regularized Jacobi algorithm and show that it also converges in iterates under very mild conditions on the objective function. Moreover, we establish conditions under which the algorithm achieves a linear convergence rate. [ABSTRACT FROM PUBLISHER]

An integer linear programming-based framework to identify the current-mode threshold logic functions is presented. The approach minimizes the transistor count and benefits from a generalized definition of threshold logic functions. It is shown that the threshold logic functions can be implemented in CMOS-based current mode logic with reduced transistor count when the input weights are not restricted to be integers. A novel implementation of rational weights is proposed. Process variations, transistor aging, and circuit parasitics are taken into consideration. Experimental results show that many more functions can be implemented with predetermined hardware overhead, and the hardware requirement of a large percentage of the existing threshold functions is reduced when comparing with the traditional CMOS-based threshold logic implementation. [ABSTRACT FROM AUTHOR]

This paper explores optimal control problems with $l_1$ objectives involving linear discrete-time systems. These problems can be efficiently solved as linear programs. They also have previously been shown to yield sparse solutions, including idle or deadbeat solutions where the input or output is respectively zero along the entire control horizon. The main contribution of this paper is to derive conditions on the problem parameters that specify when idle or deadbeat solutions occur. These results, based on analyzing the dual problem, demonstrate how different types of sparse solutions result from the choice of the problem parameters and, as a consequence, may guide the design of controllers employing $l_1$ objectives. [ABSTRACT FROM PUBLISHER]

Structural support vector machine (SVM) is an elegant approach for building complex and accurate models with structured outputs. However, its applicability relies on the availability of efficient inference algorithms—the state-of-the-art training algorithms repeatedly perform inference to compute a subgradient or to find the most violating configuration. In this paper, we propose an exact inference algorithm for maximizing nondecomposable objectives due to special type of a high-order potential having a decomposable internal structure. As an important application, our method covers the loss augmented inference, which enables the slack and margin scaling formulations of structural SVM with a variety of dissimilarity measures, e.g., Hamming loss, precision and recall, F\beta -loss, intersection over union, and many other functions that can be efficiently computed from the contingency table. We demonstrate the advantages of our approach in natural language parsing and sequence segmentation applications. [ABSTRACT FROM PUBLISHER]

Heterogeneous multi-criteria group decision making (MCGDM) is a hot topic in the decision analysis field. This paper proposes a fairness-concern-based LINMAP (Linear Programming Technique for Multidimensional Analysis of Preference) method for heterogeneous MCGDM with hesitant fuzzy linguistic (HFL) truth degrees. Heterogeneous evaluation information includes crisp numbers, interval numbers, intuitionistic fuzzy values (IFVs), trapezoidal fuzzy numbers (TrFNs) and hesitant fuzzy sets (HFSs). This paper introduces the fairness concern to calculate the HFL consistency and the HFL inconsistency indices. Based on the framework of LINMAP, a bi-objective HFL programming model is built to derive the criteria weights, the positive ideal fairness vector (PIFV) and the negative ideal fairness vector (NIFV) for each decision maker (DM) simultaneously. Based on the TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution), a multi-objective programming model is built to obtain DMs' weights. The alternatives ranking is derived by comprehensive collective relative closeness degrees. Finally, a real example is applied to verify effectiveness and superiority of this heterogeneous MCGDM method. The proposed heterogeneous MCGDM method provides a very useful approach for MCGDM with heterogeneous information. [ABSTRACT FROM AUTHOR]

This paper develops a neural network architecture and a new processing method for solving in real time, the nonlinear equality constrained multiobjective optimization problem (NECMOP), where several nonlinear objective functions must be optimized in a conflicting situation. In this processing method, the NECMOP is converted to an equivalent scalar optimization problem (SOP). The SOP is then decomposed into several-separable subproblems processable in parallel and in a reasonable time by multiplexing switched capacitor circuits. The approach which we propose makes use of a decomposition–coordination principle that allows nonlinearity to be treated at a local level and where coordination is achieved through the use of Lagrange multipliers. The modularity and the regularity of the neural networks architecture herein proposed make it suitable for very large scale integration implementation. An application to the resolution of a physical problem is given to show that the approach used here possesses some advantages of the point of algorithmic view, and provides processes of resolution often simpler than the usual techniques. [ABSTRACT FROM PUBLISHER]

Finding the optimal solution to the constrained l0 -norm minimization problems in the recovery of compressive sensed signals is an NP-hard problem and it usually requires intractable combinatorial searching operations for getting the global optimal solution, unless using other objective functions (e.g., the l1 norm or lp norm) for approximate solutions or using greedy search methods for locally optimal solutions (e.g., the orthogonal matching pursuit type algorithms). In this paper, a neurodynamic optimization method is proposed to solve the l0 -norm minimization problems for obtaining the global optimum using a recurrent neural network (RNN) model. For the RNN model, a group of modified Gaussian functions are constructed and their sum is taken as the objective function for approximating the l0 norm and for optimization. The constructed objective function sets up a convexity condition under which the neurodynamic system is guaranteed to obtain the globally convergent optimal solution. An adaptive adjustment scheme is developed for improving the performance of the optimization algorithm further. Extensive experiments are conducted to test the proposed approach in this paper and the output results validate the effectiveness of the new method. [ABSTRACT FROM PUBLISHER]

Network embedding is capable of providing low-dimensional feature representations for various machine learning applications. Current work focuses on: 1) designing the embedding as an unsupervised learning task to explicitly preserve the structural connectivity in the network or 2) generating the embedding as a by-product during the supervised learning of a specific discriminative task in a deep neural network. In this paper, we aim to take advantage of these two lines of research in the view of multi-output learning. That is, we propose a generative adversarial network embedding (GANE) model to adapt the generative adversarial framework to achieve the network embedding learning during the specific machine learning tasks. GANE has a generator to generate link edges, and a discriminator to distinguish the generated link edges from real connections (edges) in the network. Wasserstein-1 distance is adopted to train the generator to gain better stability. GANE is further extended by utilizing the pairwise connectivity of vertices to preserve the structural information in the original network. Experiments with real-world network data sets demonstrate that our models constantly outperform state-of-the-art solutions with significant improvements for the tasks of link prediction, clustering, and network alignment. [ABSTRACT FROM AUTHOR]

This paper considers the distributed optimization problem over a network, where the objective is to optimize a global function formed by a sum of local functions, using only local computation and communication. We develop an accelerated distributed Nesterov gradient descent method. When the objective function is convex and $L$ -smooth, we show that it achieves a $O(\frac{1}{t^{1.4-\epsilon }})$ convergence rate for all $\epsilon \in (0,1.4)$. We also show the convergence rate can be improved to $O(\frac{1}{t^2})$ if the objective function is a composition of a linear map and a strongly convex and smooth function. When the objective function is $\mu$ -strongly convex and $L$ -smooth, we show that it achieves a linear convergence rate of $O([ 1 - C (\frac{\mu }{L})^{5/7} ]^t)$ , where $\frac{L}{\mu }$ is the condition number of the objective, and $C>0$ is some constant that does not depend on $\frac{L}{\mu }$. [ABSTRACT FROM AUTHOR]

In this paper, a novel recurrent neural network (RNN) is presented to deal with a kind of nonsmooth nonconvex optimization problem in which the objective function may be nonsmooth and nonconvex, and the constraints include linear equations and convex inequations. Under certain suitable assumptions, from an arbitrary initial state, each solution to the proposed RNN exists globally and is bounded, and it enters the feasible region within a limited time. Moreover, the solution to the RNN with an arbitrary initial state can converge to the critical point set of the optimization problem. In particular, the RNN does not need the following: 1) abounded feasible region; 2) the computation of an exact penalty parameter; or 3) the initial state being chosen from a given bounded set. Numerical experiments are provided to show the effectiveness and advantages of the RNN. [ABSTRACT FROM AUTHOR]

The next generation of aircraft with a large number of effectors will require advanced methods for control allocation (CA) to compute the effectors’ commands needed to follow the desired objective while respecting associated constraints. Currently, the main challenge of the CA is to achieve low enough computation time with a deterministically optimal result for a real-time application. In this paper, at first, we cast the CA as a nonlinear convex programming problem which depicts the desired objective function subject to three-axis moment demands and the limits of effectors’ movement; then we develop a computationally tractable method for real-time application on the CA of the aircraft flight control system. The method can analytically and deterministically give one optimal solution for the CA and prove this optimal solution to be guaranteed within a certainly maximal computation time. Numerical testing results based on Boeing C-17 transport aircraft and Lockheed-Martin tailless fighter models demonstrate that the method is effective in terms of its computational efficiency, accuracy, and reliability. [ABSTRACT FROM AUTHOR]

The event-triggered scheduling of network transmissions has found many applications in engineering tasks operated in cyber-physical systems for its competitive advantage of system resource exploitation. This paper investigates the distributed gradient method for large-scale convex constrained problems with event-triggered consensus protocols. We show that the convergence can be ensured provided that the event-triggering threshold bound is square summable, and the stepsize satisfies specific conditions that are characterized by the Lipschitz constant of the gradient and the spectrum of the mixing matrix associated with the network topology. Stronger convergence results are derived for the strongly convex case, i.e., the local estimate of the minimizer linearly converges to the minimizer until reaching an error floor whose magnitude is shown to be proportional to the stepsize if the triggering threshold bound linearly converges. Comprehensive numerical experiments are conducted to verify the correctness of the theoretical results and advantages of the proposed algorithm over existing ones. [ABSTRACT FROM AUTHOR]

This paper proposes a randomized gradient-free distributed optimization algorithm to solve a multiagent optimization problem with set constraints. Random gradient-free oracle instead of the true gradient information is built locally such that the estimated gradient information is utilized in guiding the update of decision variables. Thus, the algorithm requires no explicit expressions but only local measurements of the cost functions. The row-stochastic and column-stochastic matrices are used as the weighting matrices during the communication with neighbors, making the algorithm convenient to implement in directed graphs as compared with the doubly stochastic weighting matrix. Without the true gradient information, we establish asymptotic convergence to the approximated optimal solution, where the optimality gap can be set arbitrarily small. Moreover, it is shown that the proposed algorithm achieves the same rate of convergence $O(\ln t/\sqrt{t})$ as the state-of-the-art gradient-based methods with similar settings, but having the advantages of less required information and more practical communication topologies. [ABSTRACT FROM AUTHOR]

This paper proposes a bound-based simulation budget allocation (BSBA) procedure for solving ranking and selection (R&S) problems in simulation optimization. For many practical applications, strict bounds on system performances can be obtained through empirical and theoretical approaches. These bounds provide additional information which may help solve R&S problems. In this paper, a new method of objective function estimation is proposed using both bound information and simulation outputs. This new estimation method is demonstrated to be particularly effective. To solve R&S problems, several asymptotic optimal allocation rules are also derived. Using these allocation rules, a BSBA procedure is proposed to achieve high efficiency in identifying the best design. Numerical experiments are provided to examine the performance of the proposed BSBA procedure. The computational results show that BSBA outperforms three compared allocation procedures, especially when bounds are tight or the simulation budget is small. [ABSTRACT FROM AUTHOR]

This paper aims to reduce the communication and computation costs of the Nash equilibrium seeking strategy for the $N$ -coalition noncooperative games. The objective is achieved by the following two manners: 1) an interference graph is introduced to describe the interactions among the agents in each coalition and 2) the Nash equilibrium seeking strategy is designed with the interference graphs considered. The convergence property of the proposed Nash equilibrium seeking strategy is analytically investigated. It is shown that the agents’ actions generated by the proposed method converge to a neighborhood of the Nash equilibrium of the graphical $N$ -coalition noncooperative games under certain conditions. Several special cases where there is only one coalition and/or there are coalitions with only one agent are considered. The results for the special cases demonstrate that the proposed seeking strategy achieves the solution seeking for noncooperative games, social cost minimization problems, and single-agent optimization problems in a unified framework. Numerical examples are presented to support the theoretical results. [ABSTRACT FROM AUTHOR]

This article examines the current work being conducted on improving integer programming with linear programming. Research conducted by Victor Reis and Thomas Rothvoss at the University of Washington utilizing an algorithm by Daniel Dadush is detailed. Topics include the incorporation of a technique from Ravi Kannan and László Lovász, as well as Noah Stephens-Davidowitz and Oded Regev’s contribution.

Control of multi-input multi-output (MIMO) hybrid nonlinear dynamic systems which are affine in control inputs is studied in this paper. It is assumed that each control input of the system can be continuous or discrete with a bound constraint. It is also assumed that the controller is discrete-time and updates the control(s) with a constant frequency. Based on these assumptions, a theorem which represents the necessary and sufficient condition for uniform convergence of the sequence of samples of the state vector of system towards the desired point is presented and proved. As a result of this theorem, a (mixed integer-) linear programming for calculation of control(s) is proposed. Two well-known nonlinear hybrid control problems with multiple inputs are solved by using the proposed method and the results are presented. [ABSTRACT FROM AUTHOR]

The output feedback l 1 control problems of positive Markov jump systems(PMJSs) are studied using event-triggered method in this paper. Firstly, two novel dynamic output event-triggered control strategies(DOETCS) are proposed for continuous-time PMJSs as well as discrete-time PMJSs, which can reduce the event-triggered frequency and save network resources. Then, by constructing multiple linear copositive Lyapunov functions, two sufficient conditions about positivity and stochastic stability with l 1 -gain performance are presented for the continuous-time closed-loop PMJSs and the discrete-time closed-loop PMJSs. Simultaneously, two desired output feedback l 1 controllers and two DOETCS are also co-designed to stabilize the PMJSs in continuous and discrete cases, respectively. All the conditions obtained in this paper are in the form of linear programming(LP). Further more, the influence of parameters in the novel DOETCS on the event-triggered frequency is investigated in detail in the section of numerical simulation. Finally, two numerical examples are provided to verify the effectiveness of the proposed design schemes. [ABSTRACT FROM AUTHOR]

This paper considers the problem of controlling discrete-time linear systems from specifications given as formulas of syntactically co-safe linear temporal logic over linear predicates in the state variables. A systematic procedure is developed for the automatic computation of sets of initial states and feedback controllers such that all the resulting trajectories of the closed-loop system satisfy the given specifications. The procedure is based on the iterative construction and refinement of an automaton that enforces the satisfaction of the formula. Linear programming based approaches are proposed to compute the polytope-to-polytope controllers that label the transitions of the automaton. Extensions to discrete-time piecewise affine systems and specifications given as formulas of full linear temporal logic are included. The algorithms developed in this paper were implemented as a software package that is available for download. Their application and effectiveness are demonstrated for several case studies. [ABSTRACT FROM PUBLISHER]

This paper presents a collective neurodynamic approach with multiple interconnected recurrent neural networks (RNNs) for distributed constrained optimization. The objective function of the distributed optimization problems to be solved is a sum of local convex objective functions, which may be nonsmooth. Subject to its local constraints, each local objective function is minimized individually by using an RNN, with consensus among others. In contrast to existing continuous-time distributed optimization methods, the proposed collective neurodynamic approach is capable of solving more general distributed optimization problems. Simulation results on three numerical examples are discussed to substantiate the effectiveness and characteristics of the proposed approach. In addition, an application to the optimal placement problem is delineated to demonstrate the viability of the approach. [ABSTRACT FROM AUTHOR]

Many existing results on fault-tolerant algorithms focus on the single fault source situation, where a trained network is affected by one kind of weight failure. In fact, a trained network may be affected by multiple kinds of weight failure. This paper first studies how the open weight fault and the multiplicative weight noise degrade the performance of radial basis function (RBF) networks. Afterward, we define the objective function for training fault-tolerant RBF networks. Based on the objective function, we then develop two learning algorithms, one batch mode and one online mode. Besides, the convergent conditions of our online algorithm are investigated. Finally, we develop a formula to estimate the test set error of faulty networks trained from our approach. This formula helps us to optimize some tuning parameters, such as RBF width. [ABSTRACT FROM PUBLISHER]

In this paper, we focus on two notions of diagnosability for labeled Time Petri net (PN) systems: $K$-diagnosability implies that any fault occurrence can be detected after at most $K$ observations, while $\tau$-diagnosability implies that any fault occurrence can be detected after at most $\tau$ time units. A procedure to analyze such properties is provided. The proposed approach uses the Modified State Class Graph, a graph the authors recently introduced for the marking estimation of labeled Time PN systems, which provides an exhaustive description of the system behavior. A preliminary diagnosabilty analysis of the underlying logic system based on classical approaches taken from the literature is required. Then, the solution of some linear programming problems should be performed to take into account the timing constraints associated with transitions. [ABSTRACT FROM AUTHOR]