Search

This paper investigates the stochastic optimization problem focusing on developing scalable parallel algorithms for deep learning tasks. Our solution involves a reformation of the objective function for stochastic optimization in neural network models, along with a novel parallel computing strategy, coined the weighted aggregating stochastic gradient descent (WASGD). Following a theoretical analysis on the characteristics of the new objective function, WASGD introduces a decentralized weighted aggregating scheme based on the performance of local workers. Without any center variable, the new method automatically gauges the importance of local workers and accepts them by their contributions. Furthermore, we have developed an enhanced version of the method, WASGD+, by (1) implementing a designed sample order and (2) upgrading the weight evaluation function. To validate the new method, we benchmark our pipeline against several popular algorithms including the state-of-the-art deep neural network classifier training techniques (e.g., elastic averaging SGD). Comprehensive validation studies have been conducted on four classic datasets: CIFAR-100, CIFAR-10, Fashion-MNIST, and MNIST. Subsequent results have firmly validated the superiority of the WASGD scheme in accelerating the training of deep architecture. Better still, the enhanced version, WASGD+, is shown to be a significant improvement over its prototype. [ABSTRACT FROM AUTHOR]

For a class of stochastic nonlinear systems in pure-feedback form with dead-zone input and multiple time-varying delays, a novel neural network (NN)-based adaptive control approach is presented in this paper through the use of backstepping approach and dynamic surface technique. By choosing proper Lyapunov–Krasovskii functionals, utilizing the characteristic of hyperbolic tangent functions and adopting the function separation technique, difficulties of controller design that introduced by the time-varying delays can be dealt with properly. Moreover, all unknown nonlinear functions are lumped together and approximated by the NN. Additionally, any information over the boundedness of dead-zone parameters is not needed in the process of controller design. The control scheme proposed in this paper ensures the boundedness in probability of all signals in the closed-loop system, besides, excellent performance of arbitrarily small tracking error will be achieved by selecting control parameters appropriately. At last, two numerical simulation examples are provided to verify the validity of the designed algorithm. [ABSTRACT FROM AUTHOR]

This paper establishes an approximate optimal critic learning algorithm based on single neural network (NN) policy iteration (PI) aiming at solving for continuous-time (CT) 2-player zero-sum games (ZSGs). In fact, we have to face the problem that the errors will disturb the dynamics and in turn identifying dynamics will generate errors. In order to prevent the effect of errors, in this paper, a single NN-based online PI algorithm is developed for the CT system, which is disturbed nonlinear ZSG. With plenty of online data, the Hamilton–Jacobi–Isaacs equation can be solved without complete dynamics. Then by the least-squares method, we can obtain the NN weights. Moreover, in the process of dealing with the undisturbed system, we find the way that obtains NN weights in this paper is equal to the way that obtains the optimal solution by the Gauss–Newton method. Based on the convergence of the Gauss–Newton method, we can efficiently obtain the optimal controller for the undisturbed system by utilizing online data. After getting the controller of the undisturbed system, it is time to take disturbance into consideration, so that we design a robust control pair to overcome the disturbance. In order to demonstrate the effectiveness of this algorithm, we design a set of simulations. The results verify that we can solve the disturbed nonlinear ZSG by this algorithm. [ABSTRACT FROM AUTHOR]

For the first time, a deep learning (DL) algorithm is presented to study the effect of the source of variability on the performance of semiconductor nanodevice. This paper reports the possibility of an alternative solution of device simulation in order to optimize the source variation. It is based on the statistical distribution of work function fluctuation (WKF) on the metal gate depending on the orientation and location of metal grains. It has been revealed that the WKF of a metal gate can lead to different fluctuations in electrical characteristics. Therefore, an emerging DL algorithm, artificial neural network (ANN) is utilized to identify the appropriate WKF patterns on the metal gate that can reduce the impact of characteristic fluctuation, i.e., $\sigma V_{TH}, \sigma I_{ON}$ and $\sigma I_{OFF}$ , simultaneously. The application of the DL-ANN algorithm to multichannel gate-all-around silicon nanosheet MOSFETs is explored to suppress the effect of WKF on the characteristic fluctuation. Consequently, it can be further utilized to investigate the implication of WKF for the process variation, modeling the nanodevices and analysis of circuit design. Notably, this technique can be extended to study the diverse random sources and process variation effects for emerging nano-CMOS devices and can effectively accelerate the device simulation and optimization. [ABSTRACT FROM AUTHOR]

A deep learning assisted sum-product detection algorithm (DL-SPDA) for faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed detection algorithm works on a modified factor graph which concatenates a neural network function node to the variable nodes of the conventional FTN factor graph to approach the maximum a posterior probabilities (MAP) error performance. In specific, the neural network performs as a function node in the modified factor graph to deal with the residual intersymbol interference (ISI) that is not considered by the conventional detector with a limited complexity. We modify the updating rule in the conventional sum-product algorithm so that the neural network assisted detector can be complemented to a turbo equalization receiver. Furthermore, we propose a compatible training technique to improve the detection performance of the proposed DL-SPDA with turbo equalization. In particular, the neural network is optimized in terms of the mutual information between the transmitted sequence and the extrinsic information. We also investigate the maximum-likelihood bit error rate (BER) performance of a finite length coded FTN system. Simulation results show that the error performance of the proposed algorithm approaches the MAP performance, which is consistent with the analytical BER. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have been widely applied for classification tasks in wireless communications such as link scheduling, user association and base station (BS) muting. When training a DNN using datasets with highly-skewed class distribution where most data belong to a few majority classes, learning performance for minority classes will degrade since the imbalanced data forces the training process to be biased towards the majority classes. In many wireless problems, multiple decisions for classification are made jointly, say deciding which BSs in a network should be muted to avoid interference. In this paper, we employ DNN to learn an optimal predictive interference coordination policy, and strive to avoid biased learning in such a multi-label multi-class classification problem. The major contribution is proposing a training method to encourage fairness among classes by minimizing the maximal cost of decisions among classes, which is converted into a problem to optimize the weighting factors on the training cost of each class. We provide a gradient descent algorithm to optimize the weighting factors and the model parameters of the DNN by alternative updates. Simulation results show that the training method can improve the learning performance for the minority class and can achieve higher network utility than existing training methods. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of quasi-synchronization (QS) for the periodic neural networks (NNs). In order to address more general NNs, the parameter and the period mismatches are both considered, that is, excluding parameters, periods of the target dynamic and the followers are also different. In addition, the constrainted target information is studied, where the logarithmic quantizer is used to overcome the limited communication capacity and Bernoulli processes are employed to model cases of the information loss. A new period is established based on the lowest common multiple period of the target dynamic and the followers to obtain an augmented synchronization error system (ASES). Further, a suboptimal iterative algorithm is proposed to cut down the QS range of the ASES and the corresponding controllers are designed. At last, the controller design method is illustrated by a numerical example. [ABSTRACT FROM AUTHOR]

Linear spectral unmixing is the practice of decomposing the mixed pixel into a linear combination of the constituent endmembers and the estimated abundances. This paper focuses on unsupervised spectral unmixing where the endmembers are unknown a priori. Conventional approaches use either geometrical- or statistical-based approaches. In this paper, we address the challenges of spectral unmixing with unsupervised deep learning models, in specific, the autoencoder models, where the decoder serves as the endmembers and the hidden layer output serves as the abundances. In several recent attempts, part-based autoencoders have been designed to solve the unsupervised spectral unmixing problem. However, the performance has not been satisfactory. In this paper, we first discuss some important findings we make on issues with part-based autoencoders. By proof of counterexample, we show that all existing part-based autoencoder networks with nonnegative and tied encoder and decoder are inherently defective by making these inappropriate assumptions on the network structure. As a result, they are not suitable for solving the spectral unmixing problem. We propose a so-called untied denoising autoencoder with sparsity, in which the encoder and decoder of the network are independent, and only the decoder of the network is enforced to be nonnegative. Furthermore, we make two critical additions to the network design. First, since denoising is an essential step for spectral unmixing, we propose to incorporate the denoising capacity into the network optimization in the format of a denoising constraint rather than cascading another denoising preprocessor in order to avoid the introduction of additional reconstruction error. Second, to be more robust to the inaccurate estimation of a number of endmembers, we adopt an $l_{21}$ -norm on the encoder of the network to reduce the redundant endmembers while decreasing the reconstruction error simultaneously. The experimental results demonstrate that the proposed approach outperforms several state-of-the-art methods, especially for highly noisy data. [ABSTRACT FROM AUTHOR]

The abnormal rise in contact temperature in switchgear may lead to overheating or even explosion. It is necessary to monitor the rise in real contact temperature to ensure the safety and stability of switchgear. The accurate measurement of real contact temperature is still a problem, and the continuous state of switchgear cannot be assessed accurately by the temperature monitor. In this paper, a particular installation scheme of temperature sensor is put forward and a calculation algorithm for temperature rise of tulip contacts by the measured temperature of bus bar in real time based on artificial neural network (ANN) is established. An electromagnetic-thermal-fluid coupled simulation model for 10-kV/4000-A switchgears was built, and the thermal characteristic was studied taking the influential factors such as load current, ambient temperature, and contact resistance into consideration. The effectiveness of simulation model is discussed by the comparison to temperature rise experiment. Then, an ANN with three layers of structure is established to calculate the temperature of tulip contacts. The sensitivity and accuracy of network are calculated and verified. The results of this paper may provide a novel method for the monitor of switchgear temperature and assess continuous condition of high-current switchgear. [ABSTRACT FROM AUTHOR]

In this paper, a novel adaptive dynamic programming (ADP) algorithm is developed for the infinite-horizon ($H_{\infty}$) optimal control problems with unknown continuous-time (CT) nonlinear systems subject to external disturbances. To facilitate the implementation of the algorithm, generalized fuzzy hyperbolic models (GFHMs) are utilized to establish an identifier–critic architecture, where the identifier is designed to reconstruct the unknown system dynamics, and the GFHM-based critic network is employed to approximate the value functions. The CT $H_{\infty}$ optimal control issue is converted into a two-player zero-sum game and the corresponding Hamilton–Jacobi–Isaacs equation is derived. The learning procedure of the critic design is adaptively implemented with the help of the reconstructed model, thus the requirement of the complete system dynamics is relaxed. Furthermore, by the means of Lyapunov direct method, the uniform ultimate boundedness stability analysis of the closed-loop control system is explicitly provided. Finally, to compare the control performances and disturbance attenuation properties of the proposed method and the existing ADP algorithms, two numerical examples are given. [ABSTRACT FROM AUTHOR]

This paper presents a practical discrete-time fractional order terminal sliding mode (DFOTSM) control strategy for high-precision tracking tasks based on a linear motor. In particular, the practical parametric uncertainties involving sliding friction, uncertain payload, and disturbance in tracking tasks are considered in this paper. Combining Grünwald–Letnikov fractional order definition and terminal sliding mode technique, the proposed method synthesizes a novel DFOTSM control law to drive the sliding mode dynamics into the stable region in finite steps theoretically, even though the system is suffering from uncertainties and disturbances, and the motion on the surface can guarantee higher tracking precision than the conventional discrete-time terminal sliding surface by selecting suitable controller parameters. The theoretical analyses give out the guideline of parameter selection, and experiments are carried out on the linear-motor-based test platform to demonstrate that the proposed controller is easily implemented and can achieve high-precision tracking, fast response, and considerable robustness to uncertainties. [ABSTRACT FROM AUTHOR]

In this paper, a novel event-sampled robust optimal controller is proposed for a class of continuous-time constrained-input nonlinear systems with unknown dynamics. In order to solve the robust optimal control problem, an online data-driven identifier is established to construct the system dynamics, and an event-sampled critic-only adaptive dynamic programming method is developed to replace the conventional time-driven actor–critic structure. The designed online identification method runs during the solving process and is not applied as a priori part for the solutions, which simplifies the architecture and reduces computational load. The proposed robust optimal control algorithm tunes the parameters of critic-only neural network (NN) by event-triggering condition and runs in a plug-n-play framework without system functions, where fewer transmissions and less computation are required as all the measurements received simultaneously. Based on the novel design, the stability of system and the convergence of critic NN are demonstrated by Lyapunov theory, where the state is asymptotically stable and weight error is guaranteed to be uniformly ultimately bounded. Finally, the applications in a basic nonlinear system and the complex rotational–translational actuator problem demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

As one of the main threats to people’s health, especially for the elderly, falls have caused a large number of accidents. Detecting falls in time can minimize the severity of the injury and save lives. Therefore, designing fall detection algorithms to assist people’s daily life has been a hot research topic. In the last decade, different fall detection approaches based on wearable sensors have been proposed. However, since the hardware resources of wearable sensors are very limited, designing accurate but energy-efficient fall detection algorithms remains an open challenge. To deal with this, an accurate but low-cost fall detection algorithm is proposed in this paper. Particularly, a novel cascade and parallel method that efficiently employs the characteristics of human falls and the advanced modeling ability of the neural network is proposed. Also, a novel design of a lightweight convolutional neural network with self-attention is proposed to achieve the best recognition/numerical complexity tradeoff. The proposed method, namely CMFALL, is evaluated together with a multitude of state-of-the-art models on a large dataset, where it performs the best with an F1-score exceeding 99%. Meanwhile, its computational cost and model size are only a few thousandths of other models. Such low computational cost and small size not only enable to embed it in a wearable sensor but also make the system power requirements quite low, which can enhance the autonomy of the wearable fall detector. [ABSTRACT FROM AUTHOR]

This paper presents a control strategy for the position and posture control of a planar four-link underactuated manipulator with a passive second link based on a back-propagation neural network (BPNN) model. First, using the differential evolution (DE) optimization algorithm, the target angles of all links are calculated. The third and fourth links are then controlled from their initial angles to zero. Next, the control process is divided into the following two steps: the control objective of the passive link is implemented in the first step, and the control objectives of the active links are implemented in the second step. In the first step, the system is reduced to a planar virtual Pendubot, a BPNN model based on the DE algorithm is trained to obtain the coupling relationship between the links of this Pendubot. This model gives the speed of the first link to move the passive link to approach the target angle. Then, a fuzzy controller is designed to indirectly control the passive link to its target angle. In the second step, the position mode of the servo controller is adopted to control all active links to their target angles. Experimental results verified the effectiveness of the proposed control strategy. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the joint signal to interference plus noise ratio (SINR) thresholds optimization and resource allocation to maximize the sum-rate of Device-to-Device (D2D) communications while still retaining the rate requirements for active cellular users (CUs), when the inactive CUs are used as opportunistic relays under three operational modes: without using network-coding (NNC), using traditional high-layer network-coding (HNC), and using physical-layer network-coding (PNC). Under Rayleigh fading, we show that, given the selections of relays, this sum-rate maximization in no-relay scheme, NNC, HNC, and PNC opportunistic relay schemes can be formulated as a mixed integer non-linear programming (MINLP), which is NP-hard in general. To find the solution to the MINLP, we propose a two-step approach to solve the problem: 1) for each possible pairing of a D2D pair and a CU, we derive the optimal SINR thresholds to obtain the maximum transmission rate of the D2D pair while satisfying the rate requirement of the CU; 2) based on the maximum transmission rates of D2D pairs for each possible pairing in the first step, we develop a bipartite-matching method to find the optimal pairing CUs for D2D pairs. Finally, according to the solution to the MINLP, we propose an iterative relay selection algorithm to find out the relays that can further improve the sum-rate of D2D communications. Extensive simulation results demonstrate that, compared with the scenario without relaying, the NNC, HNC, and PNC opportunistic relay schemes achieve a maximum performance enhancement of 106%, 138%, and 168%, respectively. [ABSTRACT FROM AUTHOR]

The shortage of training samples remains one of the main obstacles in applying the neural networks to the hyperspectral images classification. To fuse the spatial and spectral information, pixel patches are often utilized to train a model, which may further aggregate this problem. In the existing works, an artificial neural network (ANN) model supervised by centerloss (ANNC) was introduced. Training merely with spectral information, the ANNC yields discriminative spectral features suitable for the subsequent classification tasks. In this paper, we propose a novel convolutional neural network (CNN)-based spatial feature fusion (CSFF) algorithm, which allows a smart integration of spatial information to the spectral features extracted by ANNC. As a critical part of CSFF, a CNN-based discriminant model is introduced to estimate whether two pixels belong to the same class. At the testing stage, by applying the discriminant model to the pixel pairs generated by a test pixel and each of its neighbors, the local structure is estimated and represented as a customized convolutional kernel. The spectral–spatial feature is generated by a convolutional operation between the estimated kernel and the corresponding spectral features within a local region. The final label is determined by classifying the resulting spectral–spatial feature. Without increasing the number of training samples or involving pixel patches at the training stage, the CSFF framework achieves the state of the art by declining 20%–50% classification failures in experiments on three well-known hyperspectral images. [ABSTRACT FROM AUTHOR]

This paper proposes a virtual 3D eyeglasses try-on system driven by a 2D Internet image of a human face wearing with a pair of eyeglasses. The main technical challenge of this system is the automatic 3D eyeglasses model reconstruction from the 2D glasses on a frontal human face. Against this challenge, this paper first proposes an eyeglasses segmentation method using a convolutional neural network-based parsing algorithm to label the glasses pixels, followed with a proposed symmetry-based level-set optimization algorithm to refine the contour of the eyeglasses. With the precisely extracted silhouette image, we take advantages of the smoothness and the symmetry priors of the eyeglasses, and propose a silhouette-based 3D deformation method to deform a 3D eyeglasses model selected from a predefined 3D model database. The obtained model is plausibly approximated to the input 2D eyeglasses after a texture mapping step. Finally, we develop a virtual try-on system to interactively synthesize the reconstructed eyeglasses on a moving target face in real time. The experimental results demonstrate the efficient and convincing virtual try-on performance of our approach and the commercial potential of our proposed system. [ABSTRACT FROM PUBLISHER]

Neural networks with rectified linear unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being, e.g., Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have revolutionized the computer vision research and have seen unprecedented adoption for multiple tasks, such as classification, detection, and caption generation. However, they offer little transparency into their inner workings and are often treated as black boxes that deliver excellent performance. In this paper, we aim at alleviating this opaqueness of CNNs by providing visual explanations for the network’s predictions. Our approach can analyze a variety of CNN-based models trained for computer vision applications, such as object recognition and caption generation. Unlike the existing methods, we achieve this via unraveling the forward pass operation. The proposed method exploits feature dependencies across the layer hierarchy and uncovers the discriminative image locations that guide the network’s predictions. We name these locations CNN fixations, loosely analogous to human eye fixations. Our approach is a generic method that requires no architectural changes, additional training, or gradient computation, and computes the important image locations (CNN fixations). We demonstrate through a variety of applications that our approach is able to localize the discriminative image locations across different network architectures, diverse vision tasks, and data modalities. [ABSTRACT FROM AUTHOR]

This paper presents two algorithms, both based on the time of flight (ToF) of scattered waves, to locate defect in an anisotropic woven-fabric carbon fiber reinforced polymer (CFRP) plate. The first algorithm uses a probabilistic approach by constructing a probability matrix. Each element of this matrix is associated with a location and represents the probability of existing a defect at the corresponding spatial coordinates of the plate. For the probability matrix method, localization results are influenced by manually chosen parameters and by the anisotropy of the CFRP plate. The second algorithm, based on artificial neural networks (ANNs), enables us to improve the accuracy of locating defects. With this ANN method, the anisotropic feature can be “learned” by the neural network from training data. The ToF of scattered waves obtained from three sensor pairs were used directly as inputs of the neural network. The spatial coordinates of the defect are the ANN outputs. The difficulty of obtaining sufficient experimental data for ANN training was surpassed by using added blocks on the surface of the plate under test to simulate defects. This scheme was validated and proved to be effective by comparing the scattered waves from a delamination and from an added block. Conclusions have been drawn by comparing the two methods. The localization results obtained by the ANN algorithm are proved to be better. [ABSTRACT FROM AUTHOR]

The multimode power-split architecture for hybrid electric vehicle (HEV) powertrains is generally known for the complexity of its operation. This paper first addresses the challenge of developing an automated on-line near-optimal control strategy for these systems. Particularly, a machine learning logic based on supervised learning is developed for on-line selection of the HEV operating mode, thus the particular set of clutches to be engaged. An efficiency-based approach is then adopted to determine the optimal power-split between powertrain components. Later, the developed strategy finds integration in an optimal design methodology for multimode power-split HEVs considering the effectiveness and ease of on-line controllability. The obtained results are compared with the ones by the traditional HEV design methodology that only considers off-line energy management. The illustrated design methodology with on-line control reveals efficient at identifying the multimode HEV design that demonstrates optimal predicted fuel economy values and ease of on-line controllability simultaneously. Results suggest that the HEV design optimization procedure may produce different outcomes and demonstrate more effective when the evaluation of the on-line controlled operation is integrated. [ABSTRACT FROM AUTHOR]

In CANDECOMP/PARAFAC tensor decomposition, degeneracy often occurs in some difficult scenarios, especially, when the rank exceeds the tensor dimension, or when the loading components are highly collinear in several or all modes, or when CPD does not have an optimal solution. In such cases, norms of some rank-1 tensors become significantly large and cancel each other. This makes algorithms getting stuck in local minima while running a huge number of iterations does not improve the decomposition. In this paper, we propose an error preservation correction method to deal with such problem. Our aim is to seek an alternative tensor, which preserves the approximation error, but norms of rank-1 tensor components of the new tensor are minimized. Alternating and all-at-once correction algorithms have been developed for the problem. In addition, we propose a novel CPD with a bound constraint on the norm of the rank-one tensors. The method can be useful for decomposing tensors that cannot be performed by traditional algorithms. Finally, we demonstrate an application of the proposed method in image denoising and decomposition of the weight tensors in convolutional neural networks. [ABSTRACT FROM AUTHOR]

It is necessary for cost consideration to minimize the number of the used layers in a high-speed printed circuit board (PCB) design. In this paper, any independent net cannot be treated as a bus-oriented net in a high-speed PCB design because of no timing-matching constraint on an independent net. Any independent net in a high-speed PCB design can be modeled to obey the via-count constraint as the maximum number of the permitted vias for signal integrity. Clearly, the introduction of the permitted vias onto the independent nets can lead to the reduction on the number of the used layers in a high-speed PCB design. Given a set of bus-oriented nets and a set of independent nets with a via-count constraint in a high-speed PCB design, by introducing virtual vias onto independent nets and eliminating redundant vias on any used layer, a generalized algorithm can be proposed to minimize the number of the used layers with satisfying the via-count constraint on any independent net and assign the given bus-oriented nets and the separated segments inside the given independent nets onto the used layers. Compared with Yan’s algorithm with no via introduction on independent nets, the experimental results show that our proposed algorithm with ${c_{\max } = 1}$ , ${c_{\max } = 2}$ , ${c_{\max } = 3}$ , ${c_{\max } = 4}$ , and ${c_{\max } = 5}$ use reasonable CPU time to insert permitted vias to reduce 2.1, 2.8, 3.8, 4.4, and 4.6 used layers on the average for ten tested examples, respectively. Compared with a two-phase algorithm with via introduction on independent nets, the experimental results show that our proposed algorithm with ${c_{\max } = 1}$ , ${c_{\max } = 2}$ , ${c_{\max } = 3}$ , ${c_{\max } = 4}$ , and ${c_{\max } = 5}$ use less CPU time to reduce 1.6, 1.7, 1.7, 1.6, and 1.5 used layers on the average with increasing 13.3%, 16.3%, 10.6%, 4.2%, and 2.6% of the total used vias on the independent nets for ten tested examples, respectively. [ABSTRACT FROM AUTHOR]

The tensor completion issues have obtained a great deal of attention in the past few years. However, the data fidelity part minimizes a squared loss function, which may be inappropriate for the case of noisy one-bit observations. In this paper, we alleviate the mentioned difficulty by drawing on the experience of matrix scenarios. Based on the convex relation to $\ell _{1}$ norm of the tensor multi-rank, we propose a novel optimization model trying to recover the underlying tensor in case of one-bit observations. The feasibility of this model is proved by theoretical derivations. Furthermore, an alternating direction method of multipliers based algorithm is designed to find the solution. The numerical experiments demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

We study a multihop “virtual” full-duplex relay channel as a special case of a general multiple multicast relay network. For such a channel, quantize-map-and-forward (QMF) [and its generalization of noisy network coding (NNC) and short message NNC] achieves the cut-set upper bound within a constant additive gap, where the gap grows linearly with the number of relay stages $K$. This gap, however, may not be acceptable for practical communication systems with multihop transmissions (e.g., a wireless backhaul operating at high frequencies). Recently, we improved the capacity scaling by using a forward sliding-window (SW) decoding and by optimizing the quantization level at each relay, obtaining the gap that grows logarithmically as $\log {K}$. Furthermore, the improved scheme has lower decoding complexity and delay than the general QMF and NNC approaches. In this paper, we further improve the performance by presenting a mixed scheme in which each relay can perform either decode-and-forward (DF) or the improved QMF (with SW decoding) and can choose to perform rate-splitting to enable partial interference cancellation. In general, the optimization of the relay DF/QMF configuration is combinatorial. Nevertheless, we provide that a simple greedy algorithm finds an optimal configuration under some practically reasonable assumptions. We derive an achievable rate that is easily computable and show that the proposed mixed scheme outperforms the QMF-only schemes. We demonstrate that the performance improvement increases with $K$ , which indicates that the mixed scheme is indeed beneficial for multihop transmission. [ABSTRACT FROM AUTHOR]

Neuro-inspired architectures based on synaptic memory arrays have been proposed for on-chip acceleration of weighted sum and weight update in machine/deep learning algorithms. In this paper, we developed NeuroSim, a circuit-level macro model that estimates the area, latency, dynamic energy, and leakage power to facilitate the design space exploration of neuro-inspired architectures with mainstream and emerging device technologies. NeuroSim provides flexible interface and a wide variety of design options at the circuit and device level. Therefore, NeuroSim can be used by neural networks (NNs) as a supporting tool to provide circuit-level performance evaluation. With NeuroSim, an integrated framework can be built with hierarchical organization from the device level (synaptic device properties) to the circuit level (array architectures) and then to the algorithm level (NN topology), enabling instruction-accurate evaluation on the learning accuracy as well as the circuit-level performance metrics at the run-time of online learning. Using multilayer perceptron as a case-study algorithm, we investigated the impact of the “analog” emerging nonvolatile memory (eNVM)’s “nonideal” device properties and benchmarked the tradeoffs between SRAM, digital, and analog eNVM-based architectures for online learning and offline classification. [ABSTRACT FROM AUTHOR]

This paper investigates the application of support vector machine to the modeling of high-speed interconnects with largely varying and/or highly uncertain design parameters. The proposed method relies on a robust and well-established mathematical framework, yielding accurate surrogates of complex dynamical systems. An identification procedure based on the observation of a small set of system responses allows generating compact parametric relations, which can be used for design optimization and/or stochastic analysis. The feasibility and strength of the method are demonstrated based on a benchmark function and on the statistical assessment of a realistic printed circuit board interconnect, highlighting the main features and benefits of this technique over state-of-the-art solutions. Emphasis is given to the effects of the initial sample size and of input noise on the model estimation. [ABSTRACT FROM AUTHOR]

Development of microgrids and wide spread deployment of renewable generation and energy storage systems have highlighted the need for distributed control schemes to achieve maximum utilization and coordination among all the connected resources. In DC microgrids, the key goals of system-wide control are droop-based current sharing and network dc-voltage regulation. These may be achieved using a consensus algorithm secondary control that a) averages all the node voltages dynamically and maintains them at a nominal value and b) based on information from neighboring source currents to converge to a consensus in current sharing. Achieving the dynamic consensus may impose significant communication burden when implemented using conventional periodic communication methods. In this paper, an event-triggered communication based dynamic consensus algorithm is proposed that helps achieve both the stated goals of proportional current sharing along with average DC voltage regulation in a DC microgrid. The convergence and stability of the modified dynamic consensus algorithm and event triggering condition is proven using Lyapunov's stability criterion. The proposed control is applied to a DC microgrid system for secondary control of voltage and current regulation, and the performance of the proposed technique is validated under various operating conditions and disturbance in the communication system in both simulation and hardware environment. [ABSTRACT FROM AUTHOR]

In order to enable voltage ride-through operation, fast and reliable voltage dips detection is necessary. It has been noticed by the analysis of a huge number of measurement results recorded in-field, in laboratory, and obtained by simulation that the voltage dips may be characterized by specific low-order voltage harmonics pattern, which appears at the beginning and at the end of the voltage dips. This feature is named harmonic footprint. This paper proposes a new method of voltage dips detection based on harmonic footprint as part of smart algorithm for detection and classification of power quality events. The aim is to achieve reliable smart voltage dips detection in 1 ms range. Several methods for harmonic estimation in real time were compared in order to choose the optimal one. The footprint is modeled by mathematical function and applied for voltage dips smart detection using recurrent neural network. The proposed method was tested for voltage dips detection analyzing signals from measurement in the real distribution grid (680 records), computer simulations using IEEE-13 bus test grid, and testing in the laboratory. The reliability (detection rate) and speed of voltage dips detection are tested and verified. Furthermore, the future research directions for presented method are outlined. [ABSTRACT FROM AUTHOR]

Object detection plays an important role in intelligent transportation systems and intelligent vehicles. Although the topic of object detection has been studied for decades, it is still challenging to accurately detect objects under complex scenarios. The contributing factors for challenges include diversified object and background appearance, motion blur, adverse weather conditions, and complex interactions among objects. In this paper, we propose a new convolutional neural network (CNN) model for traffic object detection, by using multi-scale local and global feature representation (MFR). The proposed model consists of two components: a region proposal network that generates candidate object regions and an object detection network that incorporates multi-scale features and global information, namely MFR-CNN. These two components are jointly optimized. Once the system is trained, it can detect real-world traffic objects accurately, especially small objects and heavily occluded objects. We evaluate the proposed method on four benchmark datasets, achieving consistent improvements over the state of the art. [ABSTRACT FROM AUTHOR]

Stochastic computation has recently been proposed for implementing artificial neural networks with reduced hardware and power consumption, but at a decreased accuracy and processing speed. Most existing implementations are based on pre-training such that the weights are predetermined for neurons at different layers, thus these implementations lack the ability to update the values of the network parameters. In this paper, a stochastic computational multi-layer perceptron (SC-MLP) is proposed by implementing the backward propagation algorithm for updating the layer weights. Using extended stochastic logic (ESL), a reconfigurable stochastic computational activation unit (SCAU) is designed to implement different types of activation functions such as the $tanh$ and the rectifier function. A triple modular redundancy (TMR) technique is employed for reducing the random fluctuations in stochastic computation. A probability estimator (PE) and a divider based on the TMR and a binary search algorithm are further proposed with progressive precision for reducing the required stochastic sequence length. Therefore, the latency and energy consumption of the SC-MLP are significantly reduced. The simulation results show that the proposed design is capable of implementing both the training and inference processes. For the classification of nonlinearly separable patterns, at a slight loss of accuracy by 1.32-1.34 percent, the proposed design requires only 28.5-30.1 percent of the area and 18.9-23.9 percent of the energy consumption incurred by a design using floating point arithmetic. Compared to a fixed-point implementation, the SC-MLP consumes a smaller area (40.7-45.5 percent) and a lower energy consumption (38.0-51.0 percent) with a similar processing speed and a slight drop of accuracy by 0.15-0.33 percent. The area and the energy consumption of the proposed design is from 80.7-87.1 percent and from 71.9-93.1 percent, respectively, of a binarized neural network (BNN), with a similar accuracy. [ABSTRACT FROM AUTHOR]

Semantic segmentation tasks can be well modeled by Markov Random Field (MRF). This paper addresses semantic segmentation by incorporating high-order relations and mixture of label contexts into MRF. Unlike previous works that optimized MRFs using iterative algorithm, we solve MRF by proposing a Convolutional Neural Network (CNN), namely Deep Parsing Network (DPN), which enables deterministic end-to-end computation in a single forward pass. Specifically, DPN extends a contemporary CNN to model unary terms and additional layers are devised to approximate the mean field (MF) algorithm for pairwise terms. It has several appealing properties. First, different from the recent works that required many iterations of MF during back-propagation, DPN is able to achieve high performance by approximating one iteration of MF. Second, DPN represents various types of pairwise terms, making many existing models as its special cases. Furthermore, pairwise terms in DPN provide a unified framework to encode rich contextual information in high-dimensional data, such as images and videos. Third, DPN makes MF easier to be parallelized and speeded up, thus enabling efficient inference. DPN is thoroughly evaluated on standard semantic image/video segmentation benchmarks, where a single DPN model yields state-of-the-art segmentation accuracies on PASCAL VOC 2012, Cityscapes dataset and CamVid dataset. [ABSTRACT FROM AUTHOR]

A power transformer is one of the most crucial items of equipment in the electricity supply chain. The reliability of this valuable asset is strongly dependent on the condition of its subsystems such as insulation, core, windings, bushings and tap changer. Integration of various measured parameters of these subsystems makes it possible to evaluate the overall health condition of an in-service transformer. This paper develops an artificially intelligent algorithm based on multiple general regression neural networks to combine the operating condition of various subsystems of a transformer to form a quantitative health index. The model is developed using a training set derived from four conditional boundaries based on IEEE standards, the literature and the knowledge of transformer experts. Performance of the proposed method is compared with expert classifications using a database of 345 power transformers. This shows that the proposed method is reliable and effective for condition assessment and is sensitive to poor condition of any single subsystem. [ABSTRACT FROM PUBLISHER]

Partial discharge (PD) recognition is an important tool for online monitoring of gas insulated switchgear (GIS) and diagnosing existing defects. At present, there are two different types of data patterns used for analysis and evaluation of PD signals: phase resolved partial discharge (PRPD) mode and time resolved partial discharge (TRPD) mode. Using different types of data patterns separately can lead to inconsistent or even conflicted recognition results, but the two types of data patterns having complementary information between each other can be used together for data fusion. Dempster-Shafer (DS) evidence theory is introduced to address the problem of evidence conflict and low fusion efficiency. First, two sub-networks for PD recognition are established and compared employing the back propagation neural network (BPNN) learning algorithm under the two modes respectively. Secondly, to minimize (with possible elimination of) the possibility of misclassification, a new fusion decision-making system for PD recognition is proposed on the basis of fusing the results from the sub-networks. Finally, extensive field experiments on two artificial defects are conducted in order to evaluate the performance of the proposed method in this paper. The classification results reveal that the proposed method significantly outperforms the ways of using only single type of data patterns. The method proposed in this paper reduces the uncertainty and effectively improves the credibility of diagnosis. [ABSTRACT FROM AUTHOR]

This paper deals with a modified technique for the recognition of single stage and multiple power quality (PQ) disturbances. An algorithm based on Stockwell's transform and artificial neural network-based classifier and a rule-based decision tree is proposed in this paper. The analysis and classification of single stage PQ disturbances consisting of both events and variations such as sag, swell, interruption, harmonics, transients, notch, spike, and flicker are presented. Moreover, the proposed algorithm is also applied on multiple PQ disturbances such as harmonics with sag, swell, flicker, and interruption. A database of these PQ disturbances based on IEEE-1159 standard is generated in MATLAB for simulation studies. The proposed algorithm extracts significant features of various PQ disturbances using S-transform, which are used as input to this hybrid classifier for the classification of PQ disturbances. Satisfactory results of effective recognition and classification of PQ disturbances are obtained with the proposed algorithm. Finally, the proposed method is also implemented on real-time PQ events acquired in a laboratory to confirm the validity of this algorithm in practical conditions. [ABSTRACT FROM PUBLISHER]

Recently, it has been verified by simulation results that the ability of recognizing and classifying patterns by artificial neural network can be successfully used for anti-islanding protection of distributed synchronous generators. However, since this type of protection differs from the available commercial relays, it is important to test the algorithm performance implemented in hardware. So, this paper presents a performance evaluation of the hardware implementation of an ANN-based algorithm for anti-islanding protection of distributed generators. For the evaluation process, a complete digital relay architecture is developed in LabVIEW, thus allowing its implementation in the commercial real-time platform cRIO-9025/cRIO-9018. The proposed scheme is tested under real conditions by means of the CMC-256-plus test set, as performed for conventional intelligent electronic devices. The proposed algorithm implemented in hardware shows high effectiveness, detecting all islanding conditions within 1 s, sharing the same good performance verified by computational simulations. The algorithm implementation shows robustness, as it remains blocked for other power grid disturbances, such as frequency variation in the grid utility, massive connection or load relief. [ABSTRACT FROM PUBLISHER]

A residual networks family with hundreds or even thousands of layers dominates major image recognition tasks, but building a network by simply stacking residual blocks inevitably limits its optimization ability. This paper proposes a novel residual network architecture, residual networks of residual networks (RoR), to dig the optimization ability of residual networks. RoR substitutes optimizing residual mapping of residual mapping for optimizing original residual mapping. In particular, RoR adds levelwise shortcut connections upon original residual networks to promote the learning capability of residual networks. More importantly, RoR can be applied to various kinds of residual networks (ResNets, Pre-ResNets, and WRN) and significantly boost their performance. Our experiments demonstrate the effectiveness and versatility of RoR, where it achieves the best performance in all residual-network-like structures. Our RoR-3-WRN58-4 + SD models achieve new state-of-the-art results on CIFAR-10, CIFAR-100, and SVHN, with the test errors of 3.77%, 19.73%, and 1.59%, respectively. RoR-3 models also achieve state-of-the-art results compared with ResNets on the ImageNet data set. [ABSTRACT FROM AUTHOR]

Compared with other applications in computer vision, convolutional neural networks (CNNs) have underperformed on pedestrian detection. A breakthrough was made very recently using sophisticated deep CNN (DCNN) models, with a number of handcrafted features or explicit occlusion handling mechanism. In this paper, we show that by reusing the convolutional feature maps of a DCNN model as image features to train an ensemble of boosted decision models, we are able to achieve the best reported accuracy without using specially designed learning algorithms. We empirically identify and disclose important implementation details. We also show that pixel labeling may be simply combined with a detector to boost the detection performance. By adding complementary handcrafted features such as optical flow, the DCNN-based detector can be further improved. We advance the state-of-the-art results by lowering the log-average miss rate from 11.7% to 8.9% on the Caltech data set and from 11.2% to 8.6% on the Inria data set. We also achieve a comparable result to state-of-the-art approaches on the KITTI data set. [ABSTRACT FROM AUTHOR]

This paper is concerned with the problem of distributed weighted Kalman filter fusion (DWKFF) for a class of multisensor unreliable networked systems (MUNSs) with correlated noises. The process noise and the measurement noises are assumed to be one-step, two-step cross-correlated, and one-step autocorrelated, and the measurement noises of each sensor are one-step cross-correlated. The stochastic uncertainties in the state and measurements are described by correlated multiplicative noises. The MUNSs suffer measurement delay or loss due to their unreliability. Buffers of finite length are proposed to deal with measurement delay or loss, and an optimal local Kalman filter estimator with a buffer of finite length is derived for each subsystem. Based on the new optimal local Kalman filter estimator, the DWKFF algorithm with finite length buffers has been developed which has stronger fault-tolerance ability. Simulation results illustrate the effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

In this paper, we seek to improve deep neural networks by generalizing the pooling operations that play a central role in the current architectures. We pursue a careful exploration of approaches to allow pooling to learn and to adapt to complex and variable patterns. The two primary directions lie in: (1) learning a pooling function via (two strategies of) combining of max and average pooling, and (2) learning a pooling function in the form of a tree-structured fusion of pooling filters that are themselves learned. In our experiments every generalized pooling operation we explore improves performance when used in place of average or max pooling. We experimentally demonstrate that the proposed pooling operations provide a boost in invariance properties relative to conventional pooling and set the state of the art on several widely adopted benchmark datasets. These benefits come with only a light increase in computational overhead during training (ranging from additional 5 to 15 percent in time complexity) and a very modest increase in the number of model parameters (e.g., additional 1, 9, and 27 parameters for mixed, gated, and 2-level tree pooling operators, respectively). To gain more insights about our proposed pooling methods, we also visualize the learned pooling masks and the embeddings of the internal feature responses for different pooling operations. Our proposed pooling operations are easy to implement and can be applied within various deep neural network architectures. [ABSTRACT FROM PUBLISHER]

This paper presents a neural network-based method to generate multiple images with different exposures from a single input low dynamic range (LDR) image for improved high dynamic range (HDR) imaging. The proposed algorithm consists of three steps: 1) 2-D histogram estimation; 2) neural network-based LDR images estimation; and 3) generation of an optimal set of differently exposed images. The proposed method first generates image features by estimating a patched-based 2-D histogram. The extracted features are used in an input layer of the neural network, which plays a role to select an optimal set of LDR images. A set of LDR images is generated using a curvature-based contrast enhancement method. Experimental results show that the proposed method can generate an optimal set of LDR images using neural network and provide improved HDR images. In addition, the proposed method can be implemented as a preprocessing step in most existing HDR frameworks. The proposed HDR approach is considered as a single-input method that gives almost the same performance to multiple image-based HDR method. [ABSTRACT FROM PUBLISHER]

Following the great success of deep convolutional neural networks (CNNs) in computer vision, this paper proposes a complex-valued CNN (CV-CNN) specifically for synthetic aperture radar (SAR) image interpretation. It utilizes both amplitude and phase information of complex SAR imagery. All elements of CNN including input-output layer, convolution layer, activation function, and pooling layer are extended to the complex domain. Moreover, a complex backpropagation algorithm based on stochastic gradient descent is derived for CV-CNN training. The proposed CV-CNN is then tested on the typical polarimetric SAR image classification task which classifies each pixel into known terrain types via supervised training. Experiments with the benchmark data sets of Flevoland and Oberpfaffenhofen show that the classification error can be further reduced if employing CV-CNN instead of conventional real-valued CNN with the same degrees of freedom. The performance of CV-CNN is comparable to that of existing state-of-the-art methods in terms of overall classification accuracy. [ABSTRACT FROM PUBLISHER]

This paper proposes the energy-based controller (EC) incorporated with radical basis function (RBF) neural-network compensation (ECRBFC), which is used to swing up the Pendubot and raise it to its uppermost unstable equilibrium position. First, for the known dynamics model of the two-link arm, the EC is designed. In the EC, the singularity is successfully avoided by constructing an appropriate energy evaluation function. Second, as for the friction of the Pendubot, because of the time-varying characteristics, an accurate friction dynamics model cannot be known absolutely; thus, the RBF neural network is introduced to offset the bad effect of friction. Finally, in order to evaluate the performance of ECRBFC, the numerical simulations and the experimental results are given, and by comparing the results with that of other algorithms, it is found that ECRBFC proposed in this paper has better performance under the same conditions. [ABSTRACT FROM PUBLISHER]