Search

Despite being cost-effective, seven-level Modified Packed U-Cell (MPUC7) active rectifier tends to be unstable due to unequal dc-links. Thus, a multiobjective controller is required to stabilize voltages and currents besides preserving efficiency and power quality. While conventional finite-set model predictive control (FSMPC) can deal with the multiobjective problem, it cannot assure the system stability, and its weighing factors tuning significantly becomes tiresome as the number of objectives increases. This article presents a low-frequency adaptive FSMPC (AMPC) stabilized based on Lyapunov stability theory to overcome the design problems of FSMPC. AMPC handles four control objectives and a decoupled stability objective. The control objectives assure the standard performance of MPUC7 in terms of switching losses, d v /d t , THD, and capacitors ripple. The stability objective guarantees the rectifier reliability under unstable conditions. The weighting factors in AMPC are floating to tackle the tuning challenges where a radial basis function neural network controller (RBFC) adjusts their variations. RBFC is trained by a novel self-training method including particle swarm optimization (PSO) algorithm and some mathematical analyses without using any training data. Experimental and simulation tests also evaluate AMPC in different conditions to confirm its reliability in fulfilling the desired objectives.

Distributed fiber-optical acoustic sensing (DAS) is a new and booming technology in seismic exploration. DAS technology has been gradually applied to the exploration of vertical seismic profile (VSP) due to its strong resistance to high temperature and pressure, high sensitivity, high precision (trace interval can be accurate to about 1 m), and so on. However, real DAS-VSP data are always contaminated by both random and coherent noises, which greatly affects the quality of DAS-VSP data. In order to suppress the background noise and increase the signal-to-noise ratio (SNR), a convolutional neural network (CNN) based on leaky rectifier linear unit (ReLU) and forward modeling is proposed and named L-FM-CNN. In terms of network architecture, Leaky ReLU is adopted as the activation function of CNN, which can enhance the recovery ability of trained CNN denoising model to the weak effective signals. As for the training data set, we construct a high-authenticity theoretical pure seismic data set for DAS-VSP data through the complexity of forward models and the diversification of physical parameters. In addition, we propose a new mean square error (MSE) loss function combined with an energy ratio matrix (ERM). The ERM can adjust the SNR between the signal patch and noise patch during the network training and thus increase the robustness of trained CNN denoising model for the DAS-VSP data with different SNRs, especially the DAS-VSP data with extremely low SNR. Both synthetic and real experiments prove the effectiveness of the proposed L-FM-CNN.

There have been 350,000 tweets generated by the interaction of social networks with different cultures and educational backgrounds in the last ten years. Various sentiments are expressed in the user comments, from support to hatred. The sentiments regarded the United States General Election in 2020. This dataset has 3,000 data gotten from previous research. We augment it becomes 15,000 data to facilitate training and increase the required data. Sentiment detection is carried out using the CNN-BiLSTM architecture. It is chosen because CNN can filter essential words, and BiLSTM can remember memory in two directions. By utilizing both, the training process becomes maximum. However, this method has disadvantages in the activation. The drawback of the existing activation method, i.e., "Zero-hard Rectifier" and "ReLU Dropout" problem to become the cause of training stopped in the ReLU activation, and the exponential function cannot be set become the activation function still rigid towards output value in the SERLU activation. To overcome this problem, we propose a novel activation method to repair activation in CNN-BiLSTM architecture. It is namely the ASERLU activation function. It can adjust positive value output, negative value output, and exponential value by the setter variables. So, it adapts more conveniently to the output value and becomes a flexible activation function because it can be increased and decreased as needed. It is the first research applied in architecture. Compared with ReLU and SERLU, our proposed method gives higher accuracy based on the experiment results.

Bottom-up-fabricated crossbars promise superior circuit density and 3-D integrability compared with the traditional CMOS-based implementations. However, their inherent stochasticity presents difficulties in building complex circuits from components that demand precise patterning and high registration accuracies. With fewer terminals than active devices, passive components offer higher device densities and registration tolerances, making them amenable to bottom-up synthesized nanocrossbars. Motivated by this preference for passivity, we explore, in this article, neuromorphic classifiers based on passive neurons and passive synapses. We demonstrate via SPICE simulations how a shallow network of the diode-resistor-based passive rectifier neurons and resistive voltage summers, despite its inherent inability to buffer, amplify, and negate signals, can recognize MNIST digits with 95.4% accuracy. We introduce weight-to-conductance mappings that enable negative weights to be implemented in hardware without excessive memory overheads. The influences of soft and hard defects on the classification performance are evaluated, and the methods to boost fault-tolerance are proposed. The first-order evaluation of the area, speed, and power consumption of the passive multilayer perceptron classifiers is undertaken, and the results are compared with a benchmark study in neuromorphic hardware.

Radioactive particle tracking (RPT) is a minimally invasive nuclear technique that tracks a radioactive particle inside a volume of interest by means of a mathematical location algorithm. During the past decades, many algorithms have been developed including ones based on artificial intelligence techniques. In this study, RPT technique is applied in a simulated test section that employs a simplified mixer filled with concrete, six scintillator detectors and a137Cs radioactive particle emitting gamma rays of 662 keV. The test section was developed using MCNPX code, which is a mathematical code based on Monte Carlo simulation, and 3516 different radioactive particle positions (x,y,z) were simulated. Novelty of this paper is the use of a location algorithm based on a deep learning model, more specifically a 6-layers deep rectifier neural network (DRNN), in which hyperparameters were defined using a Bayesian optimization method. DRNN is a type of deep feedforward neural network that substitutes the usual sigmoid based activation functions, traditionally used in vanilla Multilayer Perceptron Networks, for rectified activation functions. Results show the great accuracy of the DRNN in a RPT tracking system. Root mean squared error for x, y and coordinates of the radioactive particle is, respectively, 0.03064, 0.02523 and 0.07653.

Nowadays, networked controls using cyber-physical systems (CPSs) necessitate engineers considering “faulty signals” into the control from the beginning of the design process. Therefore, synthesizing control methods, which are able to deal with faulty signals and tolerate them, must be thoroughly investigated and integrated into the design process from the commencement. This article proposes an innovative, reliable control based on a sliding mode faulty signal rectifier for active-/reactive-power-controlled, grid-connected voltage-source inverters (named GC-VSIs hereinafter). It is called “faulty-signal-tolerant” control in this article. Those faulty signals can reach the GC-VSI's controls from any sources; for example, they may arise provided that the CPSs malfunction or fail to prevent data-integrity-related issues, cyber threats, and more. The sliding mode algorithm provides the proposed controller with resilient performance via rectifying faulty signals. Besides, the proposed structure is enhanced by an adaptive mechanism, which makes it more robust against the “unknown” nature of faulty signals. The adaptation rule is able to find the unknown bounds of faulty signals (which externally impact control feedback) and incorporate them into the control by the sliding-mode-based faulty signal rectifier to form a faulty-signal-tolerant methodology. Thorough theoretical analyses, including stability assessment using the Lyapunov criterion, are provided in order to design the proposed controller. Comprehensive simulations and experimental results (associated with a GC-VSI) show the effectiveness and reliability of the faulty-signal-tolerant controller, which is proposed in this research.

We analyze the learning problem of one-hidden-layer nonoverlapping convolutional neural networks with the rectified linear unit (ReLU) activation function from the perspective of model estimation. The training outputs are assumed to be generated by the neural network with the unknown ground-truth parameters plus some additive noise, and the objective is to estimate the model parameters by minimizing a nonconvex squared loss function of the training data. Assuming that the training set contains a finite number of samples generated from the Gaussian distribution, we prove that the accelerated gradient descent (GD) algorithm with a proper initialization converges to the ground-truth parameters (up to the noise level) with a linear rate even though the learning problem is nonconvex. Moreover, the convergence rate is proved to be faster than the vanilla GD. The initialization can be achieved by the existing tensor initialization method. In contrast to the existing works that assume an infinite number of samples, we theoretically establish the sample complexity of the required number of training samples. Although the neural network considered here is not deep, this is the first work to show that accelerated GD algorithms can find the global optimizer of the nonconvex learning problem of neural networks. This is also the first work that characterizes the sample complexity of gradient-based methods in learning convolutional neural networks with the nonsmooth ReLU activation function. This work also provides the tightest bound so far of the estimation error with respect to the output noise.

The existing intelligent fault diagnosis methods of rotor-bearing system mainly focus on vibration analysis under steady operation, which has low adaptability to new scenes. In this article, a new framework for rotor-bearing system fault diagnosis under varying working conditions is proposed by using modified convolutional neural network (CNN) with transfer learning. First, infrared thermal images are collected and used to characterize the health condition of rotor-bearing system. Second, modified CNN is developed by introducing stochastic pooling and Leaky rectified linear unit to overcome the training problems in classical CNN. Finally, parameter transfer is used to enable the source modified CNN to adapt to the target domain, which solves the problem of limited available training data in the target domain. The proposed method is applied to analyze thermal images of rotor-bearing system collected under different working conditions. The results show that the proposed method outperforms other cutting edge methods in fault diagnosis of rotor-bearing system.

Machine learning can be a technique of nursing lysis that automatically develops an analytical model. It is a branch of synthetic intelligence that believes that systems are going to learn information, determine patterns of information and decide with degraded human intervention. Machine learning addresses the question of how computers can be constructed that improve mechanically through knowledge. It lies at the intersection of technology and statistics and at the center of artificial data and information science, one in all the quickest increasing technical fields of nowadays. Recent advances in machine learning were driven by the event of latest learning and theories also as by the constant explosion. The event of latest learning algorithms and also theory and the in-progress growth within the accessibility of on-line information also as low-priced computation crystal rectifier to recent progress within the field of machine learning. Additional evidence-based decision-making could be carried out in science, technology and trade, including healthcare, production, education and monetary modelling, enforcement and promotion, with adoption of mechanical learning techniques based on data-intensive methods. The results are also available. The infection can be a life-threatening disease. The bite of a nursing partner is often transmitted in dipterous Anopheles. In infected mosquitoes, plasmodium parasite is a gift. The parasite is discharged into your blood after you bite this dipterous insect once it bites you. Once your body is composed of the parasites, they mature into the liver. The mature parasites enter the blood for several days when red blood cells start to infect. In red blood cells, parasites increase over 48-72 hours, causing infected cells to divide. The parasites still infect red blood cells, which last 2 to 3 days in cycles. This paper is used for observation of protozoan infection with a deep learning idea.

The choice of activation functions in Convolutional Neural Networks (CNNs) is very important. Rectified Linear Unit (ReLU) has been widely-used in most CNNs. Recently, a series of non-monotonic activation functions gradually become the new standard to enhance performance of CNNs. Inspired by them, this paper firstly proposes a novel non-monotonic activation function called Power Function Linear Unit (PFLU). The negative part of PFLU is non-monotonic and closer to zero with the negative input decreasing, which can maintain sparsity of the negative part while introducing negative activation values and non-zero derivative values for the negative part. The positive part of PFLU does not use identity mapping but is closer to identity mapping with the positive input increasing, which can bring non-linearity property for the positive part. Next, this paper proposes faster PFLU (FPFLU). A wide range of classification experiments show that PFLU tends to work better than current state-of-the-art non-monotonic activation functions, and FPFLU can run faster than most non-monotonic activation functions.

This work briefly investigates the coexistence of continuous attractors in the neural networks with Rectified Linear Unit (RELU) transfer function. Memory is stored as a manifold of stable states, or a continuous attractor. Continuous attractors are some low-dimensional manifolds embedded in a high-dimensional state space. One neural network may possess more than one continuous attractors. More importantly, we found that these multiple continuous attractors may have different dimensional, some are 2-D plane attractors and others are 1-D line attractors. It is also an enlightenment to study the continuous attractors of high dimensional models.

Vibration signals under the same health state often have large differences due to changes in operating conditions. Likewise, the differences among vibration signals under different health states can be small under some operating conditions. Traditional deep learning methods apply fixed nonlinear transformations to all the input signals, which have a negative impact on the discriminative feature learning ability, i.e., projecting the intraclass signals into the same region and the interclass signals into distant regions. Aiming at this issue, this article develops a new activation function, i.e., adaptively parametric rectifier linear units, and inserts the activation function into deep residual networks to improve the feature learning ability, so that each input signal is trained to have its own set of nonlinear transformations. To be specific, a subnetwork is inserted as an embedded module to learn slopes to be used in the nonlinear transformation. The slopes are dependent on the input signal, and thereby the developed method has more flexible nonlinear transformations than the traditional deep learning methods. Finally, the improved performance of the developed method in learning discriminative features has been validated through fault diagnosis applications.

A vital necessity when employing state-of-the-art deep neural networks (DNNs) for topology optimization is to predict near-optimal structures while satisfying pre-defined optimization constraints and objective function. Existing studies, on the other hand, suffer from the structural disconnections which result in unexpected errors in the objective and constraints. In this study, a two-stage network model is proposed using convolutional encoder-decoder networks that incorporate a new way of loss functions to reduce the number of structural disconnection cases as well as to reduce pixel-wise error to enhance the predictive performance of DNNs for topology optimization without any iteration. In the first stage, a single DNN model architecture is proposed and used in two parallel networks using two different loss functions for each called the mean square error (MSE) and mean absolute error (MAE). Once the priori information is generated from the first stage, it is instantly fed into the second stage, which acts as a rectifier network over the priori predictions. Finally, the second stage is trained using the binary cross-entropy (BCE) loss to provide the final predictions. The proposed two-stage network with the proposed loss functions is implemented for both two-dimensional (2D) and three-dimensional (3D) topology optimization datasets to observe its generalization ability. The validation results showed that the proposed two-stage framework could improve network prediction ability compared to a single network while significantly reducing compliance and volume fraction errors.

Purpose To improve the performance of a denoising convolutional neural network (DnCNN) and to make it applicable to images with inhomogeneous noise, a refinement involving an activation function (AF) and an application of the refined method for inhomogeneous-noise images was examined in combination with parallelized image denoising. Methods Improvements in the DnCNN were performed by three approaches. One is refinement of the AF of each neural network that constructs the DnCNN. Swish was used in the DnCNN instead of rectifier linear unit. Second, blind noise removal was introduced to the DnCNN in order to adapt spatially variant noises. Third, blind noise removal was applied to parallelized image denoising, referred to herein as ParBID. The ParBID procedure is as follows: (1) adjacent 2D slice images are linearly combined to obtained higher peak SNR (PSNR) images, (2) combined images with different weight coefficients are denoised using the blind DnCNN, and (3) denoised combined images are separated into original position images by algebraic calculation. Results Experimental studies showed that the PSNR and the structural similarity index (SSIM) were improved by using Swish for all noise levels, from 2.5% to 7.5%, as compared to the conventional DnCNN. It was also shown that a well-trained CNN could remove spatially variant noises superimposed on images. Experimental studies with ParBID showed that the greatest PSNR and SSIM improvements were obtained at the middle slice when three slice images were used for linear image combination. More fine structures of images and image contrast remained when the proposed ParBID procedure was used. Conclusion Swish can improve the denoising performance of the DnCNN, and the denoising performance and effectiveness were further improved by ParBID.

Determination of the drastic changes in water quality is an urgent need in this polluted era and is more essential for the survival of the existing and growing water demand. It has been very difficult to analyze the water quality data. This study focused on the Water Quality Index (WQI) prediction of water samples collected from 1944 different wells surrounding the Vellore district. WQI prediction is carried out by ANN (i.e.) Artificial Neural Networks implementation which has used 15 groundwater variables that are collected in different parts of the Vellore district from 2008 to 2017. If 15 underground variable values meet the desired range then WQI is considered as better and appropriate for drinking. But if any one of the value doesn’t meet the desired range then it is not considered as better and hence not suitable for drinking. In this study the pre-processing of the collected data has been completed to reduce the computational time. Further feature extraction techniques are used to extract the required features. The extracted features are passed on to ANN classifiers that possess three activation functions like Tanh, Maxout, and rectifier. The novelty of this paper is that WQI is determined by combining the three activation functions like Tanh, Maxout, and rectifier. A comparative analysis has been performed for proposed work related with various methodologies.

Cardiovascular diseases (CVDs) are consider the main cause of death today According to World Health Organization (WHO),and because that ECG signal is very important tool in monitoring and diagnosis of these disease , different automatic methods were proposed based on this signal. [1]. The manual analysis of ECG signals is suffered different challenges such as differeculty of detecting and classify waveform of this signal, So, many machine learning methods are explored to describe the anomalies ECG signal accurately . Deep learning (DL) can be used in ECG classification, it can improve the quality of the automatic classification system. In this paper , we have proposed a deep learning classification system by using different layers of convolution, rectifier and pooling operations that can be used to increase feature extraction of ECG signal. We have proposed two models, one is used for input signal of 1-D, in which we designed model for classification csv type of data for ECG signal, while in the second proposed system, we used model for 2-D signal after convert it from its csv type . 2-D signal (ECG image) is used in order to augment the two dimensional signal with different methods to increase the accuracy of the model by training it with geometric transformation of the original input images such as rotation, shearing etc.The results are compared with AlexNet and other models based on the metrics, which are used to measure the performance of the proposed work, the result show that, the proposed models improve the efficiency of the classification in the two systems.

Deep learning is drawing keen attention in contemporary financial research. In this article, the authors investigate the statistical predictive power and economic significance of financial stock market data by using deep learning techniques. In particular, the authors use the equity premium as the response variable and financial variables as predictors. The deep learning techniques used in this study provide useful evidence of statistical predictability and economic significance. Considering the statistical predictive performance of the deep learning models, H2O deep learning (H2ODL) gives the smallest mean-squared forecast error (MSFE), with the corresponding highest cumulative return (CR) and Sharpe ratio (SR) in each of the out-of-sample periods. Specifically, the H2ODL with Rectifier used as the activation function outperformed the other models in this article. In the fusion results, the SAE-with-H2O using the Maxout activation function yields the smallest MSFE with the corresponding highest CR and SR in all of the out-of-sample periods. It is worth noting that the higher the CR, the higher the SR and the lower the MSFE, which concords with a rule of thumb. Overall, the empirical analysis in this study revealed that the SAE-with-H2O using the Maxout activation function produced the best statistically predictive and economically significant results with robustness across all out-of-sample periods. TOPICS:Big data/machine learning, performance measurement, quantitative methods, simulations, statistical methods Key Findings ▪ In this article, the authors use deep learning models to predict the equity premium, employing a plethora of well-known predictors. ▪ The authors employ deep learning models such as deep neural networks, a stacked autoencoder, and long short-term memory models. ▪ The statistical and economic significance of the proposed models is examined and back tested in three out-of-sample periods.

Activation functions play important roles in determining the depth and non-linearity of deep learning models. Since the Rectified Linear Unit (ReLU) was introduced, many modifications, in which noise is intentionally injected, have been proposed to avoid overfitting. Exponential Linear Unit (ELU) and their variants, with trainable parameters, have been proposed to reduce the bias shift effects which are often observed in ReLU-type activation functions. In this paper, we propose a novel activation function, called the Elastic Exponential Linear Unit (EELU), which combines the advantages of both types of activation functions in a generalized form. EELU has an elastic slope in the positive part, and preserves the negative signal by using a small non-zero gradient. We also present a new strategy to insert neuronal noise using a Gaussian distribution in the activation function to improve generalization. We demonstrated how EELU can represent a wider variety of features with random noise than other activation functions, by visualizing the latent features of convolutional neural networks. We evaluated the effectiveness of the EELU approach through extensive experiments with image classification using the CIFAR-10/CIFAR-100, ImageNet, and Tiny ImageNet datasets. Our experimental results show that EELU achieved better generalization performance and improved classification accuracy over conventional activation functions, such as ReLU, ELU, ReLU- and ELU-like variants, Scaled ELU, and Swish. EELU produced performance improvements in image classification using a smaller number of training samples, owing to its noise injection strategy, which allows significant variation in function outputs, including deactivation.

Maize kernel traits such as kernel length, kernel width, and kernel number determine the total kernel weight and, consequently, maize yield. Therefore, the measurement of kernel traits is important for maize breeding and the evaluation of maize yield. There are a few methods that allow the extraction of ear and kernel features through image processing. We evaluated the potential of deep convolutional neural networks and binary machine learning (ML) algorithms (logistic regression (LR), support vector machine (SVM), AdaBoost (ADB), Classification tree (CART), and the K-Neighbor (kNN)) for accurate maize kernel abortion detection and classification. The algorithms were trained using 75% of 66 total images, and the remaining 25% was used for testing their performance. Confusion matrix, classification accuracy, and precision were the major metrics in evaluating the performance of the algorithms. The SVM and LR algorithms were highly accurate and precise (100%) under all the abortion statuses, while the remaining algorithms had a performance greater than 95%. Deep convolutional neural networks were further evaluated using different activation and optimization techniques. The best performance (100% accuracy) was reached using the rectifier linear unit (ReLu) activation procedure and the Adam optimization technique. Maize ear with abortion were accurately detected by all tested algorithms with minimum training and testing time compared to ear without abortion. The findings suggest that deep convolutional neural networks can be used to detect the maize ear abortion status supplemented with the binary machine learning algorithms in maize breading programs. By using a convolution neural network (CNN) method, more data (big data) can be collected and processed for hundreds of maize ears, accelerating the phenotyping process.

This article focuses on a problem important to automatic machine learning: the automatic processing of a nonpreprocessed time series. The convolutional neural network (CNN) is one of the most popular neural network (NN) algorithms for pattern recognition. Seasonal time series with trends are the most common data sets used in forecasting. Both the convolutional layer and the pooling layer of a CNN can be used to extract important features and patterns that reflect the seasonality, trends, and time lag correlation coefficients in the data. The ability to identify such features and patterns makes CNN a good candidate algorithm for analyzing seasonal time-series data with trends. This article reports our experimental findings using a fully connected NN (FNN), a nonpooling CNN (NPCNN), and a CNN to study both simulated and real time-series data with seasonality and trends. We found that convolutional layers tend to improve the performance, while pooling layers tend to introduce too many negative effects. Therefore, we recommend using an NPCNN when processing seasonal time-series data with trends. Moreover, we suggest using the Adam optimizer and selecting either a rectified linear unit (ReLU) function or a linear activation function. Using an NN to analyze seasonal time series with trends has become popular in the NN community. This article provides an approach for building a network that fits time-series data with seasonality and trends automatically.

In order to realize the unsupervised extraction and identification of fault features in power electronic circuits, we proposed a fault diagnosis method based on sparse autoencoder (SAE) and broad learning system (BLS). Firstly, the feature is extracted by the sparse autoencoder, and the fault samples and feature vectors are combined as the input of the broad learning system. The broad learning system is trained based on the error precision step update method, and the system is used to the fault type identification. The simulation results of the thyristor fault diagnosis of the three-phase bridge rectifier circuit show that the method is effective and has better performance than other traditional methods.

An activation function has crucial role in a deep neural network. A simple rectified linear unit (ReLU) are widely used for the activation function. In this paper, a weighted sigmoid gate unit (WiG) is proposed as the activation function. The proposed WiG consists of a multiplication of inputs and the weighted sigmoid gate. It is shown that the WiG includes the ReLU and same activation functions as a special case. Many activation functions have been proposed to overcome the performance of the ReLU. In the literature, the performance is mainly evaluated with an object recognition task. The proposed WiG is evaluated with the object recognition task and the image restoration task. Then, the expeirmental comparisons demonstrate the proposed WiG overcomes the existing activation functions including the ReLU.

Rectified linear units (ReLUs) are currently the most popular activation function used in neural networks. Although ReLUs can solve the gradient vanishing problem and accelerate training convergence, it suffers from the dying ReLU problem in which some neurons are never activated if the weights are not updated properly. In this work, we propose a novel activation function, known as the adaptive convolutional ReLU (ConvReLU), that can better mimic brain neuron activation behaviors and overcome the dying ReLU problem. With our novel parameter sharing scheme, ConvReLUs can be applied to convolution layers that allow each input neuron to be activated by different trainable thresholds without involving a large number of extra parameters. We employ the zero initialization scheme in ConvReLU to encourage trainable thresholds to be close to zero. Finally, we develop a partial replacement strategy that only replaces the ReLUs in the early layers of the network. This resolves the dying ReLU problem and retains sparse representations for linear classifiers. Experimental results demonstrate that our proposed ConvReLU has consistently better performance compared to ReLU, LeakyReLU, and PReLU. In addition, the partial replacement strategy is shown to be effective not only for our ConvReLU but also for LeakyReLU and PReLU.

A power electronic converter is a switching system; the key to modeling it is how to describe the interaction between the discrete switch variables and the continuous state variables in a unified model. A power electronic converter modeling method based on sigmoid functions is proposed. Since the sigmoid function is continuous, smooth, differentiable, and saturated at a certain value, a sigmoid function with a large steepness factor is used to approximate the switching process of the switch; the converter is transformed from a switching system to a continuous system so that nonlinear continuous system analysis and design methods can be directly applied to the converter control system. Sigmoid function equivalent circuit models for several typical inverter and rectifier circuits are provided. A half-bridge LLC resonant converter is selected as an example, and its sigmoid function model is constructed and subsequently validated through simulation and experimentation. The results show that the constructed model is in good agreement with the circuit simulation and experimental results. The mathematical expression of the model has a simple and unified structure and definite physical meaning and is a high-accuracy large-signal continuous model with universal significance.

A key strategy to enable training of deep neural networks is to use non-saturating activation functions to reduce the vanishing gradient problem. Popular choices that saturate only in the negative domain are the rectified linear unit (ReLU), its smooth, non-linear variant, Softplus, and the exponential linear units (ELU and SELU). Other functions are non-saturating across the entire real domain, like the linear parametric ReLU (PReLU). Here we introduce a nonlinear activation function called Soft++ that extends PReLU and Softplus, parametrizing the slope in the negative domain and the exponent. We test identical network architectures with ReLU, PReLU, Softplus, ELU, SELU, and Soft++ on several machine learning problems and find that: i) convergence of networks with any activation function depends critically on the particular dataset and network architecture, emphasizing the need for parametrization, which allows to adapt the activation function to the particular problem; ii) non-linearity around the origin improves learning and generalization; iii) in many cases, non-saturation across the entire real domain further improves performance. On very difficult learning problems with deep fully-connected and convolutional networks, Soft++ outperforms all other activation functions, accelerating learning and improving generalization. Its main advantage lies in its dual parametrization, offering flexible control of the shape and gradient of the function.

The auto-transformer rectifier unit (ATRU) is one of the most widely used avionic secondary power supplies. Timely fault identification and location of the ATRU is significant in terms of system reliability. A two-level fault diagnosis method for the ATRU using multi-source features (MSF) is proposed in this paper. Based on the topology of the ATRU, three key electrical signals are selected and analyzed to extract appropriate features for fault diagnosis. Mathematic expressions and simulation results of the feature signals under different fault modes are presented in the paper. Therefore, a unique MSF system is developed and a two-level fault diagnosis method based on radial basis function network groups is proposed. On the first level, the overall fault set is classified into three subsets and then on the second level, three radial basis function neural networks are constructed and trained to realize accurate fault localization. To verify the diagnosis performance of the proposed method, several comparative tests are implemented on a 12-pulse ATRU system, which shows that this method has a lower computational cost, better diagnostic accuracy and increased stability when compared with alternative methods.

CryptoNets and subsequent work have demonstrated the capability of homomorphic encryption (HE) in the applications of private artificial intelligence (AI). In convolutional neural networks (CNNs), many computations are linear functions such as the convolution layer which can be homomorphically evaluated. However, there are layers such as the activation layer which is comprised of non-linear functions that cannot be homomorphically evaluated. One of the most commonly used methods is approximating these non-linear functions using low-degree polynomials. However, using the approximated polynomials as activation functions introduces errors which could have a significant impact on accuracy in classification tasks. In this paper, we present a systematic method to construct HE-friendly activation functions for CNNs. We first determine what properties in a good activation function contribute to performance by analyzing commonly used functions such as Rectified Linear Units (ReLU) and Sigmoid. Then, we compare polynomial approximation methods and search for an optimal range of approximation for the polynomial activation. We also propose a novel weighted polynomial approximation method tailored to the output distribution of a batch normalization layer. Finally, we demonstrate the effectiveness of our method using several datasets such as MNIST, FMNIST, CIFAR-10.

The ever-increasing computation cost of Convolutional Neural Network (CNN) makes it imperative for real-world applications to accelerate the key steps especially the inference. In this work, we propose an efficient yet general scheme called Sparse Prediction Layer (SPL) which can predict and skip the trivial elements in the CNN layer. Pruned weights are used to predict the locations of maximum values in max-pooling kernels and those of positive values before Rectified Linear Units (ReLUs). Thereafter, the precise values of these predicted important elements are calculated selectively and the complete outputs are restored from them. Our experiments on ImageNet Large Scale Visual Recognition Competition (ILSVRC) 2012 show that SPL can reduce 68.3%, 58.6% and 59.5% Floating-point Operations (FLOPs) on AlexNet, VGG-16 and ResNet-50, respectively, within an accuracy loss of less than 1% without retraining. The proposed SPL scheme can further accelerate these networks pruned by other pruning-based methods, such as a FLOP reduction of 50.2% on the ResNet-50 which has been pruned by Channel Pruning (CP) before being applied with SPLs. A special matrix multiplication called Sparse Result Matrix Multiplication (SRMM) is proposed to support the implementation of SPL, and its acceleration effect is in line with expectations.

The ascent of Industry 4.0 and smart manufacturing has emphasized the use of intelligent manufacturing techniques, tools, and methods such as predictive maintenance. The predictive maintenance function facilitates the early detection of faults and errors in machinery before they reach critical stages. This study suggests the design of an experimental predictive maintenance framework, for conveyor motors, that efficiently detects a conveyor system's impairments and considerably reduces the risk of incorrect faults diagnosis in the plant; We achieve this remarkable task by developing a machine learning model that classifies whether the abnormalities observed are production-threatening or not. We build a classification model using a combination of time-series imaging and convolutional neural network (CNN) for better accuracy. In this research, time-series represent different observations recorded from the machine over time. Our framework is designed to accommodate both univariate and multivariate time-series as inputs of the model, offering more flexibility to prepare for an Industry 4.0 environment. Because multivariate time-series are challenging to manipulate and visualize, we apply a feature extraction approach called principal component analysis (PCA) to reduce their dimensions to a maximum of two channels. The time-series are encoded into images via the Gramian Angular Field (GAF) method and used as inputs to a CNN model. We added a parameterized rectifier linear unit (PReLU) activation function option to the CNN model to improve the performance of more extensive networks. All the features listed added together contribute to the creation of a robust future proof predictive maintenance framework. The experimental results achieved in this study show the advantages of our predictive maintenance framework over traditional classification approaches.

Rotating machinery has been developed with high complexity and precision, and bearings and gears are crucial components in the machinery system. Deep learning has attracted considerable attention from researchers in this area. The convolutional neural network (CNN) is a typical deep learning model that has a strong capability for automatically extracting features from raw data. This capability minimizes dependence on expert knowledge during feature extraction and selection. In CNN, hyperparameters, such as activation functions, can directly influence the performance of the model. In this study, the improved rectified linear units (ReLU)-CNNs are proposed for machinery fault diagnosis. The model's input are raw vibration signals without feature extraction and selection. It is experimentally validated for fault diagnosis using bearing and gearbox datasets. Results show that the proposed method can obtain satisfactory accuracy with enhanced convergence speed. For both datasets, the proposed method gives better diagnosis accrues than the other compared models. The proposed model can take advantages of standard ReLU-CNN, and these advantages can overcome traditional activation functions' vanishing gradient problems. Meanwhile, the improved ReLU-CNN has a new property that makes it perform better than the standard ReLU-CNN.

A method based on a convolutional neural network for the automatic classification of odontocete echolocation clicks is presented. The proposed convolutional neural network comprises six layers: three one-dimensional convolutional layers, two fully connected layers, and a softmax classification layer. Rectified linear units were chosen as the activation function for each convolutional layer. The input to the first convolutional layer is the raw time signal of an echolocation click. Species prediction was performed for groups of m clicks, and two strategies for species label prediction were explored: the majority vote and maximum posterior. Two datasets were used to evaluate the classification performance of the proposed algorithm. Experiments showed that the convolutional neural network can model odontocete species from the raw time signal of echolocation clicks. With the increase in m, the classification accuracy of the proposed method improved. The proposed method can be employed in passive acoustic monitoring to classify different delphinid species and facilitate future studies on odontocetes.

It is of great importance to construct a convolutional neural network architecture in the frequency domain to explore the theory of deep learning in the frequency domain. However, due to the complexity of the construction mechanism of the forward and backward pipelines needed to train the convolutional neural network in the frequency domain, higher requirements are put forward for the representation strategy of the frequency-domain activation function and pooling method in the forward and backward pipelines. Therefore, to construct a full frequency-domain convolutional neural network architecture, it is necessary to construct a frequency-domain representation strategy with a high classification accuracy and excellent time performance. In this paper, based on a chunk decomposition mechanism and the construction principle of the frequency-domain unsaturated activation function, a frequency-domain convolutional neural network architecture is proposed. Two important representation strategies are introduced into the frequency-domain forward/backward pipeline: a frequency-domain randomized offset rectified linear unit and a frequency-domain chunk max pooling method. The former can alleviate the vanishing and exploding gradient phenomena in the frequency-domain forward/backward pipeline and ensure the convergence of the convolutional neural network architecture in the frequency-domain training stage; the latter can capture the partial location information and characteristic strength of the frequency-domain neurons and improve the classification performance of the convolutional neural network in the frequency domain. This full frequency-domain convolutional neural network architecture improves the training accuracy of the convolutional neural network in the frequency-domain pipeline. The results show that on the basis of ResNet-50 as the backbone framework, an NVIDIA GeForce CUDA(Compute Unified Device Architecture) as the training pipeline, and $4\times 4$ as the activation block size of the third-level output neuron’s characteristic parameter matrix, the convolutional neural network architecture proposed in this paper can lower the top-1 error from 24.90% to 17.95%, the top-5 error from 12.85% to 9.23%. Furthermore, when the batch size is equal to 128 (in the worst-case bandwidth usage scenario), the acceleration ratio of the proposed architecture can still reach 13.0375 by selecting cuDNN as the reference model. Under the same backbone framework, the proposed architecture is tested on the MetData-1 dataset, and the classification accuracy can reach the maximum value; that is, the average difference is merely 0.18. This finding shows that the proposed architecture can improve the accuracy of the deep learning-based frequency-domain convolutional neural network model without reducing the time performance and expand the frequency-domain representation strategy of the frequency-domain activation function and pooling method.

In recent years, residual learning has shown excellent performance on convolutional neural network (CNN)-based single-image super-resolution (SISR) tasks. However, CNN-based SISR approaches have focused mainly on the design of deep architectures, and the rectified linear units (ReLUs) used in these networks hinder shallow-to-deep information transfer. As a result, these methods are unable to utilize some shallow information, and improving model performance is difficult. To solve the above issues, this paper proposes an image SR reconstruction method based on a generative adversarial network with a residual dense architecture. First, before ReLU activation, the number of feature channels is expanded by a factor of 6~9 using a $1\times 1$ convolutional layer, which improves the utilization of shallow information. Next, the original discriminator is replaced with a relativistic average discriminator, thereby improving the authenticity of the discriminative network. Finally, preactivation features are used to improve the perceptual loss, thus providing stronger monitoring for brightness consistency and texture restoration. Experimental results show that the proposed algorithm improves the utilization of shallow information in a deep network. Structural similarity (SSIM) index evaluations show that the overall utilization of shallow information is increased by 105.52%. In addition, the average runtime is 0.42 sec/frame, nearly 3.6 times faster than those of traditional methods. Moreover, the recovered images have an average natural image quality evaluator value of 3.4 and high perceptual quality, showing that the proposed method is suitable for image reconstruction applications in fields such as agriculture and medicine.

We construct a highly regular and simple structured class of sparsely connected convolutional neural networks with rectifier activations that provide universal function approximation in a coarse-to-fine manner with increasing number of layers. The networks are localized in the sense that local changes in the function to be approximated only require local changes in the final layer of weights. At the core of the construction lies the fact that the characteristic function can be derived from a convolution of characteristic functions at the next coarser resolution via a rectifier passing. The latter refinement result holds for all higher order univariate B-splines.

In this article, a novel edge computing system is proposed for image recognition via memristor-based blaze block circuit, which includes a memristive convolutional neural network (MCNN) layer, two single-memristive blaze blocks (SMBBs), four double-memristive blaze blocks (DMBBs), a global Avg-pooling (GAP) layer, and a memristive full connected (MFC) layer. SMBBs and DMBBs mainly utilize the depthwise separable convolution neural network (DwCNN) that can be implemented with a much smaller memristor crossbar (MC). In the backward propagation, we use batch normalization (BN) layers to accelerate the convergence. In the forward propagation, this circuit combines DwCNN layers/CNN layers with nonseparate BN layers, which means that the required number of operational amplifiers is cut by half as long as the greatly reduced power consumption. A diode is added after the rectified linear unit (ReLU) layer to limit the output of the circuit below the threshold voltage Vt of the memristor; thus, the circuit is more stable. Experiments show that the proposed memristor-based circuit achieves an accuracy of 84.38% on the CIFAR-10 data set with advantages in computing resources, calculation time, and power consumption. Experiments also show that, when the number of multistate conductance is 2⁸ and the quantization bit of the data is 8, the circuit can achieve its best balance between power consumption and production cost.

Training of deep neural networks is difficult due to vanishing gradients. Therefore, a pre-training procedure based on restricted Boltzmann machines is suggested to resolve this problem. However, new developments in deep learning aim to resolve the problem with vanishing gradients by using rectifier linear units (ReLU). This study compares the performance of a RBM pre-trained auto-encoder with sigmoid activations to the performance of auto-encoder with ReLU activation. The results showed that the ReLU auto-encoder achieved better reconstruction and saved training time, since it doesn't require pre-training .

Digital frauds get a dramatic increase over the years and lead to considerable losses. Detecting fraudulent attempts is valuable to many industries and especially to the banking and financial sectors. To help in anticipating and accurately identifying whether a transaction is fraudulent, machine learning-based models are the key solution for banking and financial institutions. In this paper, an artificial intelligence-based model was built using deep learning and was trained using stochastic gradient descent and feedforward neural networks. The dropout regularization has been utilized to enhance the generalization capabilities of the digital transaction classification model. Different activation functions were used and explored such as the max-out, the hyperbolic tangent, the rectifier linear unit, and the exponential rectifier linear unit. The impact of the learning rate on the model performance was analyzed. For the evaluation of the model, we did use of different metrics such as the accuracy, the precision, and the recall. The obtained results are promising, and the developed model can be used effectively to defend the banking sector against digital frauds.

Penelitian ini bertujuan untuk mengetahui pengaruh National Instruments Multisim (NI-Multisim) Software terhadap penguasaan rangkaian penyearah gelombang ditinjau dari kemampuan visual spasial (KVS) mahasiswa. Penelitian ini menggunakan pra-eksperimental design. Sampel penelitian adalah mahasiswa calon guru fisika yang mengambil mata kuliah Elektronika Dasar I yang dipilih melalui teknik sampling jenuh. Alat pengumpul data berupa instrumen tes kemampuan visual spasial dan instrumen tes rangkaian penyearah gelombang. Uji korelasi, regresi berganda, dan uji independent sample t-test digunakan untuk mengetahui pengaruh kemampuan spasial-visual terhadap hasil belajar rangkaian penyearah gelombang menggunakan NI-Multisim Software. Hasil menunjukkan bahwa: 1) Hasil belajar materi rangkaian penyearah gelombang menggunakan NI-Multisim Software berada pada kategori cukup; 2) Kemampuan visual spasial mahasiswa pada indikator KVS-intepretasi berkategori sangat tinggi, KVS-pola seri berkategori tinggi, KVS-refleksi berkategori tinggi, KVS-rotasi berkategori tinggi, KVS-proyeksi berkategori tinggi, dan KVS-menggambar berkategori sedang; 3) Kemampuan visul spasial mahasiswa memiliki korelasi yang positif dan signifikan terhadap hasil belajar mahasiswa dan memiliki kontribusi pengaruh sebesar 76,20% terhadap hasil belajar; 4) Hasil belajar mahasiswa yang memiliki KVS tinggi, lebih baik daripada rerata hasil belajar mahasiswa yang memiliki KVS rendah dalam mempelajari materi rangakaian penyearah gelombang menggunakan NI-Multisim Software.

A Nuclear Power Plant (NPP) is an advanced and complex power generation system that requires a high level of safety to operate. So, when confronted with an anomalous event, such as an accident or a transient scenario, if the event is not correctly identified quickly enough, it can spread rapidly in the system leading the plant to a risky situation. Because of this, various statistical methods and machine learning techniques have been employed aiming at developing a support system able to identify the occurring event in a short amount of time. This article shows the performance of the Deep Rectifier Neural Network (DRNN) in the nuclear accident identification problem. For this, a 5-layer DRNN with rectifier linear unit (ReLu) activation function in neurons of the hidden layers was developed, and 12 postulated accidents plus the normal operation condition were used for a Brazilian pressurized water reactor (PWR). The 5-Layer DRNN demonstrated good generalization capacity compared with other techniques at the literature, achieving a precision of 99.03% for the validation dataset.

Tanh is a sigmoidal activation function that suffers from vanishing gradient problem, so researchers have proposed some alternative functions including rectified linear unit (ReLU), however those vanishing-proof functions bring some other problem such as bias shift problem and noise-sensitiveness as well. Mainly for overcoming vanishing gradient problem as well as avoiding to introduce other problems, we propose a new activation function named Rectified Linear Tanh (ReLTanh) by improving traditional Tanh. ReLTanh is constructed by replacing Tanh’s saturated waveforms in positive and negative inactive regions with two straight lines, and the slopes of the lines are calculated by the Tanh’s derivatives at two learnable thresholds. The middle Tanh waveform provides ReLTanh with the ability of nonlinear fitting, and the linear parts contribute to the relief of vanishing gradient problem. Besides, thresholds of ReLTanh that determines the slopes of line parts are learnable, so it can tolerate the variation of inputs and help to minimize the cost function and maximize the data fitting performance. Theoretical proofs by mathematical derivations demonstrate that ReLTanh is available to diminish vanishing gradient problem and feasible to train thresholds. For verifying the practical feasibility and effectiveness of ReLTanh, fault diagnosis experiments for planetary gearboxes and rolling bearings are conducted by stacked autoencoder-based deep neural network (SAE-based DNNs). ReLTanh alleviates successfully vanishing gradient problem and the it learns faster, more steadily and precisely than Tanh, which is consistent with the theoretical analysis. Additionally, ReLTanh surpasses other popular activation functions such as ReLU family, Hexpo and Swish, which shows that ReLTanh has certain applying potential and researching value.

Recent researches have shown that spectral representation provides a significant speed-up in the massive computation workload of convolution operations in the inference (feed-forward) algorithm of Convolutional Neural Networks (CNNs). This approach results in reducing the computational complexity of the classification task, which makes spectral-based CNN suitable for implementation on embedded platform that typically has constrained resources. However, a major challenge in this approach is that the mathematical formulation of a nonlinear activation function in spectral (frequency) domain is currently not available; hence, computation of the activation functions in each layer has to be performed in the spatial domain. This results in several spatial-frequency domain switchings that are computationally very costly, and as such, it would be advantageous to strictly stay in the frequency domain. Hence, in this work, a novel Spectral Rectified Linear Unit (SReLU) for the activation function is proposed, that makes it possible for the computations to remain in the frequency domain, and therefore avoids the multiple compute-intensive domain transformations. To further optimize the classification speed of the network, an efficient spectral-based CNN model is presented that uses only the lower frequency components by way of fusing the convolutional and sub-sampling layers. Additionally, we provide and utilize a frequency domain equivalent of the conventional batch normalization layer that results in improving the accuracy of the network. Experimental results indicate that the proposed spectral-based CNN model achieves up to 17.02 × and 3.45 × faster classification speed (without considerable accuracy loss) on AT&T face recognition and MNIST digit/fashion classification datasets, respectively, as compared to the equivalent models in the spatial domain, hence outperforming conventional approaches significantly.

Deep convolutional neural networks (DCNNs) are one of the most promising deep learning techniques and have been recognized as the dominant approach for almost all recognition and detection tasks. The computation of DCNNs is memory intensive due to large feature maps and neuron connections, and the performance highly depends on the capability of hardware resources. With the recent trend of wearable devices and Internet of Things, it becomes desirable to integrate the DCNNs onto embedded and portable devices that require low power and energy consumptions and small hardware footprints. Recently stochastic computing (SC)-DCNN demonstrated that SC as a low-cost substitute to binary-based computing radically simplifies the hardware implementation of arithmetic units and has the potential to satisfy the stringent power requirements in embedded devices. In SC, many arithmetic operations that are resource-consuming in binary designs can be implemented with very simple hardware logic, alleviating the extensive computational complexity. It offers a colossal design space for integration and optimization due to its reduced area and soft error resiliency. In this paper, we present HEIF, a highly efficient SC-based inference framework of the large-scale DCNNs, with broad applications including (but not limited to) LeNet-5 and AlexNet , that achieves high energy efficiency and low area/hardware cost. Compared to SC-DCNN, HEIF features: 1) the first (to the best of our knowledge) SC-based rectified linear unit activation function to catch up with the recent advances in software models and mitigate degradation in application-level accuracy; 2) the redesigned approximate parallel counter and optimized stochastic multiplication using transmission gates and inverse mirror adders; and 3) the new optimization of weight storage using clustering. Most importantly, to achieve maximum energy efficiency while maintaining acceptable accuracy, HEIF considers holistic optimizations on cascade connection of function blocks in DCNN, pipelining technique, and bit-stream length reduction. Experimental results show that in large-scale applications HEIF outperforms previous SC-DCNN by the throughput of $4.1\times $ , by area efficiency of up to $6.5\times $ , and achieves up to ${5.6\times }$ energy improvement.

Pedestrian detection and tracking remains a highlight research topic due to its paramount importance in the fields of video surveillance, human-machine interaction, and tracking analysis. At present time, pedestrian detection is still an open problem because of many challenges of image representation in the outdoor and indoor scenes. In recent years, deep learning, in particular Convolutional Neural Networks (CNNs) became the state-of-the-art in terms of accuracy in many computer vision tasks. The unsupervised learning of CNNs is still an open issue. In this paper, we study a matter of feature extraction using a special activation function. Most of CNNs share the same architecture, when each convolutional layer is followed by a nonlinear activation layer. The activation function Rectified Linear Unit (ReLU) is the most widely used as a fast alternative to sigmoid function. We propose a bounded randomized leaky ReLU working in such manner that the angle of linear part with the highest input values is tuned during learning stage, and this linear part can be directed not only upward but also downward using a variable bias for its starting point. The bounded randomized leaky ReLU was tested on Caltech Pedestrian Dataset with promising results.

Neural networks with REctified Linear Unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU networks either pose assumptions on the underlying data distribution being e.g. Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can \emph{provably} train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory., 23 pages, 7 figures, work in progress

Packet classification in software-defined network has become more important with the rapid growth of Internet. Existing approaches focused on the data structure algorithms to classify the packets. But the existing algorithms lead to the problem of time budget and fails to accommodate large rule sets. Thus the key task is to design an algorithm for packet classification that inflicts process overhead, and the algorithm should handle large databases of classification rule. These challenging issues are achieved by proposing rectified linear unit deep neural network. The aim of this work is twofold. First various hyper-parameter values are analyzed in order to examine how they affect the packet classification performance of deep neural network; and their performance is compared with that of popular methods, e.g., K-nearest neighbor and support vector machines. The open-source TensorFlow deep learning framework with the support of NVidia GPU units is used to carry out this work, thus allowing a large number of rules to predict the exact flow. The result shows that the proposed method performs well, and hence, this model is more suitable for large classification rules.

Most state-of-the-art methods in recognizing rotated, scaled, and translated (RST) palmprint image have difficulty to achieve the tradeoff between accuracy and robustness. Inspired by vision processing mechanic, we propose a double biologically inspired transform (DBIT) network. DBIT involves two stages, and each stage has several layers. In the first stage, we first build a convolution layer to highlight the edges at all direction and interval. Then, we construct another two convolution layers to measure local spatial frequency (LSF), which can improve the selectivity of feature maps. Since there is no negative spike in biological neurons, a rectified linear unit and sum pooling are employed to produce a neuron spike. Then, each spike is normalized by their original orientation edge maps, and finally, all spikes are combined to a feature vector. In the second stage, two convolution layers including orientation edge detection and single LSF followed by several subsequent operations are performed once again, which transforms image shift to an invariant spike on final feature maps. Experiments on PolyU multispectral, PolyU II, CASIA, IITD, and COEP database illustrate the effectiveness and robustness of the proposed approach. Remark progresses in feature invariance and selectivity are achieved by the proposed method. Comparing with several existing methods, our approach can achieve better recognition performance under rotation, scaling, translation, and even noise interference.

A convolutional neural network (CNN) has recently demonstrated its outstanding capability for the classification of hyperspectral images (HSIs). Typical CNN-based methods usually adopt image patches as inputs to the network. However, a fixed-size image patch in HSI with complex spatial contexts may contain multiple ground objects of different classes, which will deteriorate the classification performance of the CNN. In addition, traditional convolutional layers adopted in the CNN have a huge amount of parameters needed to be tuned, which will cause high computational cost. To address the above-mentioned issues, a novel squeeze multibias network (SMBN) is proposed for HSI classification. Specifically, the proposed SMBN first introduces the multibias module (MBM), which incorporates multibias into the rectified linear unit layers. The MBM can decouple the feature maps of input patches into multiple response maps (corresponding to different ground objects) and adaptively select the meaningful maps for classification. Furthermore, the proposed SMBN replaces the traditional convolutional layer with a squeeze convolution module, which can greatly reduce the number of parameters in the network, thus saving the running time, while still maintaining high classification accuracy. Experimental results on three real HSIs demonstrate the superiority of the proposed SMBN method over several state-of-the-art classification approaches.