Search

The paper discusses algorithmic priority inversion in mission-critical machine inference pipelines used in modern neural-network-based perception subsystems and describes a solution to mitigate its effect. In general, priority inversion occurs in computing systems when computations that are "less important" are performed together with or ahead of those that are "more important." Significant priority inversion occurs in existing machine inference pipelines when they do not differentiate between critical and less critical data. We describe a framework to resolve this problem and demonstrate that it improves a perception system's ability to react to critical inputs, while at the same time reducing platform cost. [ABSTRACT FROM AUTHOR]

Purpose: With the growth of organizations and businesses, customer acquisition and retention processes have become more complex in the long run. That is why customer lifetime value (CLV) has become crucial to sales managers. Predicting the CLV is a strategic weapon and competitive advantage in increasing profitability and identifying customers with more splendid profitability and is one of the essential key performance indicators (KPI) used in customer segmentation. Thus, this paper proposes a stacked ensemble learning method, a combination of multiple machine learning methods, for CLV prediction. Design/methodology/approach: In order to utilize customers' behavioral features for predicting the value of each customer's CLV, the data of a textile sales company was used as a case study. The proposed stacked ensemble learning method is compared with several popular predictive methods named deep neural networks, bagging support vector regression, light gradient boosting machine, random forest and extreme gradient boosting. Findings: Empirical results indicate that the regression performance of the stacked ensemble learning method outperformed other methods in terms of normalized rooted mean squared error, normalized mean absolute error and coefficient of determination, at 0.248, 0.364 and 0.848, respectively. In addition, the prediction capability of the proposed method improved significantly after optimizing its hyperparameters. Originality/value: This paper proposes a stacked ensemble learning method as a new method for accurate CLV prediction. The results and comparisons support the robustness and efficiency of the proposed method for CLV prediction. [ABSTRACT FROM AUTHOR]

• Two-phase flow regime identification using ML with enhanced interpretability. • Evaluation of various dimensionality reduction techniques to optimize feature representation. • Kernel Fisher discriminant analysis is the best performing dimensionality reduction technique. • AutoML can impartially select the most effective ML classifier. • Our workflow can develop a virtual flow regime map to automate flow regime recognition. Accurate identification of flow regimes is paramount in several industries, especially in chemical and hydrocarbon sectors. This paper describes a comprehensive data-driven workflow for flow regime identification. The workflow encompasses: i) the collection of dynamic pressure signals using an experimentally verified numerical two-phase flow model for three different flow regimes: stratified, slug and annular flow, ii) feature extraction from pressure signals using Discrete Wavelet Transformation (DWT), iii) Evaluation and testing of 12 different Dimensionality Reduction (DR) techniques, iv) the application of an AutoML framework for automated Machine Learning classifier selection among K-Nearest Neighbors, Artificial Neural Networks, Support Vector Machines, Gradient Boosting, Random Forest, and Logistic Regression, with hyper-parameter tuning. Kernel Fisher Discriminant Analysis (KFDA) is the best DR technique, exhibiting superior goodness of clustering, while KNN proved to be the top classifier with an accuracy of 92.5 % and excellent repeatability. The combination of DWT, KFDA and KNN was used to produce a virtual flow regime map. The proposed workflow represents a significant step forward in automating flow regime identification and enhancing the interpretability of ML classifiers, allowing its application to opaque pipes fitted with pressure sensors for achieving flow assurance and automatic monitoring of two-phase flow in various process industries. [ABSTRACT FROM AUTHOR]

In this paper, we propose a variance reduction approach for Markov chains based on additive control variates and the minimization of an appropriate estimate for the asymptotic variance. We focus on the particular case when control variates are represented as deep neural networks. We derive the optimal convergence rate of the asymptotic variance under various ergodicity assumptions on the underlying Markov chain. The proposed approach relies upon recent results on the stochastic errors of variance reduction algorithms and function approximation theory. [ABSTRACT FROM AUTHOR]

This paper proposes a novel L1-based predictor–corrector method for the fractional differential equations involving generalized-Caputo type derivative. A decomposition scheme is used to obtain the three-point predictor–corrector formula. The error and stability of the proposed method are given in detail. A computer virus and a five-dimensional Hopfield neural network models are solved using the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, stochastic predictive computing networks are exploited to investigate the dynamics of the SIS with vaccination impact based epidemic model (SISV-EM) represented by nonlinear systems of stochastic differential equations (SDEs) by exploitation of artificial neural networks (ANNs) with the backpropagated Levenberg-Marquardt technique (BLMT) i.e., (ANNs-BLMT) to approximate the solution behavior. The stochastic nonlinear SISV-EM is governed with three classes: susceptible, infectious, and vaccinated populations. The referenced or target datasets for ANNs-BLMT are constructed by employing Euler-Maruyama (EM) scheme for solving stochastic differential systems in case of sufficiently various nonlinear SISV-EM scenarios by varying the percentage of vaccination for newly born, the coefficient of transmission, the natural mortality rates, the infectious rates of recovery, the rate at which vaccinated people lose their immunity, the rate of death caused by disease, the proportion of vaccinated against susceptible and the white noise in the environment. Based on arbitrary training, testing, and validation samples from the referenced dataset, the ANNs-BLMT provides an approximate solution for the stochastic nonlinear SISV-EM, with significant correlations to the referenced results. Exhaustive simulation-based results using error histograms, mean square errors, and regression analyses further demonstrate that the proposed ANNs-BLMT is efficient, consistent, and accurate for solving SISV-EM. • A two-layer framework of ANNs-BLMT is proposed as an innovative technique based on a stochastic computing paradigm to investigate the dynamics of stochastic nonlinear SISV-EM. • Innovative technique based on a stochastic computing ANNs-BLMT to investigate the dynamics of stochastic nonlinear SISV-EM Reference dataset for ANNs -BLMT is developed with Euler-Maruyama (EM) scheme nonlinear SISV-EM with varying parametersMean square error criteria is used for training of ANNs-BLMT to find approximate solutions to a variety of SISV-EM scenarios Computation of error histogram illustration, regression metrics, and MSE learning curves prove the performance of ANNs-BLMT. • The mean square error criteria are effectively utilized in approximation theory to develop an objective function for training the ANNs-BLMT to determine approximate solutions to a variety of stochastic nonlinear SISV-EM scenarios. • The computation of error histogram illustration, regression metrics, and MSE learning curves significantly improve the performance, precision, and consistency of the ANNs-BLMT for solving the stochastic nonlinear SISV-EM. [ABSTRACT FROM AUTHOR]

This article explores the question of whether artificial intelligence (AI) should be considered a coauthor of scientific papers. It provides a historical overview of AI, from its early beginnings to its current capabilities. The author argues that while AI can generate text, it lacks the ability to make new discoveries or truly understand concepts. The article also raises ethical concerns about the accuracy and truthfulness of AI-generated content. It concludes that while AI can assist in research, the responsibility for its use lies with human researchers, as AI tools are not yet capable of taking ethical responsibility for the research. [Extracted from the article]

In response to nonrenewable energy consumption and environmental pollution, the development of electric vehicles has accelerated. The safety of electric vehicles has garnered considerable attention. There are numerous incalculable foreign objects on the road that may collide with or scratch the battery-pack's frontal, resulting in battery-pack system damage or even explosion. This poses a significant risk to the safety of passengers and drivers. In this paper, a diversity of mechanical safety prediction models for battery-pack systems are proposed. These models support data-driven structural optimization of the battery-pack system by utilizing the numerical results of the bottom shell deformation. These simulation-based prediction models combine response surface method and machine learning algorithms. First, a nonlinear finite element model of the battery-pack system is established. The efficacy of the model is verified using constrained modal analysis in a variety of commercial software packages. Second, the collision simulations are executed and the data in various collision conditions are collected. Different sample sizes are used to develop various response surface models and machine learning models. The machine learning algorithms adopt support vector machine, Gaussian process regression, and neural network models. Third, the prediction accuracy of multiple prediction models is investigated according to error functions. The results show that the neural network model can predict the most accurate deformation under the condition of low speed frontal impact. The prediction average absolute percentage error of the neural network model is only 0.34% within the design domain, and only 2.54% outside the design domain. The proposed prediction model can be used for reliable design of the battery-pack system in electric vehicles. It also can be employed to design the early warning system of the battery-packs. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) for numerical solutions to partial differential equations (PDEs) have exhibited their remarkable merits of meshless methods, dimensionless features, and nonlinear approximation powers. The majority of conventional DNNs are mainly concentrated on the low- or high-dimensional PDE with smooth solutions. In singular problems the DNN does not behave so efficiently as in the smooth problem because of low regularities of the solution. In this paper, we propose a novel adaptive DNN for the PDE with corner singular solutions. In the corner singular problem gradients of solution vary rapidly around some particular vertices or lines. This poses a difficulty in development of DNN solvers. We design adaptive DNN techniques for the corner singularity, which incorporates (a) an adaptive loss function that assigns radial weights to local losses, (b) an adaptive activation function that varies the choice of activation functions from neuron to neuron, and (c) an adaptive sampling strategy that selectively focuses on sampling points near the singular locations. The adaptive DNN with these adaptive schemes greatly enhances approximation accuracy and computation efficiency. This is verified by a comprehensive series of numerical experiments of 2D and 3D corner singular problems. The comparison analyzes with the existing DNN solvers are also made to demonstrate effectiveness of the new method. The L 2 relative errors of the proposed adaptive DNN are reduced by an order of magnitude with respect to several conventional DNN methods. [ABSTRACT FROM AUTHOR]

The path planning and tracking control problem of autonomous vehicle with model matched uncertainty and external disturbance is studied in this paper. A safe lane change zone is designed by considering lane change time, distance, and driving speed. Meanwhile, the candidate paths are generated using cubic quasi-uniform B-spline curve, and the optimal path is selected based on feasibility, comfort, and efficiency criteria. The proposed planning method ensures an efficient and comfortable lane change path without collision with surrounding vehicles. Model uncertainty and external disturbance may affect lane change tracking control of autonomous vehicle. Hence, a fault-tolerant path tracking controller is implemented by using deep neural networks and adjustable zonotope-tube model predictive control. A deep neural network-based controller is proposed to compensate for model matched uncertainty, thereby enhancing the fault tolerance of the control system. Then, an adjustable zonotope-tube model predictive controller is designed to enclose the actual trajectory within a zonotope-tube with the nominal state as the center and the disturbance set as the radius by using the feedback control to achieve the asymptotic stabilization of the closed-loop system. The effectiveness of this path planning and tracking control method is finally verified by numerical simulation. • A lane change path planning method suitable for structured roads is proposed, which is based on the utilization of a safe lane change zone and cubic quasi-uniform B-spline curves. The proposed method ensures an efficient and comfortable lane change path. • A fault-tolerant path tracking controller is presented for autonomous vehicle, which incorporates a deep neural network and adjustable zonotope-tube model predictive control. The controller effectively handles model uncertainty and external disturbances. • The computation of robust positive invariant set is performed using the adjustable zonotope-tube method, which enhances scalability and reduces computational cost. [ABSTRACT FROM AUTHOR]

This paper concerns a new incremental approach to mining data streams. It is known that patterns in a data stream may evolve over time. In many cases, we need to track and analyze the nature of these changes. In the paper, the probabilistic neural networks are considered as basic models for tracking changes in data streams. We present globally convergent stream data mining algorithms applied to problems of regression, classification, and density estimation in a time-varying (drifting) environment. The algorithms are derived from the Parzen kernel-based probabilistic neural networks working in the online mode. For each problem, a theorem is presented ensuring the L 2 convergence of the algorithm designed for tracking drifting regression, density, or discriminant functions. Illustrative examples explain in detail how to choose the bandwidth of the Parzen kernel and the learning rate of the online algorithm. The performance of all algorithms is shown in exemplary simulations. It should be noted that this paper is one of very few, in the existing literature, presenting mathematically justified stream data mining algorithms. • The incremental version of the Generalized Regression Neural Network (IGRNN) able to track drifting regression functions. • The incremental version of the Probabilistic Neural Network (IPNN) working in non-stationary environments. • Application of IPNN for tracking drifting discriminant functions. • Mathematical proofs of the L 2 convergence of all the proposed estimators. [ABSTRACT FROM AUTHOR]

In this paper, we construct two deep neural network (DNN) approaches based on the hybridized discontinuous Galerkin (HDG) method for solving some elliptic problems. The main idea is to use an HDG scheme for spatial discretization and then estimate solutions using the deep learning idea. In fact, by employing DNN, we can achieve robust methods compared to classical methods for solving noisy and high-dimensional problems. A brief analysis shows that the loss functions corresponding to the proposed methods, which are called DNN-HDG-I and DNN-HDG-II, converge to zero as the mesh step size reduces. By testing several examples in one, two, and three dimensions, we demonstrate the performance of the proposed methods that show their apparent advantages especially compared to classical HDG methods. As we expected, the DNN-HDG methods can efficiently and accurately extract the pattern of the solutions in all three dimensions. To show the superiority of the proposed schemes, the DNN-HDG methods are compared with the classical HDG method for solving a problem involving some noisy data. Specifically, we employ the proposed methods for solving a problem which its exact solution is not accessible and compare it with the HDG solution. In this paper, the uniformly random data and Jacobi polynomial roots are considered, respectively, as the training data on the boundary and quadrature points for approximating integrals. Also, we find that the proposed methods are not required a high number of neurons (at most 30) and hidden layers (at most 10) for satisfactory results with acceptable accuracy. [ABSTRACT FROM AUTHOR]

Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation LRP to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study. • We present a novel audio dataset consisting of 30,000 audio samples of spoken digits. • We use LRP to explain predictions of two different models in the audio domain. • We confirm hypotheses about the neural networks' use of features from explanations. • We present audible explanations and demonstrate their superior interpretability. [ABSTRACT FROM AUTHOR]

Deep learning, with its powerful multilayer nonlinear representation of deep neural networks, enables models trained based on deep learning to describe the true distribution of data more accurately and thus have better generalization capabilities. However, the training data are usually assumed to obey independent identical distribution. Industrial process data nowadays usually have complex characteristics such as non-Gaussian, nonlinear, dynamic, strong correlation, and high-dimensional, which make it difficult to meet the assumptions of deep learning. Using industrial process data directly would result in low model accuracy and bias in the model output. This paper proposes a fault diagnosis method based on independent slow feature analysis (ISFA) and stacked sparse autoencoder (SSAE) network. The statistical properties possessed by ISFA are used to perform preliminary feature extraction of industrial process data, and then the extracted preliminary features are input to the SSAE network for fine-grained feature extraction. The proposed two-step feature extraction method not only makes the data input to the SSAE network as close as possible to obey independent identical distribution and provides convincing fault diagnosis results, but also makes full use of the powerful feature extraction capability of the SSAE network, which makes the proposed method also significantly improve the detection accuracy of incipient faults. The feasibility and superiority of the proposed method is verified through the TE process. • A two-step feature extraction fault diagnosis method is proposed. • Higher-order information and finer-grained features can be extracted. • The performance of both fault detection and classification has been validated. [ABSTRACT FROM AUTHOR]

In the paper, the global exponential stability for impulsive delayed Clifford-valued coupled neural networks is studied. Firstly, a delayed Clifford-valued coupled neural network model is established. So as to avoid the non-commutativity issue of Clifford numbers multiplication, the original n -dimensional Clifford-valued models are decomposed into 2 m n -dimensional real-valued models. Then, by using Lyapunov-Krasovskii (L-K) functional approach and the technique about linear matrix inequality, some new sufficient conditions are presented for the global exponential stability of the considered neural network models. Finally, the effectiveness of the results are verified by two numerical simulations. [ABSTRACT FROM AUTHOR]

For a class of nontriangular nonlinear systems in presence of unknown disturbances, we propose a predictor-based neural dynamic surface control (PNDSC) strategy in this paper. This nontriangular system is transformed via the mean value theorem, and a predictor is then constructed. To avoid an algebraic loop problem, partial state vectors are employed as input signals of neural networks (NNs) for approximating unknown dynamics, and compensation items are designed to compensate for approximation errors from NNs. Different from the traditional NDSC, the PNDSC in this paper utilizes prediction errors to update learning parameters for improving NNs’ learning behaviors with overlarge adaptive gains. On the basis of improved NNs’ approximation behaviors, a predictor-based NNs disturbance observer (PNNDO) is constructed for compensation for external disturbances and approximation errors from NNs. Furthermore, with predictors, a normalization method of weights is developed to reduce the number of online learning parameters. On the basis of the aforementioned result, measurement noises are taken into account in our predictor-based neural control strategy. We employ predictor states, rather than measurement information paralyzed by noises, in design of our control strategy. This reduces high-frequency oscillations in control input. A Lyapunov-based stability analysis shows that all signals are ultimately bounded in the closed-loop system. Finally, the effectiveness of the proposed control strategy is verified by a numerical example and a permanent magnet brushless DC motor system. [ABSTRACT FROM AUTHOR]

This paper investigates the problem of master-slave synchronization of stochastic quaternion-valued neural networks (SQVNNs) with mixed time-varying delays. A linear feedback controller is developed to explore the global synchronization of the proposed system by utilizing the complete information of the time-delay state. Sufficient conditions for synchronization of the proposed model are derived by constructing appropriate Lyapunov–Krasovskii functional by applying the master-slave synchronization method of master-slave and some integral inequality techniques. Finally, a corresponding numerical simulation is presented to demonstrate the accuracy of the theoretical results. This paper introduces a unique and efficient image encryption algorithm based on SQVNNs. This technique utilizes the solution set of SQVNNs to generate the high-level randomness secret keys to encrypt the source image. Finally, we conclude that the algorithm yields a source image cipher with excellent diffusion and confusion properties. A few test clinical images are utilized to show the validity of the proposed method. Several performance analyses show that the proposed algorithm for image encryption gives an efficient and secure way to deal with the Internet of Health Things (IoHT). [ABSTRACT FROM AUTHOR]

The complex-valued flow matrix Drazin inverse has recently attracted considerable interest from researchers due to its great academic value. In this paper, a fixed-time convergence integral-enhanced zeroing neural network (FTCIEZNN) model is proposed and investigated for calculating the Drazin inverse of complex-valued flow matrix. Since the FTCIEZNN model possesses fixed-time convergence, its upper limit of convergence time is irrelevant to initial conditions and can be adjusted by specified system parameters. Meanwhile, by adopting the newly designed reformed nonlinear activation function (RNAF) and variable parameters, the FTCIEZNN model converges rapidly in a relatively fast fixed-time and its robustness is dramatically strengthened. In addition, the upper limit of the convergence time in the absence of noise and the upper limit of the steady-state error in the presence of time-varying bounded noise are given by a scrupulous mathematical logic calculation. Furthermore, the outcomes of the numerical simulations demonstrate that the FTCIEZNN model outshines existing zeroing neural network models in calculating complex-valued flow matrix Drazin inverse. Finally, an application based on the FTCIEZNN model in image encryption fully illustrates the practical value of the FCIEZNN model. [ABSTRACT FROM AUTHOR]

At present, there are few studies on solving time-variant linear equality and inequality systems (TVLEIS) under noise interference, and the numerical algorithm has limitations in solving the TVLEIS problems. Therefore, to determine the online solution of the TVLEIS in a complex environment, two prescribed-time robust zeroing neural network (PTRZNN) models are proposed, investigated, and verified in this paper. The PTRZNN models have a faster convergence rate and superior robustness compared with other zeroing neural network models activated by common activation functions. In addition, the detailed theoretical derivation of the prescribed-time convergence and robustness of the PTRZNN models is provided. The effectiveness and superiority of the PTRZNN models for determining the TVLEIS are further demonstrated by simulation results. It is worth mentioning that the design idea of the PTRZNN models is applied to the multi-agent system, which shows the practical value of the PTRZNN models. [ABSTRACT FROM AUTHOR]

This article deals with global μ -stabilization of quaternion-valued inertial bidirectional associative memory neural networks (QVIBAMNNs) with time-varying delays via time-delayed impulsive control. Firstly, the existence and uniqueness of the equilibrium point are proved by the homeomorphism mapping method. Then, based on an improved differential inequality, sufficient conditions for global μ -stabilization of QVIBAMNNs via time-delayed impulsive control are obtained. Significantly, the activation function in this paper is an extension of Lipschitz condition, which is superior to general quaternion-valued activation function. And quaternion is not required to be decomposed into real or complex values in this paper. Ultimately, some numerical simulations have proved the feasibility of the theorems. [ABSTRACT FROM AUTHOR]

SRAM based computing-in-memory (SRAM-CIM) techniques have been widely studied for neural networks (NNs) to solve the “Von Neumann bottleneck”. However, as the scale of the NN model increasingly expands, the weight cannot be fully stored on-chip owing to the big device size (limited capacity) of SRAM. In this case, the NN weight data have to be frequently loaded from external memories, such as DRAM and Flash memory, which results in high energy consumption and low efficiency. In this paper, we propose a hybrid-device computing-in-memory (HD-CIM) architecture based on SRAM and MRAM (magnetic random-access memory). In our HD-CIM, the NN weight data are stored in on-chip MRAM and are loaded into SRAM-CIM core, significantly reducing energy and latency. Besides, in order to improve the data transfer efficiency between MRAM and SRAM, a high-speed pipelined MRAM readout structure is proposed to reduce the BL charging time. Our results show that the NN weight data loading energy in our design is only 0.242 pJ/bit, which is 289 $\times $ less in comparison with that from off-chip DRAM. Moreover, the energy breakdown and efficiency are analyzed based on different NN models, such as VGG19, ResNet18 and MobileNetV1. Our design can improve $\mathbf {58\times \,\,to\,\,124\times }$ energy efficiency. [ABSTRACT FROM AUTHOR]

Despite their massive size, successful deep artificial neural networks can exhibit a remarkably small gap between training and test performance. Conventional wisdom attributes small generalization error either to properties of the model family or to the regularization techniques used during training. Through extensive systematic experiments, we show how these traditional approaches fail to explain why large neural networks generalize well in practice. Specifically, our experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data. This phenomenon is qualitatively unaffected by explicit regularization and occurs even if we replace the true images by completely unstructured random noise. We corroborate these experimental findings with a theoretical construction showing that simple depth two neural networks already have perfect finite sample expressivity as soon as the number of parameters exceeds the number of data points as it usually does in practice. We interpret our experimental findings by comparison with traditional models. We supplement this republication with a new section at the end summarizing recent progresses in the field since the original version of this paper. [ABSTRACT FROM AUTHOR]

An automatic inductor design process is helpful to reduce the design cycle of radio frequency (RF) integrated circuit (IC). This paper proposed an efficient synthesis-analysis machine (SAM) for on-chip inductor synthesis and modeling, as well as an automatic dataset generation (ADG) topology for the generation of artificial neural networks (ANNs) training dataset. The SAM consists of a synthesis ANN, two analysis ANNs and a proposed self-inspection machine (SIM). For a given design request, the SAM synthesizes a layout first, followed by analyzing the performance of the layout automatically, and finally improves the layout confidence through self-inspection. Compared to the modeling functions of analysis ANN, the synthesis ANN behaves as an inverse model, of which the inputs are desired inductor performances while the outputs are geometrical parameters of the layout. However, multi-value problems might occur in obtaining geometries of inductors, suggesting with evidence from mathematic relationships between geometrical and electrical parameters of an inductor. A multi-valued training dataset will mislead the synthesis ANN, resulting in unqualified layouts. To deal with this issue, a solution space contraction technique (SSCM) is also proposed. Furthermore, the SIM suspends most of the failures by comparing the output of analysis ANN back to the input of synthesis ANN. An electromagnetic simulation tool is used for experiments, and the effectiveness of the proposed SAM is proven by 6,500 inductor samples. [ABSTRACT FROM AUTHOR]

When deploying deep neural networks (DNNs), edge devices training is practical to improve model adaptivity for various user-specific scenarios while avoiding privacy disclosure. However, the training computation is intolerable for edge devices. It inspires sparse DNN training (SDT) into the limelight, which reduces training computation by dynamic weight pruning. Generally, SDT has two strategies based on the pruning granularity: the structured or the unstructured. Unfortunately, both of them suffer from limited training efficiency due to the gap between pruning granularity and hardware implementation. The former is hardware-friendly but has a low pruning ratio, indicating limited computation reduction. The latter has a high pruning ratio, but the unbalanced workload decreases utilization and irregular sparsity distribution causes considerable sparsity processing overhead. This paper proposes a software-hardware co- design to bridge the gap for improving the efficiency of SDT. On the algorithm side, a sub-structured pruning method, achieved with hybrid shape-wise and line-wise pruning, generates a high sparsity ratio and keeps the hardware-friendly property. On the hardware side, a sub-structured weight processing unit (SWPU) effectively handles the hybrid sparsity with three techniques. First, SWPU dynamically reorders the computation sequence with hamming-distance-based clustering, balancing the irregular workload. Second, SWPU performs runtime scheduling by exploiting the feature of sub-structured sparse convolution through a detect-before-load controller, which skips redundant memory access and sparsity processing. Third, SWPU performs sparse convolution by compressing operands with spatial disconnect log-based routing and recovers their location with bi-directional switching, avoiding the power-consumed routing logic. Synthesized with 28nm CMOS technology, SWPU can enable 0.56V-to-1.0V supply voltage with a maximum frequency of 675 MHz. It achieves a 50.1% higher pruning ratio than structured pruning and $1.53\times $ higher energy efficiency than unstructured pruning. The peak energy efficiency of SWPU is 126.04TFLOPS/W, outperforming the state-of-the-art training processor by $1.67\times $. When training a ResNet-18 model, SWPU reduces $3.72\times $ energy and offers $4.69\times $ speedup than previous sparse training processors. [ABSTRACT FROM AUTHOR]

This paper proposes CMOS synapse and neuron for use in spiking neural networks to perform cognitive functions in a bio-inspired manner. The proposed synapse can trace the eligibility of the timing relationship between pre- and post-synaptic spikes, supporting a bio-plausible local learning rule called the spike timing-dependent plasticity (STDP) in an energy- and area-efficient manner. The proposed neuron can support neural functions such as synaptic current integration, threshold-based firing, neuronal leaking, membrane potential resetting, and adjustable refractory period with improved energy and area efficiency. The STDP curve shape of the synapse and the firing rate of the neuron can be adjusted as desired. Their variability due to process, voltage, and temperature (PVT) variations can also be minimized. The proposed CMOS neuron and synapse circuits were designed in a 28-nm CMOS process. The performance evaluation results indicate that the proposed synapse reduces energy consumption and area by up to 94% and 43% compared to conventional CMOS synapses. They also indicate that the proposed neuron achieves energy and area reductions of 37% and 23%, respectively, compared to conventional CMOS neurons. An associative neural network composed of the proposed neuron and synapse was designed to verify that they together work well for performing a cognitive function of associative learning and inferencing. [ABSTRACT FROM AUTHOR]

• Local and global self-attention mechanisms are used for character embedding. • CNN with multi-size filters are used to extract character information for NER. • A cross-attention method that fuses character and word embedding for NER is proposed • A modified Mogrifier LSTM is presented to improve the performance of NER. • Proposed methods integrated with a transformer-based model achieve good performance. Clinical and biomedical concept extraction is critical in medical analysis using clinical and biomedical documents from professional literature, EHRs and PHRs. Named entity recognition (NER) accurately marks essential information in the literature based on the characteristics of the target entity, providing a method for extracting clinical and biomedical concepts. The performance of NER is heavily embedding-dependent, so recent studies have proposed the method of generating word embedding from character-level information, which can strengthen the representation ability for word embedding. In this paper, we present a novel neural network model including an attention mechanism network and a convolutional neural network (CNN) to further improve character-level embedding. First, an attention mechanism is applied simultaneously to the local and global character embedding. Then, a CNN with multi-size filters is used to extract more information from the character level, which can capture more meaningful features from words with various lengths. In addition, a cross-attention method is used to leverage the interaction between word embedding and character embedding to generate the final word representation. Finally, we modified Mogrifier LSTM to make it suitable for NER tasks and integrated it into our model. Experimental results show that our method is effective and that the model performs better than the baseline models. We also apply our methods proposed in this paper to the transformer-based model and obtain a 90.36 F1-score on NCBI-Disease. [ABSTRACT FROM AUTHOR]

The divider is one of the most complex hardware units in Stochastic Computing (SC); even though several new designs have been presented to reduce the computation latency of the conventional divider, all of them still require a considerable number of clock cycles. Moreover, they incur in low performance due to the employed arithmetic computational scheme. In this paper, a Delta Sigma Modulator (DSM) based stochastic divider is proposed. As an entirely digital circuit, the proposed divider offers the best computation latency and accuracy over all existing stochastic dividers found in the technical literature (with a typical reduction between 66.8% and 96.9% in the number of clock cycles and a reduction from $10^{\mathrm {-3.4}}$ to $10^{\mathrm {-3.9}}$ in the average mean square error for a 10-bit resolution). An SC-based Neural Network (NN) is considered as an initial case study to evaluate the advantages of the proposed design in an emerging application; results show that the proposed divider enables an SC-based NN to achieve a higher classification accuracy and hardware efficiency than existing designs. To show the flexibility of the proposed divider design, its application to Sobol-based sequences is also presented; also in this case, its superiority over other designs is confirmed. These features make the proposed design very attractive for hardware-constrained platforms; moreover, such a novel design approach that incorporates ideas from analog/mixed signal circuit design into a digital circuit design, can motivate other researchers to design efficient SC designs using similar schemes. [ABSTRACT FROM AUTHOR]

The implications of neural synchronization extend beyond brain function, and can impact the development of artificial neural networks. This paper explores the synchronization of multiple fractional-order fuzzy complex-valued spatiotemporal neural networks (MFOFCVSNNs), which is novel and characterized using fuzzy logic and fractional-order partial differential equations, making it more adaptable and versatile. We first establish a new fractional-order complex-valued partial differential inequality, an integer-order complex-valued partial differential inequality, and an equation. Then, by combining the Lyapunov method with fuzzy set theory, employing newly established inequalities and equations, along with a newly designed fuzzy pinning controller, we derive two linear matrix inequality (LMI) formulations of synchronization criteria for MFOFCVSNNs using a direct non-complex decomposition approach. These criteria exhibit different dependencies on the membership function, with one being independent and the other dependent. Importantly, the criterion based on the membership function demonstrates reduced conservatism compared to its independent counterpart. By leveraging M -matrix theory, we present the synchronization criteria in a concise low-dimensional form. Moreover, this paper extends and enhances previous findings, resulting in reduced conservatism. Finally, we validate our theoretical analysis through numerical simulations. • A new model is proposed-multiple fractional-order fuzzy complex-valued spatiotemporal neural networks, which is novel and characterized using fuzzy logic and fractional-order partial differential equations, making it more adaptable and versatile. • Proposing a novel fuzzy pinning controller for large-scale networks, it streamlines implementation complexity and enhances design flexibility by not requiring identical fuzzy parameters as the model. • Exploring synchronization among multiple fractional-order fuzzy complex-valued delayed spatiotemporal neural networks without utilizing the complex-valued decomposition method. • The membership-function-independent synchronization criterion and the membership-function-dependent synchronization criterion are established. [ABSTRACT FROM AUTHOR]

This paper studies the fixed-time synchronization of a class of quaternion-valued neural networks (QVNNs) with time delays and impulses. In contrast to some existing decomposition methods for studying this problem, this paper presents several strategies to simplify the implicit Lyapunov function method from both controller and implicit function equation perspectives. The benefits of these strategies are, in the first strategy, the controller does not contain the Lyapunov function so that the controller will be more general, and in the second strategy, the implicit function equation is simpler so that the restriction on the bounded Lyapunov function can be dispensed with, which also makes the technique of handling time delays simpler. Furthermore, three sufficient conditions for fixed-time synchronization of the above QVNN are presented, despite the impulsive influence. Finally, we verify the feasibility of our methods with three examples. • Using a flexible implicit function approach to study fixed-time synchronization of QVNNs. • Fixed-time synchronization is studied without sign functions in the controller. • Non-decomposition method is utilized. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are vulnerable to adversarial examples generated by adding subtle perturbations to benign inputs. While these perturbations are somewhat small due to the L p norm constraint, they are still easily spotted by human eyes. This paper proposes Perceptual Sensitive Attack (PS Attack) to address this flaw with a perceptually adaptive scheme. We add Just Noticeable Difference (JND) as prior information into adversarial attacks, making image changes in areas that are insensitive to the human eyes. By integrating the JND matrix into the L p norm, PS Attack projects perturbations onto the JND space around clean data, resulting in more imperceivable adversarial perturbations. PS Attack also mitigates the trade-off between the imperceptibility and transferability of adversarial images by adjusting a visual coefficient. Extensive experiments manifest that combining PS attacks with state-of-the-art black-box approaches can significantly promote the naturalness of adversarial examples while maintaining their attack ability. Compared to the state-of-the-art transferable attacks, our attacks reduce LPIPS by 8% on average when attacking typically-trained and defense models. [ABSTRACT FROM AUTHOR]

Recent researches show that deep learning model is susceptible to backdoor attacks. Many defenses against backdoor attacks have been proposed. However, existing defense works require high computational overhead or backdoor attack information such as the trigger size. In this paper, we propose a novel backdoor detection method based on intentional adversarial perturbations. The proposed method leverages intentional adversarial perturbation to detect whether an image contains a trigger, which can be applied in both the training stage and the inference stage (sanitize the training set in training stage or detect the backdoor instances in inference stage). Specifically, given an untrusted image, the adversarial perturbation is added to the image intentionally. If the prediction of the model on the perturbed image is consistent with that on the unperturbed image, the input image will be considered as a backdoor instance. Compared with most existing defense works, the proposed method is faster and introduces less computational overhead during backdoor detection process. Moreover, the proposed method maintains the visual quality of the image (as the ℓ 2 norm of the added perturbation is as low as 2.8715, 3.0513 and 2.4362 on Fashion-MNIST, CIFAR-10 and GTSRB datasets, respectively). Experimental results show that, for general backdoor attack, the backdoor detection rate of the proposed defense method is 99.63%, 99.76% and 99.91% on Fashion-MNIST, CIFAR-10 and GTSRB datasets, respectively. For invisible backdoor attack, the backdoor detection rate of the proposed defense method is 99.75% against blended backdoor attack and 98.00% against sample-specific backdoor attack. It is also demonstrated that the proposed method can achieve high defense performance against backdoor attacks under different attack settings (trigger transparency, trigger size and trigger pattern). In addition, the experimental comparison with related work demonstrates that the proposed method has better detection performance and higher detection efficiency. [ABSTRACT FROM AUTHOR]

To further enhance the reliability of Machine Learning (ML) systems, considerable efforts have been dedicated to developing privacy protection techniques. Recently, membership privacy has gained increasing attention, with a focus on determining whether a specific data point is present in the confidential training set of an ML model. However, most current attacks only prioritize attack accuracy and fail to extend their range to the evidence that contributes to the member/non-member classification. This limitation greatly reduces the practicality of Membership Inference Attacks (MIA), as real-world data typically includes multiple features, making it challenging to identify which features are involved in the sensitive training set. Therefore, this paper targets one of the fundamental challenges in membership inference attack: measuring the distance between an attack sample and a member sample. Specifically, we propose a novel threat model called Membership Reconstruction Attack (MRA), which aims to reconstruct the exact distribution of the target training set. MRA achieves this by marking each input dimension (e.g., pixels) according to its similarity to the target dataset in feature space. Our attack demonstrates its effectiveness across various settings, including different major datasets (MNIST, CIFAR-10, CIFAR-100) and different model architectures (AlexNet, ResNet, DenseNet, and generative models). Additionally, we evaluate MRA from the defenders' perspective and test several defense approaches against our attack. [ABSTRACT FROM AUTHOR]

This paper presents a novel interface technique employing Artificial Neural Networks (ANN) for efficient data transfer between incompressible fluid and deformable solid domains in a partitioned fluid–structure interaction (FSI) framework. Governing differential equations (GDE) with potential flow assumptions are solved to obtain pressure values at computational nodes in the fluid domain. Pressure values are then used to train an ANN-based model to interpolate pressure loads for application at non-collocated nodes on the solid boundary. Both finite element method (FEM) and meshfree element free Galerkin (EFG) formulations are used to calculate the solid deformation. Proposed methodology is applied to solve a two-dimensional (2D) unidirectional flow problem, consisting of a flexible cantilever beam immersed in a laminar, steady, and inviscid fluid. Overwhelming mathematical intricacies of computational solvers are avoided for the sake of simplistic interface treatment, flux transfer and associated phenomena. Dedicated partitioned solvers are developed for both solid and fluid domains, which are individually benchmarked for established problems from the literature. The presented ANN-based interpolation scheme provides an alternative to higher-order polynomial algorithms at the interface boundary. The proposed scheme is simple to employ, computationally efficient, and offers competitive accuracy as well as stability. • A novel ANN-based interface coupling approach for partitioned FSI problems. • Separate solvers, employing FDM for fluid domain, and FEM/EFG for solid domain. • Partitioned solvers are coupled at interface using ANN-based approximation function. • Steady unidirectional (one-way) FSI to couple FDM-ANN-FEM and FDM-ANN-EFG solvers. • ANN interpolate pressure from fluid to solid, better than polynomial-based fitting. • A generic ANN framework, extendable to problems with large deformations and velocities. [Display omitted] [ABSTRACT FROM AUTHOR]

Deep neural networks easily suffer from weak generalization caused by overfitting on biased data. One popular remedy to alleviate this issue is sample reweighting methods that adaptively adjust the importance of biased samples. Separate from the effort to reduce bias, recent works show that the generalization power can be improved by auxiliary tasks. Inspired by the two lines of works, we extend the sample reweighting methods to auxiliary tasks. In this paper, we propose a novel auxiliary learning framework that improves the primary task by adaptively adjusting the weights of samples from multiple tasks rather than samples from a single task using a weighting function. The weighting function is optimized by meta-learning along the gradient of the loss for meta-data, which is a small unbiased validation data. We also present a task-activation score that indicates the correlation between the learning tendency of the training samples and meta-data samples. This score is utilized as a regularizer for meta-learning objective. Our framework can obtain powerful representations for the primary task on biased data by automatically identifying effective combinations of tasks. Our experiments demonstrate that our proposed method consistently outperforms all baselines and state-of-the-art methods on both corrupted labels and class imbalance settings. [ABSTRACT FROM AUTHOR]

• Second law aspects of hybrid nanofluid in a MCHS whit porous fins are examined. • The water based silver/graphene hybrid nanofluid is used as coolant. • Frictional entropy generation augments by boosting the nanoparticle concentration. • By rising the flow rate, thermal entropy generation declines. • Artificial neural network is employed to obtain a model for the entropy generation. This paper evaluates the characteristic of second law of thermodynamic, including Bejan number and entropy generation for hybrid nanofluid containing graphene-silver nanofluid through a MCHS whit porous fins. Finite-volume technique is utilized to solve the governing equations. To simulate the problem, different porous medium thicknesses, nanoparticle concentrations, and inlet mass flow rates are used while the heat flux remains constant. The minimum values of the frictional and thermal entropy generation are 5 × 10−4 and 6.25 × 10−2, while the maximum values are 3.2 × 10−4 and 9.75 × 10−2. With increasing nanoparticle concentration up to 0.06% wt at constant porous thickness t p =200 µm, frictional entropy generation rises up by 3 × 10−5 and heat transfer rate go up while, thermal entropy generation decreases by 1.5 × 10−2. In addition, by doubling the input mass flow rate and reaching 0.02% at constant nanoparticle concentration (0.06%), thermal entropy generation decreases by 2 × 10−2 while the frictional entropy generation increases by 2.4 × 10−4. The minimum magnitude of Bejan number is 0.994. This show that the irreversibility is derived significantly from thermal entropy generation rate. Finally, an artificial neural network is employed to obtain a model for entropy generation. [ABSTRACT FROM AUTHOR]

In recent years we have seen a growing number of edge devices adopted by consumers, in their homes (e.g., smart cameras and doorbells), in their cars (e.g., driver assisted systems), and even on their persons (e.g., smart watches and rings). Similar growth is reported in industries including aerospace, agriculture, healthcare, transport, and manufacturing. At the same time that devices are getting smaller, Deep Neural Networks (DNN) that power most forms of artificial intelligence are getting larger, requiring more compute power, memory, and bandwidth. This creates a growing disconnect between advances in artificial intelligence and the ability to develop smart devices at the edge. In this paper, we present a novel approach to running state-of-the-art AI algorithms at the edge. We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks (BWN) and XNOR-Networks. In BWN, the filters are approximated with binary values resulting in 32× memory saving. In XNOR-Networks, both the filters and the input to convolutional layers are binary. XNOR-Networks approximate convolutions using primarily binary operations. This results in 58× faster convolutional operations (in terms of number of the high precision operations) and 32× memory savings. XNORNets offer the possibility of running state-of-the-art networks on CPUs (rather than GPUs) in real-time. Our binary networks are simple, accurate, efficient, and work on challenging visual tasks. We evaluate our approach on the ImageNet classification task. The classification accuracy with a BWN version of AlexNet is the same as the full-precision AlexNet. [ABSTRACT FROM AUTHOR]

The complexity and size of recent deep neural network (DNN) models have increased significantly in pursuit of high inference accuracy. Chiplet-based accelerator is considered a viable scaling approach to provide substantial computation capability and on-chip memory for efficient process of such DNN models. However, communication using metallic interconnects in prior chiplet-based accelerators poses a major challenge to system performance, energy efficiency, and scalability. Photonic interconnects can adequately support communication across chiplets due to features such as distance-independent latency, high bandwidth density, and high energy efficiency. Furthermore, the salient ease of broadcast property makes photonic interconnects suitable for DNN inference which often incurs prevalent broadcast communication. In this paper, we propose a scalable chiplet-based DNN accelerator with photonic interconnects named ASCEND. ASCEND introduces (1) a novel photonic network that supports seamless intra- and inter- chiplet broadcast communication, and flexible mapping of diverse convolution layers, and (2) a tailored dataflow that exploits the ease of broadcast property and maximizes parallelism by simultaneously processing computations with shared input data. Simulation results using multiple DNN models show that ASCEND achieves 71% and 67% reduction in execution time and energy consumption, respectively, as compared to other state-of-the-art chiplet-based DNN accelerators with metallic or photonic interconnects. [ABSTRACT FROM AUTHOR]

As model size and the number of layers increase, Deep Neural Networks (DNNs) demand enormous computational power and throughput to meet exceedingly high prediction accuracy’s of today’s machine learning (ML) applications. Spatial hardware accelerators have been proposed that optimize the dataflow and exploit sparsity to provide a significant decrease in power consumption. As spatial architectures are traditionally designed with metallic interconnects, significant power is expended for data movement for different dataflows. In this paper, we exploit extended wireless technology to design a power-efficient and high-throughput DNN accelerator, e-WiNN, that can be configured for all representative dataflows and arithmetic precisions. We leverage novel circuit design by utilizing Dadda-algorithm based Multiply-and-Accumulate (MAC) circuits for 4-bit, 8-bit and 16-bit inputs to reduce area, power and delay constraints in 14 nm predictive technology. Our novel wireless transmitter integrates on- off keying (OOK) modulator with power amplifier that results in significant energy savings. To reduce the area overhead, we cluster wireless transceivers into groups of four such that both weights and input features can be effectively multicast to reduce the data movement. The energy efficient transceiver circuit is implemented in state-of-the-art BSIM 32 nm FinFET technology model and our link budget considers required RF power for different frequencies and inter-PE distance at three different antenna directivities including isotropic. Our detailed RTL modeling and cycle-accurate simulation results show that e-WiNN achieves 36.3% latency reduction and 76.1% energy saving when compared to state-of-art wire interconnected accelerators; 70.3% area reduction and 41.6% energy saving at the cost of 11% latency increase when compared to prior wireless accelerators on various neural networks (AlexNet, VGG16, and ResNet-9/50). [ABSTRACT FROM AUTHOR]

Neural networks (NNs) with large scales of artificial neurons are increasingly used in recognition and classification tasks. In power-constrained scenarios, the tradeoff between performance and hardware consumptions must be carefully evaluated before silicon tape-out. In this paper, we proposed a systematic approach to design ultra-low power NN system. This work is motivated by the facts that NNs are resilient to approximation in many of the computations and NNs are outputting statistical tensors which are acceptable to less-than-perfect results. We resort to the front-back end approach with a twofold aim: (1) a fast and accurate design approach is proposed by estimating the computing quality of low-power approximate adder arrays, and it is adopted to evaluate the neural network system; (2) a quality configurable engine with different approximation degrees while processing NNs is implemented. The proposed work is demonstrated with a comprehensive keyword spotting (KWS) system as an ultra-low power NN engine. The experimental environment is setup with ten keywords from the google speech command dataset (GSCD) using an industrial 22-nm ultra-low-leakage (ULL) process. Comparing to the state-of-the-art KWS processors, the proposed approximate NN engine can demonstrate over 60% improvement in power efficiency and $1.1\times $ area efficiency while achieving similar recognition accuracy. [ABSTRACT FROM AUTHOR]

This paper deals with global synchronization problem of multiple discrete-time Markovian jump memristor-based neural networks (DTMJMNNs) with mixed mode-dependent delays via a novel event-triggered impulsive coupling control (ETICC). The parameters of the multiple DTMJMNNs and the mixed time delays (both discrete and distributed delays) switch randomly according to a Markov chain. In the ETICC strategy, the controller does not work all the time, but only works at impulse instants determined by specific events. In particular, the coupling matrix can be non-Laplacian. By using the Lyapunov stability theory, linear matrix inequalities (LMIs), and the Kronecker product, some sufficient conditions for global synchronization of multiple DTMJMNNs under the event-triggered strategy are derived. Two examples are presented to test the validity of the theoretical analysis results. [ABSTRACT FROM AUTHOR]

This paper proposes a battery management system, including a fast battery charger, battery aging diagnosis, and charge estimation and balancing. The charger adopts a single-inductor single-input dual-output architecture to achieve charge balancing among battery cells. Interleaved pulse charging is proposed to reduce the charging time and slow down the aging process of batteries as well. This method also significantly suppresses the variations of the temperature of battery cells and is beneficial to the implementation of charge balancing. An artificial neural network is proposed to detect the state of health (SOH) of battery cells and improve the accuracy of the state of charge (SOC) estimation. The prototype is implemented in TSMC 0.35- $\mu \text{m}$ process and TensorFlow tools are used. Measurement results show that the interleaved pulse charging reduces 30% variation of the battery temperature and saves 24% charging time when charging four battery cells concurrently. A mean absolute error of SOC estimation of 0.35% is achieved in this work. [ABSTRACT FROM AUTHOR]

Purpose: Customer churn prediction is one of the most well-known approaches to manage and improve customer retention. Machine learning techniques, especially classification algorithms, are very popular tools to predict the churners. In this paper, three ensemble classifiers are proposed based on bagging and boosting for customer churn prediction. Design/methodology/approach: In this paper, three ensemble classifiers are proposed based on bagging and boosting for customer churn prediction. The first classifier, which is called boosted bagging, uses boosting for each bagging sample. In this approach, before concluding the final results in a bagging algorithm, the authors try to improve the prediction by applying a boosting algorithm for each bootstrap sample. The second proposed ensemble classifier, which is called bagged bagging, combines bagging with itself. In the other words, the authors apply bagging for each sample of bagging algorithm. Finally, the third approach uses bagging of neural network with learning based on a genetic algorithm. Findings: To examine the performance of all proposed ensemble classifiers, they are applied to two datasets. Numerical simulations illustrate that the proposed hybrid approaches outperform the simple bagging and boosting algorithms as well as base classifiers. Especially, bagged bagging provides high accuracy and precision results. Originality/value: In this paper, three novel ensemble classifiers are proposed based on bagging and boosting for customer churn prediction. Not only the proposed approaches can be applied for customer churn prediction but also can be used for any other binary classification algorithms. [ABSTRACT FROM AUTHOR]

In this paper, the observer based neural adaptive finite-time containment control strategy for non-strict feedback nonlinear multi-agent systems is studied. The finite-time command filter is used to overcome the explosion of complexity problem and the established fractional power based error compensation signal is applied to compensate the filtering error caused by the filter. The distributed finite-time command filtered backstepping control method combines with the neural adaptive control technology and state observer is given, which ensures the containment control errors reach to the desired neighborhood of the origin in finite-time in the presence of uncertain dynamics and unmeasurable states in the system. The given numerical simulations show the effectiveness of the proposed control strategy. [ABSTRACT FROM AUTHOR]

This paper investigates the secure consensus tracking problem for continuous-time nonlinear multiagent systems with sensor attacks. By designing a secure data selector, the unattacked output data is extracted from a group of output measurements under sparse sensor attacks. Then, by virtue of the obtained data and neural networks, a state observer is constructed to estimate the unavailable system states, where the convex combination theory is introduced to reduce the difficulty of solving observation gains. To utilize the limited communication resources reasonably, a novel saturated threshold event-triggered control strategy is proposed to reduce control updates, and then each update is encoded into a binary signal (0 or 1) to further reduce the occupation of communication bandwidth. The designed control scheme ensures that all closed-loop signals are semi-globally uniformly ultimately bounded, and its effectiveness is verified via a simulation of attitude control of unmanned aerial vehicles. [ABSTRACT FROM AUTHOR]

The successful application of modern deep neural network models is deeply influenced by the choice of hyperparameters. As one of the most important hyperparameters, the Learning Rate (LR) needs to be fine-tuned. In order to train models that can perform better, finding a suitable LR schedule quickly has become a pressing problem. Current research often relies on manual use of parallel search methods such as grid search, or serial methods such as Bayesian. However, the variety and number of models and datasets result in a very large LR space, and these methods consume significant resources to find only sub-optimal static LR schedules within a given time frame. It was found that in the currently widely used Distributed Data Parallel (DDP) deep learning, where multiple nodes themselves constitute the population, it is possible to use population algorithms to optimise the searched LR schedule dynamically throughout the training process with little additional training time. To maximise the potential of both the searched LR schedule and the population optimisation of the participating multi-nodes, this paper proposes a Leader Population Learning Rate Schedule (LPLRS) for the DDP deep learning environment, which continuously explores for better learning rates in the neighbourhood of the searched LR schedule and guides the subsequent training process. LPLRS achieves higher model classification accuracy on the Cifar 10&100 test datasets on the state-of-the-art Wide Residual Network with Sharpness-Aware Minimization (WRN(SAM)) model compared to the latest SGDR, CLR, and StepLR learning rate schedules. [ABSTRACT FROM AUTHOR]

• A 14-layer model with high predictive performance is proposed. • The model fuses three types of multi-source and multi-frequency features. • The model addresses three limitations of LSTM. • The model is validated through four comparison experiments on four stock indices. • Each component of the model improves the predictive performance. The most widely used model in stock price forecasting is the long short-term memory network (LSTM). However, LSTM has its limitations, as it does not recognize and extract features well and has a representational bottleneck. Furthermore, the factors affecting stock prices are multi-source and multi-frequency information, making neural network models difficult to handle. In this paper, we introduce a feature fusion residual LSTM (FFRL) model to answer these two questions – how to compensate for the three limitations of LSTM and how to fuse the multi-source and multi-frequency information. FFRL consists of three modules to improve the three limitations of LSTM, namely the feature selection module, feature extraction module, and residual module. To learn features from multi-source and multi-frequency information, FFRL applies the feature selection module to emphasize important features and the feature extraction module to extract deeper features. We demonstrate significant performance improvements of FFRL over comparison models, ablation networks, and visualization methods on a variety of Chinese stocks. [ABSTRACT FROM AUTHOR]

It has been a long-standing goal in natural language processing (NLP) to learn a general linguistic intelligence model that can perform well on many different NLP tasks continually evolving over time while avoiding revisiting all previous data at each stage. Most existing deep neural networks suffer from catastrophic forgetting when dealing with sequential tasks in an incremental way, leading to dramatic performance degradation due to the missing training data of old tasks. In this paper, we propose a Lifelong language method with Adaptive Uncertainty Regularization (LAUR), which can adapt a single BERT model to work with continuously arriving text examples from different NLP tasks. Specifically, LAUR is built on the Bayesian online learning framework, and three uncertainty regularization terms are devised to collaboratively control the parameters so as to resolve the stability-plasticity dilemma in lifelong language learning. The previous posterior constrain parameters that strongly determine the output results, preventing these parameters from changing drastically, while other parameters are encouraged to be updated over time. In addition, we propose a task-specific residual adaptation module in parallel to each layer of BERT to endow LAUR with the capacity to learn better task-specific knowledge. This configuration makes LAUR less prone to losing the knowledge stored in the base BERT network when learning a new task. Experimental results show that LAUR outperforms state-of-the-art lifelong learning models on a variety of NLP tasks. For reproducibility, we submit the code and data at: https://github.com/kiujhytgtrfd2021/LAUR. [ABSTRACT FROM AUTHOR]

It has been a long-standing goal in natural language processing (NLP) to learn a general linguistic intelligence model that can perform well on many different NLP tasks continually evolving over time while avoiding revisiting all previous data at each stage. Most existing deep neural networks suffer from catastrophic forgetting when dealing with sequential tasks in an incremental way, leading to dramatic performance degradation due to the missing training data of old tasks. In this paper, we propose a Lifelong language method with Adaptive Uncertainty Regularization (LAUR), which can adapt a single BERT model to work with continuously arriving text examples from different NLP tasks. Specifically, LAUR is built on the Bayesian online learning framework, and three uncertainty regularization terms are devised to collaboratively control the parameters so as to resolve the stability-plasticity dilemma in lifelong language learning. The previous posterior constrain parameters that strongly determine the output results, preventing these parameters from changing drastically, while other parameters are encouraged to be updated over time. In addition, we propose a task-specific residual adaptation module in parallel to each layer of BERT to endow LAUR with the capacity to learn better task-specific knowledge. This configuration makes LAUR less prone to losing the knowledge stored in the base BERT network when learning a new task. Experimental results show that LAUR outperforms state-of-the-art lifelong learning models on a variety of NLP tasks. For reproducibility, we submit the code and data at: https://github.com/kiujhytgtrfd2021/LAUR. [ABSTRACT FROM AUTHOR]

• A 14-layer model with high predictive performance is proposed. • The model fuses three types of multi-source and multi-frequency features. • The model addresses three limitations of LSTM. • The model is validated through four comparison experiments on four stock indices. • Each component of the model improves the predictive performance. The most widely used model in stock price forecasting is the long short-term memory network (LSTM). However, LSTM has its limitations, as it does not recognize and extract features well and has a representational bottleneck. Furthermore, the factors affecting stock prices are multi-source and multi-frequency information, making neural network models difficult to handle. In this paper, we introduce a feature fusion residual LSTM (FFRL) model to answer these two questions – how to compensate for the three limitations of LSTM and how to fuse the multi-source and multi-frequency information. FFRL consists of three modules to improve the three limitations of LSTM, namely the feature selection module, feature extraction module, and residual module. To learn features from multi-source and multi-frequency information, FFRL applies the feature selection module to emphasize important features and the feature extraction module to extract deeper features. We demonstrate significant performance improvements of FFRL over comparison models, ablation networks, and visualization methods on a variety of Chinese stocks. [ABSTRACT FROM AUTHOR]

Existing graph neural networks (GNNs) associate nodes in networks with specific samples in datasets, and thus ignore the conceptual information hidden in object-attribute clusters in datasets. Besides, processing data without structural information is a problem for GNNs since structural information is the input of networks. In this paper, we aim to integrate conceptual information into the message passing of GNNs. To this end, concept lattice theory is fused into existing GNNs. A concept lattice is a powerful tool for describing generalization and specialization relations between formal concepts. And formal concepts, basic elements of concept lattices, effectively explain dependencies between features and samples. On this basis, we propose a new GNN framework induced by a concept lattice to overcome the intrinsic limitations of GNNs. And the novel GNN framework not only joins conceptual information into the message passing but also enables a GNN to process data with or without structural information. Furthermore, the proposed framework is validated under transductive and inductive learning conditions, respectively. The experimental results show that GNNs induced by concept lattices can handle the information hidden in datasets effectively and improve classification accuracies on most benchmark datasets over previous methods. [ABSTRACT FROM AUTHOR]