Search

The purpose of this study is to investigate whether there are differences in handwritten Chinese signatures on different media including paper and electronic devices. Participants were asked to sign specified names on various types of media and the signatures were scanned or saved digitally for subsequent analysis. In this study, using convolutional neural networks and Siamese neural networks as classifiers and comparators, the performance plunge is revealed and thus considerable dissimilarity between the signatures on different media is implied. To further explore this, cubic Bézier curves are fitted to the signatures using the least square method for quantitative statistical analysis. By analyzing the visual changes in the morphology of strokes, several features of signatures are selected and computed, and the paired t‐test and the Wilcoxon signed‐rank test are implemented, which provides a deeper substantiation and explanation of the findings. [ABSTRACT FROM AUTHOR]

Forecasting has extreme importance in industry due to the numerous competitive advantages that it provides, allowing to foresee what might happen and adjust management decisions accordingly. Industries increasingly use sensors, which allow for large-scale data collection. Big datasets enable training, testing and application of complex predictive algorithms based on machine learning models. The present paper focuses on predicting values from sensors installed on a pulp paper press, using data collected over three years. The variables analyzed are electric current, pressure, temperature, torque, oil level and velocity. The results of XGBoost and artificial neural networks, with different feature vectors, are compared. They show that it is possible to predict sensor data in the long term and thus predict the asset's behaviour several days in advance. [ABSTRACT FROM AUTHOR]

Copyright of Informatica (03505596) is the property of Slovene Society Informatika and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Fibres & Textiles in Eastern Europe is the property of Sciendo and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Machine learning (ML), a form of artificial intelligence (AI), is being increasingly employed in neurology. Reported performance metrics often match or exceed the efficiency of average clinicians. The neurologist is easily baffled by the underlying concepts and terminologies associated with ML studies. The superlative performance metrics of ML algorithms often hide the opaque nature of its inner workings. Questions regarding ML model's interpretability and reproducibility of its results in real-world scenarios, need emphasis. Given an abundance of time and information, the expert clinician should be able to deliver comparable predictions to ML models, a useful benchmark while evaluating its performance. Predictive performance metrics of ML models should not be confused with causal inference between its input and output. ML and clinical gestalt should compete in a randomized controlled trial before they can complement each other for screening, triaging, providing second opinions and modifying treatment. [ABSTRACT FROM AUTHOR]

Introduction: Many studies show that words learned early in life are read more easily than the ones learned later and are less vulnerable to brain damage. Methods: the first part of the current study, 25 primary school students in the 5th grade read the word groups learned initially during a previous grade. The words used in the experiments were 327 Farsi monosyllable words matched on the other factors involved in Farsi word naming. Results: The analysis of covariance (the consistency and frequency as covariates) showed that words learned in earlier grades were read more easily than the ones learned later, showing the known effect of the Age of Acquisition (AoA). In the second part of the study, it was tried to simulate AoA in word naming by a neural network model developed earlier based on connectionist approach. While previous studies used random patterns, in the current study words from primary school books were used. Likewise, words learned early by the model were read better than words learned later. However, there was a failure in replicating previous simulation of AoA in English reading by an algorithm called Quick prop for Farsi. In addition, the model was lesioned by removing some hidden units to see its effect on word reading. As a result, words learned earlier were less vulnerable to damage compared with the ones learned later. Conclusion: The findings showed that words learned earlier, compared to those learned later, were read better and were less vulnerable to damage. These effects are explained by considering the nature of learning in neural networks trained by error back-propagation. [ABSTRACT FROM AUTHOR]

This paper presents a study of the impact of two important parameters, moisture and aging of the oil/paper dielectric used as insulation in power transformers.The way in which these two parameters influence different parameters of the Frequency Domain Spectroscopy (FDS) measurements, is emphasized.Different FDS parameters were measured by varying the moisturecontent and the aging degree of the oil impregnated paper.The use of two types of neural networks for analysis of the results was necessary in order to help discriminating the impact of moisture and aging on the FDS measurements and, in some cases, to estimate the aging duration of the paper impregnated with oil. [ABSTRACT FROM PUBLISHER]

Aiming at the imbalance of industrial control system data and the poor detection effect of industrial control intrusion detection systems on network attack traffic problems, we propose an ETM-TBD model based on hybrid machine learning and neural network models. Aiming at the problem of high dimensionality and imbalance in the amount of sample data in the massive data of industrial control systems, this paper proposes an IG-based feature selection method and an oversampling method for SMOTE. In the ETM-TBD model, we propose a hyperparameter optimization method based on Bayesian optimization used to optimize the parameters of the four basic machine learners in the model. By introducing a multi-head-attention mechanism, the Transformer module increases the attention between local features and global features, enabling the discovery of the internal relationship between features. Additionally, the BiGRU is used to preserve the temporal features of the dataset, while the DNN is used to extract deeper features. Finally, the SoftMax classifier is used to classify the output. By analyzing the results of the comparison and ablation experiments, it can be concluded that the F1-score of the ETM-TBD model on a robotic arm dataset is 0.9665 and the model has very low FNR and FPR scores of 0.0263 and 0.0081, respectively. It can be seen that the model in this paper is better than the traditional single machine learning algorithm as well as the algorithm lacking any of the modules. [ABSTRACT FROM AUTHOR]

De-mining operations are of critical importance for humanitarian efforts and safety in conflict-affected regions. In this paper, we address the challenge of enhancing the accuracy and efficiency of mine detection systems. We present an innovative Deep Learning architecture tailored for pulse induction-based Metallic Mine Detectors (MMD), so called DL-MMD. Our methodology leverages deep neural networks to distinguish amongst nine distinct materials with an exceptional validation accuracy of 93.5%. This high level of precision enables us not only to differentiate between anti-personnel mines, without metal plates but also to detect minuscule 0.2-g vertical paper pins in both mineralized soil and non-mineralized environments. Moreover, through comparative analysis, we demonstrate a substantial 3% and 7% improvement (approx.) in accuracy performance compared to the traditional K-Nearest Neighbors and Support Vector Machine classifiers, respectively. The fusion of deep neural networks with the pulse induction-based MMD not only presents a cost-effective solution but also significantly expedites decision-making processes in de-mining operations, ultimately contributing to improved safety and effectiveness in these critical endeavors. [ABSTRACT FROM AUTHOR]

Industrial archived process data represent a convenient source of information for data-driven models, such as artificial neural network (ANN), that can be used for safety and efficiency improvement like early or even predictive fault detection and diagnosis (FDD). Nonetheless, most of the data used for model generation are representative of the process nominal states and therefore are not enough for classification problems intended to determine abnormal process conditions. This work proposes the use of techniques to augment the original real data standards, dismissing the need for experiments that could jeopardize process safety. It uses the Monte Carlo technique to artificially increase the number of model inputs coupled to the nearest neighbor search (NNS) by geometric distances to consistently classify the generated patterns in normal or faulty statuses. Finally, a radial basis function neural network is trained with the augmented data. The methodology was validated by a study case in which 3381 pulp and paper industrial data points were expanded to monitor the formation of particles in a recovery boiler. Only 5.8% of the original process data were examples of faulty conditions, but the new expanded and balanced data collection leveraged the classification performance of the neural network, allowing its future use for monitoring purpose. [ABSTRACT FROM AUTHOR]

The first issue of Artificial Intelligence and Law journal was published in 1992. This paper provides commentaries on landmark papers from the first decade of that journal. The topics discussed include reasoning with cases, argumentation, normative reasoning, dialogue, representing legal knowledge and neural networks. [ABSTRACT FROM AUTHOR]

This paper presents the latest advances in machine learning techniques and highlights deep learning (DL) methods in recent studies. This technology has recently received great attention as it can solve complex problems. This paper focuses on covering one of the deep learning algorithms (deep neural network) and learning about its types such as convolutional neural network (CNN), Recurrent Neural Networks (RNN), etc. We have discussed recent changes, such as advanced DL technologies. Next, we continue analyzing and discussing the challenges and possible solutions of machine learning, such as big data and object detection, studying more papers in deep learning, and knowing the main experiments and future directions. In addition, this review also identifies some successful deep learning applications in recent years. Moreover, in this paper, one of the deep learning methods, convolutional neural networks, is applied to detect objects in images by using the You Only Look One model and comparing it with RetinaNet using pre-trained models. The results found that CNN (using YOLOv3) is a more accurate model than RetinaNet. [ABSTRACT FROM AUTHOR]

With the advent of the big data era, edge cloud computing has developed rapidly. In this era of popular digital music, various technologies have brought great convenience to online music education. But vast databases of digital music prevent educators from making specific-purpose choices. Music recommendation will be a potential development direction for online music education. In this paper, we propose a deep learning model based on multi-source information fusion for music recommendation under the scenario of edge-cloud computing. First, we use the music latent factor vector obtained by the Weighted Matrix Factorization (WMF) algorithm as the ground truth. Second, we build a neural network model to fuse multiple sources of music information, including music spectrum extracted from extra music information to predict the latent spatial features of music. Finally, we predict the user's preference for music through the inner product of the user vector and the music vector for recommendation. Experimental results on public datasets and real music data collected by edge devices demonstrate the effectiveness of the proposed method in music recommendation. [ABSTRACT FROM AUTHOR]

Neural network models are often used to analyse non-linear systems; here, in cases of small datasets, we review our complementary approach to deep learning with the purpose of highlighting the importance and roles (linear, non-linear or threshold) of certain variables (assumed as causal) in determining the behaviour of a target variable; this also allows us to make predictions for future scenarios of these causal variables. We present a neural tool endowed with an ensemble strategy and its applications to influence analyses in terms of pruning, attribution and future predictions (free code issued). We describe some case studies on climatic applications which show reliable results and the potentialities of our method for medical studies. The discovery of the importance and role (linear, non-linear or threshold) of causal variables and the possibility of applying the relationships found to future scenarios could lead to very interesting applications in medical research and the study and treatment of cancer, which are proposed in this paper. [ABSTRACT FROM AUTHOR]

This paper serves as a survey of recent advances in large margin training and its theoretical foundations, mostly for (nonlinear) deep neural networks (DNNs) that are probably the most prominent machine learning models for large-scale data in the community over the past decade. We generalize the formulation of classification margins from classical research to latest DNNs, summarize theoretical connections between the margin, network generalization, and robustness, and introduce recent efforts in enlarging the margins for DNNs comprehensively. Since the viewpoint of different methods is discrepant, we categorize them into groups for ease of comparison and discussion in the paper. Hopefully, our discussions and overview inspire new research work in the community that aim to improve the performance of DNNs, and we also point to directions where the large margin principle can be verified to provide theoretical evidence why certain regularizations for DNNs function well in practice. We managed to shorten the paper such that the crucial spirit of large margin learning and related methods are better emphasized. [ABSTRACT FROM AUTHOR]

This paper explores the application of multi-objective optimization techniques, including MOPSO, NSGA II, and SPEA2, to optimize the hyperparameters of artificial neural networks (ANNs) and support vector machines (SVMs) for predicting the physical properties of textile polymer composite materials (TPCMs). The optimization process utilizes data on the physical characteristics of the constituent fibers and fabrics used to manufacture these composites. By employing optimization algorithms, we aim to enhance the predictive accuracy of the ANN and SVM models, thereby facilitating the design and development of high-performance textile polymer composites. The effectiveness of the proposed approach is demonstrated through comparative analyses and validation experiments, highlighting its potential for optimizing complex material systems. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach to sentiment analysis specifically customized for predicting stock market movements, bypassing the need for external dictionaries that are often unavailable for many languages. Our methodology directly analyzes textual data, with a particular focus on context-specific sentiment words within neural network models. This specificity ensures that our sentiment analysis is both relevant and accurate in identifying trends in the stock market. We employ sophisticated mathematical modeling techniques to enhance both the precision and interpretability of our models. Through meticulous data handling and advanced machine learning methods, we leverage large datasets from Twitter and financial markets to examine the impact of social media sentiment on financial trends. We achieved an accuracy exceeding 75%, highlighting the effectiveness of our modeling approach, which we further refined into a convolutional neural network model. This achievement contributes valuable insights into sentiment analysis within the financial domain, thereby improving the overall clarity of forecasting in this field. [ABSTRACT FROM AUTHOR]

This paper presents the results of a study on data preprocessing and modeling for predicting corrosion in water pipelines of a steel industrial plant. The use case is a cooling circuit consisting of both direct and indirect cooling. In the direct cooling circuit, water comes into direct contact with the product, whereas in the indirect one, it does not. In this study, advanced machine learning techniques, such as extreme gradient boosting and deep neural networks, have been employed for two distinct applications. Firstly, a virtual sensor was created to estimate the corrosion rate based on influencing process variables, such as pH and temperature. Secondly, a predictive tool was designed to foresee the future evolution of the corrosion rate, considering past values of both influencing variables and the corrosion rate. The results show that the most suitable algorithm for the virtual sensor approach is the dense neural network, with MAPE values of (25 ± 4)% and (11 ± 4)% for the direct and indirect circuits, respectively. In contrast, different results are obtained for the two circuits when following the predictive tool approach. For the primary circuit, the convolutional neural network yields the best results, with MAPE = 4% on the testing set, whereas for the secondary circuit, the LSTM recurrent network shows the highest prediction accuracy, with MAPE = 9%. In general, models employing temporal windows have emerged as more suitable for corrosion prediction, with model performance significantly improving with a larger dataset. [ABSTRACT FROM AUTHOR]

The 6G wireless communication system will utilize the terahertz (THz) frequency band (0.1-10 THz) to meet customer demand for increased data rates and ultra-high-speed communication in future applications. The exponential surge in data traffic, which is supported by dynamic resource allocation. To mitigate this challenge, the use of artificial intelligence-based methods, such as narrow neural network (NNN), can help to smooth the performance of the network. In this paper, an NNN-based approach for dynamic base station allocation for 6G wireless networks is proposed 14 different 6G parameters used to train the NNN model, initially achieving an accuracy of 89.5% and an F1 score of 0.72 for 200 users. Results demonstrate the efficacy of the proposed NNN approach for dynamic decision-making in 6G networks and its potential for application in other domains where similar problems exist. Moreover, the proposed narrow neural network model shows improved results with an increase in number of users and decrease in fully connected layers and regularization strength (lambda). The validation accuracy received is 98.9% and 99.6% for thousand users with single fully connected layer, none (linear) activation function and regularization strength lambda values of 0.01 and 0.001. [ABSTRACT FROM AUTHOR]

Demand response management (DRM) plays a crucial role in the prospective development of smart grids. The precise estimation of electricity demand for individual houses is vital for optimizing the operation and planning of the power system. Accurate forecasting of the required components holds significance as it can substantially impact the final cost, mitigate risks, and support informed decision-making. In this paper, a forecasting approach employing neural networks for smart grid demand-side management is proposed. The study explores various enhanced artificial neural network (ANN) architectures for forecasting smart grid consumption. The performance of the ANN approach in predicting energy demands is evaluated through a comparison with three statistical models: a time series model, an auto-regressive model, and a hybrid model. Experimental results demonstrate the ability of the proposed neural network framework to deliver accurate and reliable energy demand forecasts. [ABSTRACT FROM AUTHOR]

The planning, regulation and effectiveness of the product design process depend on various characteristics. Recently, bio-inspired collective intelligence approaches have been applied in this process in order to create more appealing product forms and optimize the design process. In fact, the use of neural network models in product form design analysis is a complex process, in which the type of network has to be determined, as well as the structure of the network layers and the neurons in them; the connection coefficients, inputs and outputs have to be explored; and the data have to be collected. In this paper, an impulsive discrete fractional neural network modeling approach is introduced for product design analysis. The proposed model extends and complements several existing integer-order neural network models to the generalized impulsive discrete fractional-order setting, which is a more flexible mechanism to study product form design. Since control and stability methods are fundamental in the construction and practical significance of a neural network model, appropriate impulsive controllers are designed, and practical Mittag-Leffler stability criteria are proposed. The Lyapunov function strategy is applied in providing the stability criteria and their efficiency is demonstrated via examples and a discussion. The established examples also illustrate the role of impulsive controllers in stabilizing the behavior of the neuronal states. The proposed modeling approach and the stability results are applicable to numerous industrial design tasks in which multi-agent systems are implemented. [ABSTRACT FROM AUTHOR]

Nowadays, trajectory control is a significant issue for unmanned micro aerial vehicles (MAVs) due to large disturbances such as wind and storms. Trajectory control is typically implemented using a proportional–integral–derivative (PID) controller. In order to achieve high accuracy in trajectory tracking, it is essential to set the PID gain parameters to optimum values. For this reason, separate gain values are set for roll, pitch and yaw movements before autonomous flight in quadrotor systems. Traditionally, this adjustment is performed manually or automatically in autotune mode. Given the constraints of narrow orchard corridors, the use of manual or autotune mode is neither practical nor effective, as the quadrotor system has to fly in narrow apple orchard corridors covered with hail nets. These reasons require the development of an innovative solution specific to quadrotor vehicles designed for constrained areas such as apple orchards. This paper recognizes the need for effective trajectory control in quadrotors and proposes a novel neural network-based approach to tuning the optimal PID control parameters. This new approach not only improves trajectory control efficiency but also addresses the unique challenges posed by environments with constrained locational characteristics. Flight simulations using the proposed neural network models have demonstrated successful trajectory tracking performance and highlighted the superiority of the feed-forward back propagation network (FFBPN), especially in latitude tracking within 7.52745 × 10−5 RMSE trajectory error. Simulation results support the high performance of the proposed approach for the development of automatic flight capabilities in challenging environments. [ABSTRACT FROM AUTHOR]

Simple Summary: Due to various causes, such as the thinning of the ozone layer, climate change, or the fashion for artificial tanning, the incidence of skin cancer has increased recently. Early detection of cancerous skin lesions is the only chance to prevent dangerous developments. This paper proposes two models of support systems for the detection of some skin lesions including dangerous melanoma, based on two different methods of assembling neural networks and making global decisions by fusing individual decisions. The system based on the fusion of some intermediate binary classification subsystems is a new solution and offers an accuracy of 91.04%, better than that offered by the classic system based on the vote with multiple weights of the constituent networks. In addition, it is found that the individual performance indicators depend on the type of skin lesion. For example, the F1 score varied from 81.36% to 94.17%. Today, skin cancer, and especially melanoma, is an increasing and dangerous health disease. The high mortality rate of some types of skin cancers needs to be detected in the early stages and treated urgently. The use of neural network ensembles for the detection of objects of interest in images has gained more and more interest due to the increased performance of the results. In this sense, this paper proposes two ensembles of neural networks, based on the fusion of the decisions of the component neural networks for the detection of four skin lesions (basal cancer cell, melanoma, benign keratosis, and melanocytic nevi). The first system is based on separate learning of three neural networks (MobileNet V2, DenseNet 169, and EfficientNet B2), with multiple weights for the four classes of lesions and weighted overall prediction. The second system is made up of six binary models (one for each pair of classes) for each network; the fusion and prediction are conducted by weighted summation per class and per model. In total, 18 such binary models will be considered. The 91.04% global accuracy of this set of binary models is superior to the first system (89.62%). Separately, only for the binary classifications within the system was the individual accuracy better. The individual F1 score for each class and the global system varied from 81.36% to 94.17%. Finally, a critical comparison is made with similar works from the literature. [ABSTRACT FROM AUTHOR]

Synchronization is a key topic of research in neuroscience, medicine, and artificial neural networks; however, understanding its principle is difficult, both scientifically and mathematically. Specifically, the synchronization of the FitzHugh-Nagumo network with a hierarchical architecture has previously been studied; however, a mathematical analysis has not been conducted, owing to the network complexity. Therefore, in this paper, we saught to understand synchronization through mathematical analyses. In particular, we consider the most common types of hierarchical architecture and present a condition of the hierarchical architecture to induce synchronization. First, we provide mathematical analyses of a Lyapunov function for each layer, from which we obtain sufficient conditions guaranteeing synchronization and show that the Lyapunov function decreases exponentially. Moreover, we show that the internal connectivity critically affects synchronization in the first layer; however, in the second and subsequent layers, the internal connectivity is not important for synchronization, and the connectivity up to the first layer critically affects synchronization. We expect that the results and mathematical methodology can be applied to study other similar neural models with hierarchical architectures. [ABSTRACT FROM AUTHOR]

The accurate identification of a driver's braking intention is crucial to the formulation of regenerative braking control strategies for electric vehicles. In this paper, a braking intention recognition model based on the sample entropy of the braking signal and a probabilistic neural network (PNN) is proposed to achieve the accurate recognition of different braking intentions. Firstly, the brake pedal travel signal is decomposed to extract the effective components via variational modal decomposition (VMD); then, the features of the decomposed signal are extracted using sample entropy to obtain the multidimensional feature vector of the braking signal; finally, the sparrow search algorithm (SSA) and probabilistic neural network are combined to optimize the smoothing factor with the sparrow search algorithm and the cross-entropy loss function as the fitness function to establish a braking intention recognition model. The experimental validation results show that combining the sample entropy features of the braking signal with the probabilistic neural network can effectively identify the braking intention, and the SSA-PNN algorithm has higher recognition accuracy compared with the traditional machine learning algorithm. [ABSTRACT FROM AUTHOR]

Safety concerns on the deep neural networks (DNNs) have been raised when they are applied to critical sectors. In this paper, we define safety risks by requesting the alignment of network's decision with human perception. To enable a general methodology for quantifying safety risks, we define a generic safety property and instantiate it to express various safety risks. For the quantification of risks, we take the maximum radius of safe norm balls, in which no safety risk exists. The computation of the maximum safe radius is reduced to the computation of their respective Lipschitz metrics—the quantities to be computed. In addition to the known adversarial example, reachability example, and invariant example, in this paper, we identify a new class of risk—uncertainty example—on which humans can tell easily, but the network is unsure. We develop an algorithm, inspired by derivative-free optimization techniques and accelerated by tensor-based parallelization on GPUs, to support an efficient computation of the metrics. We perform evaluations on several benchmark neural networks, including ACSC-Xu, MNIST, CIFAR-10, and ImageNet networks. The experiments show that our method can achieve competitive performance on safety quantification in terms of the tightness and the efficiency of computation. Importantly, as a generic approach, our method can work with a broad class of safety risks and without restrictions on the structure of neural networks. [ABSTRACT FROM AUTHOR]

Blade icing seriously affects wind turbines' aerodynamic performance and output power. Timely and accurately predicting blade icing status is crucial to improving the economy and safety of wind farms. However, existing blade icing prediction methods cannot effectively solve the problems of unbalanced icing/non-icing data and low prediction accuracy. In order to solve the above problems, this paper proposes a wind turbine blade icing prediction method based on the focal loss function and CNN-Attention-GRU. First, the recursive feature elimination method combined with the physical mechanism of icing is used to extract features highly correlated with blade icing, and a new feature subset is formed through a sliding window algorithm. Then, the focal loss function is utilized to assign more weight to the ice samples with a lower proportion, addressing the significant class imbalance between the ice and non-ice categories. Finally, based on the CNN-Attention-GRU algorithm, a blade icing prediction model is established using continuous 24-h historical data as the input and the icing status of the next 24 h as the output. The model is compared with advanced neural network models. The results show that the proposed method improves the prediction accuracy and F1 score by an average of 6.41% and 4.27%, respectively, demonstrating the accuracy and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning is a popular topic among machine learning researchers nowadays, with great strides being made in recent years to develop robust artificial neural networks for faster convergence to a reasonable accuracy. Network architecture and hyperparameters of the model are fundamental aspects of model convergence. One such important parameter is the initial values of weights, also known as weight initialization. In this paper, we perform two research tasks concerned with the weights of neural networks. First, we develop three novel weight initialization algorithms inspired by the neuroscientific construction of the mammalian brains and then test them on benchmark datasets against other algorithms to compare and assess their performance. We call these algorithms the lognormal weight initialization, modified lognormal weight initialization, and skewed weight initialization. We observe from our results that these initialization algorithms provide state-of-the-art results on all of the benchmark datasets. Second, we analyze the influence of training an artificial neural network on its weight distribution by measuring the correlation between the quantitative metrics of skewness and kurtosis against the model accuracy using linear regression for different weight initializations. Results indicate a positive correlation between network accuracy and skewness of the weight distribution but no affirmative relation between accuracy and kurtosis. This analysis provides further insight into understanding the inner mechanism of neural network training using the shape of weight distribution. Overall, the works in this paper are the first of their kind in incorporating neuroscientific knowledge into the domain of artificial neural network weights. [ABSTRACT FROM AUTHOR]

During the development of the nervous system, neuronal cells extend axons and dendrites that form complex neuronal networks, which are essential for transmitting and processing information. Understanding the physical processes that underlie the formation of neuronal networks is essential for gaining a deeper insight into higher-order brain functions such as sensory processing, learning, and memory. In the process of creating networks, axons travel towards other recipient neurons, directed by a combination of internal and external cues that include genetic instructions, biochemical signals, as well as external mechanical and geometrical stimuli. Although there have been significant recent advances, the basic principles governing axonal growth, collective dynamics, and the development of neuronal networks remain poorly understood. In this paper, we present a detailed analysis of nonlinear dynamics for axonal growth on surfaces with periodic geometrical patterns. We show that axonal growth on these surfaces is described by nonlinear Langevin equations with speed-dependent deterministic terms and gaussian stochastic noise. This theoretical model yields a comprehensive description of axonal growth at both intermediate and long time scales (tens of hours after cell plating), and predicts key dynamical parameters, such as speed and angular correlation functions, axonal mean squared lengths, and diffusion (cell motility) coefficients. We use this model to perform simulations of axonal trajectories on the growth surfaces, in turn demonstrating very good agreement between simulated growth and the experimental results. These results provide important insights into the current understanding of the dynamical behavior of neurons, the self-wiring of the nervous system, as well as for designing innovative biomimetic neural network models. [ABSTRACT FROM AUTHOR]

Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time. [ABSTRACT FROM AUTHOR]

The prevalence of world-wide access to the Internet has come at a cost. A lot of misleading information is posted on public news websites and social media. Many news writers and organizations manipulate their posted data to propagate false information that target different societies and in different languages. Accurate and timely detection of false news is made possible in large part using machine learning-based technologies. This paper targets the problem of detecting fake news in Arabic language using machine learning models. A hybrid model of two deep neural networks is used to classify Arabic news articles in order to detect fake articles. The two types of neural networks are convolutional and bi-directional long-short term memory. Robust features are extracted using two different word vectors and a complex model of a convolutional neural network. Moreover, a set of auxiliary output layers are used to enhance the model accuracy. Multi-class classification is achieved via modifying the primary output layer. Results show an accuracy of 88% and 78% for binary classification and multi-class classification, respectively. [ABSTRACT FROM AUTHOR]

The work presents an approach to enhance the forecasting capabilities of In-Situ Leaching processes during both the production stage and early prognosis. ISL, a crucial method for resource extraction, demands rapid on-site forecasting to guide the deployment of new technological blocks. Traditional modeling techniques, though effective, are hindered by their computational demands and network throughput requirements, particularly when dealing with substantial datasets or remote computing needs. The integration of AI technologies, specifically neural networks, offers a promising opportunity for expedited calculations by leveraging the power of forward propagation through pretrained neural models. However, a critical challenge lies in transforming conventional numerical datasets into a format suitable for neural modeling. Furthermore, the scarcity of training data during the production phase, where vital parameters are concealed underground, poses an additional challenge in training AI models for In-Situ Leaching processes. This research addresses these challenges by proposing a methodology for generating training data tailored to the most resource-intensive Computational Fluid Dynamics problems encountered during modeling. Traditional numerical modeling techniques are harnessed to construct training datasets comprising input and corresponding expected output data, with a particular focus on varying well network patterns. Subsequent efforts are directed at the conversion of the acquired data into a format compatible with neural networks. The data is normalized to align with the data ranges stipulated by the activation functions employed within the neural network architecture. This preprocessing step ensures that the neural model can effectively learn from the generated data, facilitating accurate forecasting of In-Situ Leaching processes. An advantage of proposed technique lies in provision of large, reliable datasets to train neural network to predict hydrodynamic properties based on technological regimes currently active or expected on ISL site. A major implication of this approach lies in applicability of pre-trained AI technologies to forecast future or determine current hydrodynamic regime in the stratum circumventing cost deterministic simulations currently deployed at mining sites. Hence, innovative approach outlined in this paper holds promise for optimizing forecasting, allowing for quicker and more efficient decision-making in resource extraction operations while getting around the computational barriers associated with traditional methods. [ABSTRACT FROM AUTHOR]

Safety is a critical component for human–robot cohabitation. The control barrier function (CBF) provides an effective tool to build up the constraint and ensure the safety of human–robot interaction. However, since the human and robot keep moving during human–robot interaction, the closest points between parts of them also change. Especially, the human trajectories are not known in prior, which may cause the above safety constraint to fail. In this paper, we construct the safe constraints based on discrete control barrier function (DCBF) by redefining the distance between each link of the robot and each part of the human body as the distance between two line segments in the space. In addition, the look-backward-and-forward strategy is applied to update the neural network model for predicting of human's motion trajectory effectively. Meanwhile, the root mean square estimation error is included in the safe constraints as the metric of uncertainty to compensate the estimation error of the predicted trajectory. Combining the discrete-time control Lyapunov function, a comprehensive control method under human–robot-coexistence environment is formed. The trajectory of a human's right arm collected by Qualisys capture system. The experiment are set up by integrating above testbed with a virtual KUKA iiwa model built by MATLAB. The results show that the robot can maintain a safe distance from the human when the DCBF-based constraints with prediction information are used, which verifies the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, the Levenberg-Marquardt backpropagation scheme is used to develop a neural network model for the examination of the fluid flow on a magnetized flat surface with slip boundaries. The tangent hyperbolic fluid is considered along with heat generation, velocity, and thermal slip effects at the surface. The problem is modelled in terms of a non-linear differential system and Lie symmetry is used to get the scaling group of transformation. The order reduction of differential equations is done by using Lie transformation. The reduced system is solved by the shooting method. The surface quantity, namely skin friction, is evaluated at the surface for the absence and presence of an externally applied magnetic field. A total of 88 sample values are estimated for developing an artificial neural network model to predict skin friction coefficient (SFC). Weissenberg number, magnetic field parameter, and power law index are considered three inputs in the first layer, while 10 neurons are taken in the hidden layer. 62 (70%), 13 (15%), and 13 (15%) samples are used for training, validation, and testing, respectively. The Levenberg-Marquardt backpropagation is used to train the network by entertaining the random 62 sample values. Both mean square error and regression analysis are used to check the performance of the developed neural networking model. The SFC is noticed to be high at a magnetized surface for power law index and Weissenberg number. [ABSTRACT FROM AUTHOR]

Transfer-based attacks, a type of adversarial attack, have become a growing threat in recent years with the proliferation of cloud services. Deep neural networks that exploit human cognitive bias (Loosely Symmetric-Deep Neural Network, LS-DNN) are known as a defensive technique against transfer-based attacks. LS-DNN can prevent malfunctions caused by adversarial examples with a high probability by incorporating human learning characteristics into the neural network's nodes. However, maintaining accuracy against normal data and reducing Training Time is challenging. This paper proposes a new model called "LS+-DNN" inspired by the Dropout method to solve this problem. Evaluation experiments on two datasets show that the proposed model can achieve both in high dimensions. In addition, we analyze the proposed model focusing on the variance and confidence score of the training parameters of the proposed model. As a result, we point out that the confidence score is an important indicator of robust models against transfer-based attacks. [ABSTRACT FROM AUTHOR]

Diabetes is a dangerous disease in which the body is incapable of controlling blood sugar due to inadequate insulin hormone levels. This chronic disease increases blood sugar in patients. Therefore, if it is not controlled, it will cause many complications. A considerable number of people in the world suffer from this disease owing to its damage and lack of its initial diagnosis. The patient visits the doctor frequently to diagnose his/her illness and conducts various tests that are boring and costly. Increasing machine learning approaches through heuristics, and novel methods can somewhat decrease the problems. The current study aims to propose a model that can predict diabetes in patients with high accuracy. The paper introduces a new method based on the assortment of metaheuristic algorithms of a particle swarm and fuzzy inference system. The proposed method utilizes fuzzy systems to binary the particle swarm algorithm. The achieved model is applied to the diabetes dataset and then evaluated using a neural network classifier. The results indicate an increase in classification accuracy to 95.47% compared to other existing methods. [ABSTRACT FROM AUTHOR]

Stochastic adaptive gradient decent algorithms, such as AdaGrad and Adam, are extensively used to train deep neural networks. However, randomly sampling gradient information introduces instability to the learning rates, leading to adaptive methods with poor generalization. To address this issue, the ABNGrad algorithm, which leverages the absolute value operation and the normalization technique, is proposed. More specifically, the absolute value function is first incorporated into the iteration of the second-order moment estimate to ensure that it monotonically increases. Then, the normalization technique is employed to prevent a rapid decrease in the learning rate. In particular, the techniques used in this paper can also be integrated into other existing adaptive algorithms, such as Adam, AdamW, AdaBound, and RAdam, yielding good performance. Additionally, it is shown that ABNGrad can attain the optimal regret bound for solving online convex optimization problems. Finally, many experimental results illustrate the effectiveness of ABNGrad. For a comprehensive exploration of the advantages of the proposed approach and the specifics of its detailed implementation, the readers are referred to the following https://github.com/Wenhan-Jiang/ABNGrad.git [ABSTRACT FROM AUTHOR]

Resistive random access memory (ReRAM) is often considered as a strong candidate for storing the weights in non-von Neumann neuromorphic computing systems. This paper studies how nonideal memory characteristics, which include programing error, read fluctuation, and retention, affect the inference accuracy of the analog ReRAM neural networks by incorporating memory characteristics extracted from 1-Mb ReRAM into a simulated inference-only neural network. This paper also shows that the different layer in the network can tolerate different amount of such imperfects. We learned four key points: 1) the conductance range of memory with less relative fluctuation is preferred for designing the weight-conductance mapping; 2) the control of programing error is essential for high inference accuracy; 3) retention-induced conductance drift can be fatal to the neuromorphic system. A compensation scheme is proposed in this paper which can effectively recover the inference accuracy; and 4) for multilayer networks, avoiding weight errors in the front layers can help to maintain the inference accuracy by reducing calculation error which may otherwise accumulate and pass down the networks. The concepts and approaches of this paper can also be applied to evaluate other types of nonvolatile memories for artificial neural networks. [ABSTRACT FROM AUTHOR]

In this paper we aim to provide an implementation of the FAIR Data Points (FDP) spec, that will apply our bias detection algorithm and automatically calculate a FAIRness score (FNS). FAIR metrics would be themselves represented as FDOs, and could be presented via a visual dashboard, and be machine accessible (Mons 2020, Wilkinson et al. 2016). This will enable dataset owners to monitor the level of FAIRness of their data. This is a step forward in making data FAIR, i.e., Findable, Accessible, Interoperable, and Reusable; or simply, Fully AI Ready data. First we may discuss the context of this topic with respect to Deep Learning (DL) problems. Why are Bayesian Networks (BN, explained below) beneficial for such issues? • Explainability - Obtaining a directed acyclic graph (DAG) from a BN training provides coherent information about independence variables in the data base. In a generic DL problem, features are functions of these variables. Thus, one can derive which variables are dominant in our system. When customers or business units are interested in the cause of a neural net outcome, this DAG structure can be both a source to provide importance and clarify the model. • Dimension Reduction -- BN provides the joint distribution of our variables and their associations. The latter may play a role in reducing the features that we induce to the DL engine: If we know that for random variables X,Y the conditional entropy of X in Y are low, we may omit X since Y provides its nearly entireinformation. We have, therefore, a tool that can statistically exclude redundant variables • Tagging Behavior - This section can be less evident for those who work in domains such as vision or voice. In some frameworks, labeling can be an obscure task (to illustrate, consider a sentiment problem with many categories that may overlap). When we tag the data, we may rely on some features within the datasets and generate conditional probability. Training BN, when we initialize an empty DAG, may provide outcomes in which the target is a parent of other nodes. Observing several tested examples, these outcomes reflect these "taggers' manners". We can therefore use DAGs not merely for the purpose of model development in machine learning but mainly learning taggers policy and improve it if needed. • The conjunction of DL and Casual inference -- Causal Inference is a highly developed domain in data analytics. It offers tools to resolve questions that on the one hand, DL models commonly do not and, on the other hand, the real-world raises. There is a need to find a framework in which these tools will work in conjunction. Indeed, such frameworks already exist (e.g., GNN). But a mechanism that merges typical DL problems causality is less common. We believe that the flow, as described in this paper, is a good step in the direction of achieving benefits from this conjunction. • Fairness and Bias - Bayesian networks, in their essence, are not a tool for bias detection but they reveal which of the columns (or which of the data items) is dominant and modify other variables. When we discuss noise and bias, we address these faults to the column and not to the model or to the entire data base. However, assume we have a set of tools to measure bias (Purian et al. 2022). Bayesian networks can provide information about the prominence of these columns (as they are "cause" or "effect" in the data), thus allow us to assess the overall bias in the database. What are Bayesian Networks? The motivation for using Bayesian Networks (BN) is to learn the dependencies within a set of random variables. The networks themselves are directed acyclic graphs (DAG), which mimic the joint distribution of the random variables (e.g., Perrier et al. (2008)). The graph structure follows the probabilistic dependencies factorization of the joint distribution: a node V depends only on its parents (a r.v X independent of the other nodes will be presented as a parent free node). Real-World Example: In this paper we present a way of using the DL engine tabular data, with the python package bnlearn. Since this project is commercial, the variable names were masked; thus, they will have meaningless names. Constructing Our DAG: We begin by finding our optimal DAG. Where rows are sources (namely the direction of the arc is from the left column to the elements in the row) and columns are targets (i.e., the header of the column receives the arcs). When we begin drawing the obtained DAG, we get for one set of variables the following image: Fig. 1b. We can see that the target node in the rectangle is a source for many nodes. We can see that it still points arrows itself to two nodes. We will discuss this in the discussion (i.e., Rauber 2021). We have more variables, therefore I increased the number of nodes. Adding the information provided a new source for the target (i.e., its entire row is "False"). The obtained graph is the following: Fig. 1c. So, we know how to construct a DAG. Now we need to train its parameters. Code-wise we perform this as follows: We can change 'maximulikelihood' with 'bayes' as described beyond. The outcome of this training is a set of factorized conditional distributions that reflect the DAG's structure. It has this form for a given variable: Fig. 1d. The code to create DAG presentation is provided in Fig. 2. Discussion: In this paper we have presented some of the theoretical concepts of Bayesian Networks and the usage they provide in constructing an approximated DAG for a set of variables. In addition, we presented a real-world example of end to end DAG learning: Constructing it using BN, training its parameters using maximum likelihood estimation (MLE) methods, and performing and inference. FAIR metrics, represented as FDOs, can also be visualised and monitored, taking care of data FAIRness. [ABSTRACT FROM AUTHOR]

This paper suggests an approach to process and model the data obtained in biomonitoring studies. The approach is validated on data obtained from biomonitoring studies performed in the Republic of Moldova in 2015. Using the preliminary data, the decomposition on the basis of the pollution spectrum for the most polluted and cleanest sites is first carried out. The deviations of model predictions from the actual measurements are considered. A correlation analysis is next performed to evidence the correlation of two geographical coordinates with chemical elements. Factor analysis and regression analysis are applied to highlight the nonlinear mechanisms specific to the obtained data. A multilayer neural network-based model is derived to describe the relationship of the pollution rank to the geographic coordinates. The predictive capabilities of the model are represented graphically. [ABSTRACT FROM AUTHOR]

Model pruning is one of the main methods of deep neural network model compression. However, the existing model pruning methods are inefficient, there is still a lot of redundancy in the network, and the pruning has a great impact on the accuracy. In addition, the traditional pruning process usually needs to fine-tune the network to restore accuracy, and fine-tuning has a limited effect on accuracy recovery, so it is difficult to achieve a high level of accuracy. In this paper, we propose a new neural network pruning framework: the channel sparsity is realized by introducing a scale factor, and the sparse network is pruned by setting a global threshold, which greatly improves the efficiency of pruning. After pruning, this paper proposes a rewind method to restore the accuracy, that is, save the weight after training, and then reload it on the network for training to restore the accuracy. In addition, we also study the best rewind point of the three networks. The experimental results show that our method significantly reduces the number of parameters and FLOPs without affecting or even improving the accuracy, and the rewind method proposed by us achieves a better accuracy recovery effect than fine-tuning. At the same time, we find that the epoch with the highest accuracy is the best rewind point, and the accuracy is the highest after saving its corresponding weight and retraining the model. [ABSTRACT FROM AUTHOR]

Computer-assisted surgery (CAS) has occupied an important position in modern surgery, further stimulating the progress of methodology and technology. In recent years, a large number of computer vision-based methods have been widely used in surgical workflow recognition tasks. For training the models, a lot of annotated data are necessary. However, the annotation of surgical data requires expert knowledge and thus becomes difficult and time-consuming. In this paper, we focus on the problem of data deficiency and propose a knowledge transfer learning method based on artificial neural network to compensate a small amount of labeled training data. To solve this problem, we propose an unsupervised method for pre-training a Convolutional-De-Convolutional (CDC) neural network for sequencing surgical workflow frames, which performs neural convolution in space (for semantic abstraction) and neural de-convolution in time (for frame level resolution) simultaneously. Specifically, through neural convolution transfer learning, we only fine-tuned the CDC neural network to classify the surgical phase. We performed some experiments for validating the model, and it showed that the proposed model can effectively extract the surgical feature and determine the surgical phase. The accuracy (Acc), recall, precision (Pres) of our model reached 91.4, 78.9, and 82.5%, respectively. [ABSTRACT FROM AUTHOR]

Real-time and accurate state-of-charge estimation performs an important role in the smooth operation of various electric vehicle battery management systems. Neural network theory represents one of the most effective and commonly used methods of SOC prediction. However, traditional neural network methods are disadvantaged by such issues as the limited range of application, limited generalization ability, and low accuracy, which makes it difficult to meet the increasing safety requirements on electric vehicles. In view of these problems, an ensemble learning algorithm based on the AdaBoost.Rt is proposed in this paper. AdaBoost.Rt recurrent neural network model is purposed to ensure the accurate prediction of lithium battery SOC. Relying on a chain-connected recurrent neural network model, this method enables the correlation adaptability of sample data in the spatio-temporal dimension. The ensemble learning method was adopted to devise a method of multi-RNN model integration, with the RNN model as the base learner, thus constructing the AdaBoost.Rt-RNN strong learner model. According to the results of simulation and experimental comparisons, the integrated algorithm proposed in this paper is applicable to improve the accuracy of SOC prediction and the generalization performance of the model. [ABSTRACT FROM AUTHOR]

A fuel cell is an energy conversion device that utilizes hydrogen energy through an electrochemical reaction. Despite their many advantages, such as high efficiency, zero emissions, and fast startup, fuel cells have not yet been fully commercialized due to deficiencies in service life, cost, and performance. Efficient evaluation methods for performance and service life are critical for the design and optimization of fuel cells. The purpose of this paper was to review the application of common machine learning algorithms in fuel cells. The significance and status of machine learning applications in fuel cells are briefly described. Common machine learning algorithms, such as artificial neural networks, support vector machines, and random forests are introduced, and their applications in fuel cell performance prediction and optimization are comprehensively elaborated. The review revealed that machine learning algorithms can be successfully used for performance prediction, service life prediction, and fault diagnosis in fuel cells, with good accuracy in solving nonlinear problems. Combined with optimization algorithms, machine learning models can further carry out the optimization of design and operating parameters to achieve multiple optimization goals with good accuracy and efficiency. It is expected that this review paper could help the reader comprehend the state of the art of machine learning applications in fuel fuels and shed light on further development directions in fuel cell research. [ABSTRACT FROM AUTHOR]

Short-term load forecasting (STLF) is critical for the energy industry. Accurate predictions of future electricity demand are necessary to ensure power systems' reliable and efficient operation. Various STLF models have been proposed in recent years, each with strengths and weaknesses. This paper comprehensively reviews some STLF models, including time series, artificial neural networks (ANNs), regression-based, and hybrid models. It first introduces the fundamental concepts and challenges of STLF, then discusses each model class's main features and assumptions. The paper compares the models in terms of their accuracy, robustness, computational efficiency, scalability, and adaptability and identifies each approach's advantages and limitations. Although this study suggests that ANNs and hybrid models may be the most promising ways to achieve accurate and reliable STLF, additional research is required to handle multiple input features, manage massive data sets, and adjust to shifting energy conditions. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs), including convolutional neural networks (CNNs) and residual networks (ResNets) models, are able to learn abstract representations from the input data by considering a deep hierarchy of layers that perform advanced feature extraction. The combination of these models with visual attention techniques can assist with the identification of the most representative parts of the data from a visual standpoint, obtained through more detailed filtering of the features extracted by the operational layers of the network. This is of significant interest for analyzing remotely sensed hyperspectral images (HSIs), characterized by their very high spectral dimensionality. However, few efforts have been conducted in the literature in order to adapt visual attention methods to remotely sensed HSI data analysis. In this paper, we introduce a new visual attention-driven technique for the HSI classification. Specifically, we incorporate attention mechanisms to a ResNet in order to better characterize the spectral–spatial information contained in the data. Our newly proposed method calculates a mask that is applied to the features obtained by the network in order to identify the most desirable ones for classification purposes. Our experiments, conducted using four widely used HSI data sets, reveal that the proposed deep attention model provides competitive advantages in terms of classification accuracy when compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]