Search

Cardiovascular diseases are currently the major causes of death globally. Among the strategies to prevent cardiovascular issues, the automated classification of heart sound abnormalities is an efficient way to detect early signs of cardiac conditions leading to heart failure or other, even asymptomatic, complications, quite effective for timely interventions. Despite the significant improvements in this field, there are still limitations due to the lack of solutions, available data-sets and poor (mainly binary — normal vs abnormal) classification models and algorithms. This paper presents a Medical Cyber–Physical System (MCPS) for the automatic classification of heart valve diseases onsite, in a timely manner. The proposed MCPS, indeed, can be deployed into personal and mobile devices, addressing the limitations of existing solutions for patients, healthcare practitioners, and researchers, through an efficient and easy accessible tool. It combines different neural network models trained on a new Italian dataset of 132 adult patients covering 9 heart sound categories (1 normal and 8 abnormal), also validated against two main open-access (Physionet/CinC Challenge 2016 and Korean) datasets. The overall MCPS performance (time, processing and energy resource utilization) and the high accuracy of the models (up to 98%) demonstrated the feasibility of the proposed solution, even with few data. The dataset supporting the findings of this paper is available upon request to the authors. • DB of 132 heart sounds of Italian patients in 9 categories, available upon request. • MCPS for automated HVD classification via hierarchical ML model structures. • Compared six ML classifiers (RF, SVM, NB, LR, DNN, LSTM) for HVD and majority voting. • MCPS validated on two open access datasets (Physionet/CinC 2016 and Korean). • Time, processing and energy efficiency of MCPS validated, with usage guidelines. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

The conventional standardized theoretical models (such as Webster, Alcelik, Indo-HCM) used for the delay estimation revolve around the mathematical hypothesis and assumptions, making them static with limitations in accommodating the dynamic traffic behaviors. Recent advances in artificial intelligence make machine learning techniques suitable for estimating vehicle control delay compared to conventional methods. This paper demonstrates the application of several machine learning models developed by focusing on the fluctuations of traffic observed at an intersection having heterogeneous traffic conditions in Ahmedabad city for delay estimation. Several parameters are extracted from on-field and video surveys. However, not all parameters are relevant for accurately estimating vehicle control delay. Hence, a feature selection process consisting of several feature-scoring techniques from filter, wrapper, and embedded methods is applied to all the parameters. This process gave insights into the most statistically relevant independent parameters, and out of all the parameters, cycle time was found to be insignificant, with a feature score of 0 from almost all techniques. Hence, it was removed, and then the prominent parameters were then used to build a vehicle control delay model using support vector regression (SVR), K-nearest neighbor (KNN), artificial neural network (ANN), random forest regression (RF), and decision tree regression (DT). Error distribution, standard deviation of errors, coefficient of Determination (R2), and Root Mean Squared Error (RMSE) are the parameters used for evaluating the performance of the machine learning models. RF outperformed them all with a standard deviation of errors, R2, and RMSE of 11.065, 0.926, and 11.081 on testing data. But ANN, KNN, and DT also performed satisfactorily. Compared with conventional standardized theoretical models, all the machine learning models except SVR performed better. [ABSTRACT FROM AUTHOR]

• A scheme to recognize the distraction level was proposed. • MLP-AutoEncoder and DPC algorithms were used to define distraction-level labels. • Factors affecting driver distraction level were explored with mixed-model analyses. • A distraction-level prediction model with stacking ensemble learning was made. Driver distraction-level recognition while performing secondary tasks in full-touch in-vehicle information systems (FTIVISs) is essential for the harmonious co-driving of human and intelligent vehicle systems. However, there has been little research on this topic. To respond to this issue, this paper proposes a distraction-level recognition framework with a combination of semi-supervised learning, unsupervised learning, and supervised learning. First, unsupervised learning and semi-supervised is used to divide the collected unlabeled samples of driving distraction behavior into three categories of distraction levels: high, medium, and low. Second, the factors influencing the distraction level are explored through a mixed model analysis. Finally, a stacking-based ensemble learning model is proposed to recognize the driver distraction level by supervised learning, with the influencing factors of the distraction level used as model input parameters. To improve the recognition performance of the stacking model, four heterogeneous classifiers selected as the base classifiers, and principal component analysis (PCA) is introduced in the base classifier layer, which improves the ability of the meta -classifier to process raw features based on overfitting resistance. We conducted a real road experiment under different road and FTIVIS task conditions, and the proposed model performed better than traditional machine learning models. In addition, the model exhibited the greatest advantage when the deep neural network (DNN) algorithm was used as the meta -classifier of the model, with a recognition accuracy of 92.5%. The study findings are significant for developing a human–machine co-driving control strategy and improving vehicle driving safety. [ABSTRACT FROM AUTHOR]

Simulation-driven structural health monitoring (SHM) is crucial for enhancing the safety and reliability of structures. Its primary aim is to utilize numerical simulations to design efficient SHM systems, reducing costs while maintaining effectiveness. However, challenges arise due to disparities between simulations and physical measurements, stemming from uncertainties, noise, and simplifications. These challenges also extend to SHM methods using machine learning, particularly deep neural networks (DNNs). This paper bridges the gap between DNNs trained on simulated data and real-world experiments by investigating three transfer learning (TL) techniques: feature augmentation, direct fine-tuning, and boosting-based fine-tuning. The focus is on guided wave damage interaction coefficients (WDICs) as damage-sensitive features, modeled by DNNs for damage identification in thin plates, commonly found in aerospace structures like aircraft fuselages. This classification task compares measured and predicted WDICs in a common guided wave SHM configuration with one actuator and three sensors. Training and extensive testing datasets are generated using advanced finite element simulations and laser Doppler vibrometer experiments. The study demonstrates that TL techniques significantly enhance DNN prediction accuracy and improve damage identification performance for untrained scenarios compared to purely simulation-based DNNs. Among the TL approaches explored, feature augmentation with appropriate activation functions and boosting-based fine-tuning yield the most promising results. Using four out of twelve TL datasets, feature augmentation reduces the simulation-based mean absolute percentage error from 46.80% to 35.98% and increases coefficient of determination R 2 values from 0.02 to 0.72. With six TL datasets and a boosting-based fine tuning, the mean absolute percentage error decreases from 47.42% to 35.20%, maintaining similar R 2 values. While TL substantially enhances prediction accuracy, its impact on damage identification is slightly less pronounced, with improvements in relative classification accuracy by approximately 5% compared to simulation-based DNNs. These findings highlight the efficacy of TL techniques for damage identification in thin plates using WDICs and DNNs, emphasizing the influence of TL training and DNN model variations. Furthermore, they showcase the potential of TL methods to enhance DNN performance in future SHM applications, particularly emphasizing the promise of feature augmentation and boosting. • Novel simulation-driven structural health monitoring technique using guided waves. • Deep neural network modeling to predict wave damage interaction coefficients. • Bridging disparity between simulations and experiments through transfer learning. • Study of transfer learning techniques based on feature augmentation and fine tuning. • Demonstrated significant enhancement in simulation-based neural network performance. [ABSTRACT FROM AUTHOR]

Researchers and practitioners have attempted to predict the financial market by analyzing textual (e.g., news articles and social media) and numeric data (e.g., hourly stock prices, and moving averages). Among textual data, while many papers have been published that analyze social media, news content has gained limited attention in predicting the stock market. Acknowledging that news is critical in predicting the stock market, the focus of this systematic review is on papers investigating machine learning and text mining techniques to predict the stock market using news. Using Kitchenham's methodology, we present a systematic review of the literature on intelligent financial market prediction, examining data mining and machine learning approaches and the employed datasets. From five digital libraries, we identified 61 studies from 2015–2022 for synthesis and interpretation. We present notable gaps and barriers to predicting financial markets, then recommend future research scopes. Various input data, including numerical (stock prices and technical indicators) and textual data (news text and sentiment), have been employed for news-based stock market prediction. News data collection can be costly and time-consuming: most studies have used custom crawlers to gather news articles; however, there are financial news databases available that could significantly facilitate news collection. Furthermore, although most datasets have covered fewer than 100K records, deep learning and more sophisticated artificial neural networks can process enormous datasets faster, improving future model performance. There is a growing trend toward using artificial neural networks, particularly recurrent neural networks and deep learning models, from 2018 to 2021. Furthermore, regression and gradient-boosting models have been developed for stock market prediction during the last four years. Although word embedding approaches for feature representation have been employed recently with good accuracy, emerging language models may be a focus for future research. Advanced natural language processing methods like transformers have undeniably contributed to intelligent stock market prediction. However, stock market prediction has not yet taken full advantage of them. • Emerging articles (2015–2021) on news-based stock market prediction are reviewed. • There is a growing trend toward using news headlines for stock market prediction. • Recurrent neural networks are advantageous models for stock market prediction. • Stock market prediction not fully took advantage of Natural language processing yet. • Quarterly and annual reports of companies are authorized credible data sources. [ABSTRACT FROM AUTHOR]

• Volatility forecasts are improved by stacking machine learning algorithms. • Merging Artificial Neural Networks with other models increases its forecasting power. • Regardless of the volatility model, high volatile regimes lead to higher error rates. • Risk measurements precision is increased by stacking machine learning models. An appropriate calibration and forecasting of volatility and market risk are some of the main challenges faced by companies that have to manage the uncertainty inherent to their investments or funding operations such as banks, pension funds or insurance companies. This has become even more evident after the 2007–2008 Financial Crisis, when the forecasting models assessing the market risk and volatility failed. Since then, a significant number of theoretical developments and methodologies have appeared to improve the accuracy of the volatility forecasts and market risk assessments. Following this line of thinking, this paper introduces a model based on using a set of Machine Learning techniques, such as Gradient Descent Boosting, Random Forest, Support Vector Machine and Artificial Neural Network, where those algorithms are stacked to predict S&P500 volatility. The results suggest that our construction outperforms other habitual models on the ability to forecast the level of volatility, leading to a more accurate assessment of the market risk. [ABSTRACT FROM AUTHOR]

Highlights • We define market shocks as the innovation of ARMA-GARCH. • We develop a methodology ARMA-GARCH-NN for market shock prediction. • A nearest-K cross-validation method is proposed and applied. • The predictability of market shocks is confirmed by experiments on the S&P 500 data. • The prediction result is used as a new signal for trading strategy development. Abstract Discovering hidden patterns under unexpected market shocks is a significant and challenging problem, which continually attracts attention from research communities of mathematics, economics, and data science. Classic financial pricing models present unsatisfactory prediction accuracy when applied to real-world data due to limited capacity in depicting complex market movements. In this paper, we develop a machine learning approach, called ARMA-GARCH-NN, to capture intra-day patterns for stock market shock forecasting. Specifically, we integrate classical financial pricing models with artificial neural networks, with explicitly designed feature selection and cross-validation methods. We conduct empirical studies on high-frequency data of the U.S. stock market for evaluation. Our results provide initial evidence of the predictability of market shocks. Additionally, we confirm the effectiveness of ARMA-GARCH-NN by recognizing patterns in massive stock data without strong assumptions on distribution. Our method can serve as a portable methodology that integrates the advantages of traditional financial models and data-driven methods to reveal hidden patterns in large-scale financial data. [ABSTRACT FROM AUTHOR]

Highlights • An information system is designed, and expert knowledge mapped on it to establish a weighted digraph. • This digraph is analyzed using graph-theoretical concepts to identify the critical components of the system. • The system model is analyzed under three scenarios of component capacities. • Policy intervention strategies are designed based on the critical properties of the system. Abstract This paper develops a graph-theoretic approach to analyzing information system priorities using expert knowledge. The constructed information system model lends itself to the identification and quantification of priority pathways of information flow in the system where agents have different learning and information distribution capacities. The approach is developed in four steps. In step 1, a policy issue is formulated using systems approach. In step 2, through a workshop, expert knowledge is gathered on the interactions of system components. Distribution and learning capacities of individual components are approximated through a questionnaire. In step 3, with the aid of graph-theoretic concepts, system priorities are quantitatively refined. In step 4, using the findings from graph-theoretic analysis, specific policy interventions are developed for effective targeting of the policy issue. The approach developed is then illustrated with an application to food and nutrition security information system in Azerbaijan. Specific policy intervention strategies are proposed. [ABSTRACT FROM AUTHOR]

Highlights • Novel use of neural networks and multitask learning for yield curve forecasting. • Multilayer perceptron using the most relevant features achieved the best results. • The most relevant features depend on target yield and forecasting horizon. • Synthetic data from linear regression model tends to improve forecasting accuracy. • Encouraging results for the development of robust forecasting systems for bond market. Abstract The yield curve is the centrepiece in bond markets, a massive asset class with an overall size of USD 100 trillion that remains relatively under-investigated using machine learning. This paper is the first comprehensive study using artificial neural networks in the context of yield curve forecasting. Specifically, two models were used for forecasting the European yield curve: multivariate linear regression and multilayer perceptron (MLP), at five forecasting horizons, from next day to 20 days ahead. Five variants of the MLP were analysed with different sets of features: target to predict (univariate); the most relevant features; all generated features; and the former two incorporating synthetic data generated by the linear regression model. Additionally, two different techniques of multitask learning were employed: simultaneous modelling and transformation into multiple single task learning. The results show that considering all forecasting horizons, the MLP using the most relevant features achieved the best results and the addition of synthetic data tends to improve accuracy. Furthermore, different targets and forecasting horizons resulted in different relevant features, reinforcing the importance of custom-built models. In the two multitask learning methodologies no clear differentiation could be demonstrated, and several explaining factors are identified. Overall, the outcome is very encouraging for the development of better forecasting systems for fixed income markets. [ABSTRACT FROM AUTHOR]

Highlights • We achieve state-of-the-art results for five fine art-related classification tasks. • Different convolutional neural network fine-tuning strategies are explored. • Impact of various source domain-dependent weight initialization is studied. • Networks pre-trained for scene and sentiment recognition perform best for art tasks. • Fine-tuned models can be used to retrieve images similar in style or content. Abstract The increasing availability of large digitized fine art collections opens new research perspectives in the intersection of artificial intelligence and art history. Motivated by the successful performance of Convolutional Neural Networks (CNN) for a wide variety of computer vision tasks, in this paper we explore their applicability for art-related image classification tasks. We perform extensive CNN fine-tuning experiments and consolidate in one place the results for five different art-related classification tasks on three large fine art datasets. Along with addressing the previously explored tasks of artist, genre, style and time period classification, we introduce a novel task of classifying artworks based on their association with a specific national artistic context. We present state-of-the-art classification results of the addressed tasks, signifying the impact of our method on computational analysis of art, as well as other image classification related research areas. Furthermore, in order to question transferability of deep representations across various source and target domains, we systematically compare the effects of domain-specific weight initialization by evaluating networks pre-trained for different tasks, varying from object and scene recognition to sentiment and memorability labelling. We show that fine-tuning networks pre-trained for scene recognition and sentiment prediction yields better results than fine-tuning networks pre-trained for object recognition. This novel outcome of our work suggests that the semantic correlation between different domains could be inherent in the CNN weights. Additionally, we address the practical applicability of our results by analysing different aspects of image similarity. We show that features derived from fine-tuned networks can be employed to retrieve images similar in either style or content, which can be used to enhance capabilities of search systems in different online art collections. [ABSTRACT FROM AUTHOR]

Web traffic refers to the amount of data that is sent and received by people visiting online websites. Web traffic anomalies represent abnormal changes in time series traffic, and it is important to perform detection quickly and accurately for the efficient operation of complex computer networks systems. In this paper, we propose a C-LSTM neural network for effectively modeling the spatial and temporal information contained in traffic data, which is a one-dimensional time series signal. We also provide a method for automatically extracting robust features of spatial-temporal information from raw data. Experiments demonstrate that our C-LSTM method can extract more complex features by combining a convolutional neural network (CNN), long short-term memory (LSTM), and deep neural network (DNN). The CNN layer is used to reduce the frequency variation in spatial information; the LSTM layer is suitable for modeling time information; and the DNN layer is used to map data into a more separable space. Our C-LSTM method also achieves nearly perfect anomaly detection performance for web traffic data, even for very similar signals that were previously considered to be very difficult to classify. Finally, the C-LSTM method outperforms other state-of-the-art machine learning techniques on Yahoo's well-known Webscope S5 dataset, achieving an overall accuracy of 98.6% and recall of 89.7% on the test dataset. [ABSTRACT FROM AUTHOR]

With the explosive growth in size of datasets, it becomes more significant to develop effective learning schemes for neural networks to deal with large scale data modelling. This paper proposes an iterative approximate Newton-type learning algorithm to build neural networks with random weights (NNRWs) for problem solving, where the whole training samples are divided into some small subsets under certain assumptions, and each subset is employed to construct a local learner model for integrating a unified classifier. The convergence of the output weights of the unified learner model is given. Experimental results on UCI datasets with comparisons demonstrate that the proposed algorithm is promising for large scale datasets. [ABSTRACT FROM AUTHOR]

One of the main ingredients to learn a visual representation of an object using the Convolutional Neural Networks is a large and carefully annotated dataset. Acquiring a dataset in a demanded scale is not a straightforward task; therefore, the community attempts to solve this problem by creating noisy datasets gathered from web sources. In this paper, this issue is tackled by designing a vehicle recognition system using Convolutional Neural Networks and noisy web data. In the proposed system, the transfer learning technique is employed, and behavior of several deep architectures trained on a noisy dataset are studied. In addition, the external noise of the gathered dataset is reduced by exploiting an unsupervised method called Isolation Forest, and the new training results are examined. Based on the experiments, high recognition accuracies were achieved by training two states of the art networks on the noisy dataset, and the obtained results were slightly improved by using the proposed noise reduction framework. Finally, a demonstration application is provided to show the capability and the performance of the proposed approach. [ABSTRACT FROM AUTHOR]

Technical and quantitative analysis in financial trading use mathematical and statistical tools to help investors decide on the optimum moment to initiate and close orders. While these traditional approaches have served their purpose to some extent, new techniques arising from the field of computational intelligence such as machine learning and data mining have emerged to analyse financial information. While the main financial engineering research has focused on complex computational models such as Neural Networks and Support Vector Machines, there are also simpler models that have demonstrated their usefulness in applications other than financial trading, and are worth considering to determine their advantages and inherent limitations when used as trading analysis tools. This paper analyses the role of simple machine learning models to achieve profitable trading through a series of trading simulations in the FOREX market. It assesses the performance of the models and how particular setups of the models produce systematic and consistent predictions for profitable trading. Due to the inherent complexities of financial time series the role of attribute selection, periodic retraining and training set size are discussed in order to obtain a combination of those parameters not only capable of generating positive cumulative returns for each one of the machine learning models but also to demonstrate how simple algorithms traditionally precluded from financial forecasting for trading applications presents similar performances as their more complex counterparts. The paper discusses how a combination of attributes in addition to technical indicators that has been used as inputs of the machine learning-based predictors such as price related features, seasonality features and lagged values used in classical time series analysis are used to enhance the classification capabilities that impacts directly into the final profitability. [ABSTRACT FROM AUTHOR]

The paper focuses on the task of predicting future values of stock market index. Two indices namely CNX Nifty and S&P Bombay Stock Exchange (BSE) Sensex from Indian stock markets are selected for experimental evaluation. Experiments are based on 10 years of historical data of these two indices. The predictions are made for 1–10, 15 and 30 days in advance. The paper proposes two stage fusion approach involving Support Vector Regression (SVR) in the first stage. The second stage of the fusion approach uses Artificial Neural Network (ANN), Random Forest (RF) and SVR resulting into SVR–ANN, SVR–RF and SVR–SVR fusion prediction models. The prediction performance of these hybrid models is compared with the single stage scenarios where ANN, RF and SVR are used single-handedly. Ten technical indicators are selected as the inputs to each of the prediction models. [ABSTRACT FROM AUTHOR]

The QArt-Learn approach for style painting categorization based on Qualitative Color Descriptors (QCD), color similarity ( SimQCD ), and quantitative global features (i.e. average of brightness, hue, saturation and lightness and brightness contrast) is presented in this paper. k -Nearest Neighbor ( k -NN) and support vector machine (SVM) techniques have been used for learning the features of paintings from the Baroque, Impressionism and Post-Impressionism styles. Specifically two classifiers are built, and two different parameterizations have been applied for the QCD. For testing QArt-Learn approach, the Painting-91 dataset has been used, from which the paintings corresponding to Velázquez, Vermeer, Monet, Renoir, van Gogh and Gauguin were extracted, resulting in a set of 252 paintings. The results obtained have shown categorization accuracies higher than 65%, which are comparable to accuracies obtained in the literature. However, QArt-Learn uses qualitative color names which can describe style color palettes linguistically, so that they can be better understood by non-experts in art since QCDs are aligned with human perception. [ABSTRACT FROM AUTHOR]

Classification algorithms are used in many domains to extract information from data, predict the entry probability of events of interest, and, eventually, support decision making. This paper explores the potential of extreme learning machines (ELM), a recently proposed type of artificial neural network, for consumer credit risk management. ELM possess some interesting properties, which might enable them to improve the quality of model-based decision support. To test this, we empirically compare ELM to established scoring techniques according to three performance criteria: ease of use, resource consumption, and predictive accuracy. The mathematical roots of ELM suggest that they are especially suitable as a base model within ensemble classifiers. Therefore, to obtain a holistic picture of their potential, we assess ELM in isolation and in conjunction with different ensemble frameworks. The empirical results confirm the conceptual advantages of ELM and indicate that they are a valuable alternative to other credit risk modelling methods. [ABSTRACT FROM AUTHOR]

A number of industries use human inspection to visually classify the quality of their products and the raw materials used in the production process, this process could be done automatically through digital image processing. The industries are not always interested in the most accurate technique for a given problem, but most appropriate for the expected results, there must be a balance between accuracy and computational cost. This paper investigates the classification of the quality of wood boards based on their images. For such, it compares the use of deep learning, particularly Convolutional Neural Networks, with the combination of texture-based feature extraction techniques and traditional techniques: Decision tree induction algorithms, Neural Networks, Nearest neighbors and Support vector machines. Reported studies show that Deep Learning techniques applied to image processing tasks have achieved predictive performance superior to traditional classification techniques, mainly in high complex scenarios. One of the reasons pointed out is their embedded feature extraction mechanism. Deep Learning techniques directly identify and extract features, considered by them to be relevant, in a given image dataset. However, empirical results for the image data set have shown that the texture descriptor method proposed, regardless of the strategy employed is very competitive when compared with Convolutional Neural Network for all the performed experiments. The best performance of the texture descriptor method could be caused by the nature of the image dataset. Finally are pointed out some perspectives of futures developments with the application of Active learning and Semi supervised methods. [ABSTRACT FROM AUTHOR]

In the context of recommendation systems, metadata information from reviews written for businesses has rarely been considered in traditional systems developed using content-based and collaborative filtering approaches. Collaborative filtering and content-based filtering are popular memory-based methods for recommending new products to the users but suffer from some limitations and fail to provide effective recommendations in many situations. In this paper, we present a deep learning neural network framework that utilizes reviews in addition to content-based features to generate model based predictions for the business-user combinations. We show that a set of content and collaborative features allows for the development of a neural network model with the goal of minimizing logloss and rating misclassification error using stochastic gradient descent optimization algorithm. We empirically show that the hybrid approach is a very promising solution when compared to standalone memory-based collaborative filtering method. [ABSTRACT FROM AUTHOR]

In this paper, we provide an experimental study for two unsupervised processes, namely, the random initialization and the Hebbian learning, which can be used to determine the input weights in Single-hidden Layer Feedforward Neural Networks (SLFNs). In addition, a fusion technique that combines the two feature spaces is proposed. Experiments are conducted on six publicly available facial image datasets. Experimental results show that the proposed fusion technique can improve the performance of Hebbian and random feature spaces when they achieve similar performance. In the cases where the difference in performance of the two feature spaces is high, the proposed fusion scheme preserves the power of the most discriminating one and outperforms the average fused feature space. The experimental results show that there is a trade-off between the generalization of the Hebbian feature space and the low computational cost of the random one. [ABSTRACT FROM AUTHOR]

Collaborative training of deep neural networks using edge devices has attracted substantial research interest recently. The two main architecture approaches for the training process are centrally orchestrated Federated Learning and fully decentralized peer-to-peer learning. In decentralized systems, edge devices, known as agents, collaborate in a peer-to-peer architecture, avoiding the need for a central system to orchestrate the process. Decentralized peer-to-peer (P2P) learning techniques are well researched under the assumption of independent and identically distributed (IID) data across the agents. IID data is seldom observed in real-world distributed systems, and the training performance varies significantly with non-IID data. This paper proposes a decentralized learning variant of the P2P gossip averaging method with Batch Normalization (BN) adaptation for P2P architectures. It is well-known that BN layers accelerate the convergence of the non-distributed deep learning models. Recent research confirms that Federated Learning methods benefit from using the BN method with some aggregation alterations. Our work demonstrated BN effectiveness in P2P architectures by mitigating the non-IID data characteristics across decentralized agents. We also introduce a variant of the early stopping technique that, combined with BN layers, acts as a fine-tuning technique for agent models. We validated our approach by conducting numerous simulations of different model-topology-communication combinations and comparing them to other decentralized baseline approaches. The evaluations were conducted on the next word prediction task using user comments from the Reddit and StackOverflow datasets representing comments from two different domains. Simulations showed that our approach, on average, achieves a mean relative top accuracy increase of 16.9% in ring (19.9% for Reddit, 13.9% for StackOverflow) and 29.8% in sparse (32.9% for Reddit, 26.6% for StackOverflow) communication topologies compared to the best baseline approach. Our code is available at https://github.com/fipu-lab/p2p_bn. • Batch Normalization layers improve decentralized learning on non-IID data. • P2P-BN improves decentralized learning by fine-tuning the Batch Normalization layers. • P2P-BN produces stable models even in sparse communication topologies. [ABSTRACT FROM AUTHOR]

Deep neural networks and model-based methods are both popular for their wide and great success in many inference problems. In this paper, resorting to deep learning, we study the efficient algorithms for two popular nonconvex regularization methods, smoothly clipped absolute deviation (SCAD) and minimax concave penalty (MCP). Approximate message passing (AMP) algorithm can be effective to optimize nonconvex regularization models. First, we unroll the AMP-based algorithm as the feed-forward neural network through leveraging the novel activation functions of neurons, dubbed as Unrolled-AMP. And then, for the case where the measurement matrix deviates from the i.i.d. Gaussian distribution, we propose two improved iterative algorithms based on the "Vector AMP (VAMP)" algorithm to solve the nonconvex regularization methods. Further, we unroll them as the feed-forward neural network, dubbed as Unrolled-VAMP. These two novel neural network architectures use a back-propagation algorithm to learn all their parameters from the training data. Finally, the convergence of Unrolled-AMP algorithm is analyzed, and the efficiency of the proposed networks is demonstrated through experiments on sparse signal reconstruction and 5G wireless communication. • We develop efficient algorithms for solving nonconvex regularization methods. • Two types iterative algorithms are unrolled as deep learning architectures. • We give convergence analysis of the proposed networks. • The proposed networks require few layers to achieve a specific level of accuracy. [ABSTRACT FROM AUTHOR]

Due to computational complexities of artificial neural networks (ANNs), there is an increasing demand for third parties and MLaaS (machine learning as a service) to take charge of the training procedure. Therefore, making ANNs robust against adversarial attacks has received a lot of attention. Backdoor attacks, which causes targeted mis-classification while the accuracy on clean data is not affected, are among the most efficient attacks. In this paper, we propose a method called link-pruning with scale-freeness (LPSF), in which the dormant threatening links from the neurons in the input layer to other neurons of feed-forward neural network are eliminated according to the information gained from a portion of clean input data and the essential links are strengthened by changing the fully-connected networks to scale-free structures. To the best of our knowledge, it is the first defense method that makes the network significantly robust against backdoor attack (BD) before the network is attacked. LPSF is evaluated on feed-forward neural networks and with malicious MNIST, FMNIST, handwritten Chinese characters and HODA datasets. Through LPSF strategy, we achieve a sufficiently high and stable accuracy on clean data and an exceeding reduction range of 50 % − 94 % for attack success rate. • Background on scale-free structures as complex networks and applications to ANNs. • Previous works on detecting and mitigating techniques against different types of attacks to ANNs. • A new method based on scale-free networks to make the feed-forward neural network robust. • Experiments has shown the effectiveness of the proposed method against backdoor attacks. [ABSTRACT FROM AUTHOR]

• An interval type-2 fuzzy model for online prediction of stock market's future trend. • Capable of considering a high amount of uncertainty in stock data. • Membership degrees are extracted from information of data not human knowledge. • The proposed algorithm incrementally extracts type-2 fuzzy sets. • The method has low computational complexity. Due to its complex and unstable nature, stock market prediction is one of the most challenging problems. Lots of research has tried to explore the possibility of predicting stock trends using numerous machine learning methods. However, little effort has been made to predict the future trend online. In this paper, we propose an incremental type-2 fuzzy rule-base classifier capable of considering a high amount of uncertainty in stock data in addition to self-adaptation in an online manner. In this approach, interval type-2 fuzzy sets are built by combining type-1 sets that are directly extracted from the data. The proposed classifier can dynamically modify its parameters in order to react to the alteration of concepts without re-training the model. The efficiency of the proposed method is tested on some market indexes such as SP500, TAIEX, etc. and compared to various incremental methods. Moreover, to show the effectiveness of the proposed technique even in offline mode, we have compared it against some outstanding offline approaches, including deep neural networks. The obtained results demonstrate that our proposed method performs well under different settings. [ABSTRACT FROM AUTHOR]

A smart camera is a vision system capable of extracting application-specific information from the captured images. The paper proposes a decentralized and efficient solution for visual parking lot occupancy detection based on a deep Convolutional Neural Network (CNN) specifically designed for smart cameras. This solution is compared with state-of-the-art approaches using two visual datasets: PKLot, already existing in literature, and CNRPark-EXT. The former is an existing dataset, that allowed us to exhaustively compare with previous works. The latter dataset has been created in the context of this research, accumulating data across various seasons of the year, to test our approach in particularly challenging situations, exhibiting occlusions, and diverse and difficult viewpoints. This dataset is public available to the scientific community and is another contribution of our research. Our experiments show that our solution outperforms and generalizes the best performing approaches on both datasets. The performance of our proposed CNN architecture on the parking lot occupancy detection task, is comparable to the well-known AlexNet, which is three orders of magnitude larger. [ABSTRACT FROM AUTHOR]

Multi-class classification is one of the major challenges in real world application. Classification algorithms are generally binary in nature and must be extended for multi-class problems. Therefore, in this paper, we proposed an enhanced Genetically Optimized Neural Network (GONN) algorithm, for solving multi-class classification problems. We used a multi-tree GONN representation which integrates multiple GONN trees; each individual is a single GONN classifier. Thus enhanced classifier is an integrated version of individual GONN classifiers for all classes. The integrated version of classifiers is evolved genetically to optimize its architecture for multi-class classification. To demonstrate our results, we had taken seven datasets from UCI Machine Learning repository and compared the classification accuracy and training time of enhanced GONN with classical Koza’s model and classical Back propagation model. Our algorithm gives better classification accuracy of almost 5% and 8% than Koza’s model and Back propagation model respectively even for complex and real multi-class data in lesser amount of time. This enhanced GONN algorithm produces better results than popular classification algorithms like Genetic Algorithm, Support Vector Machine and Neural Network which makes it a good alternative to the well-known machine learning methods for solving multi-class classification problems. Even for datasets containing noise and complex features, the results produced by enhanced GONN is much better than other machine learning algorithms. The proposed enhanced GONN can be applied to expert and intelligent systems for effectively classifying large, complex and noisy real time multi-class data. [ABSTRACT FROM AUTHOR]

• A three-phase RDA-based clustering undersampling method to handle class imbalance. • An RDA-based clustering feature extraction technique to reduce data dimensionality. • The proposed method could efficiently address the class imbalance problem. • The performance of SVM, ANN, KNN, DT, and an ensemble method is compared. • Decision tree outperformed other classifiers with the value of 0.95 for AUC. The prevalence of chronic kidney diseases has raised the demand for kidney replacement therapy over recent years. Although kidney transplantation provides better life quality than other alternatives, the replacement might face rejection and endanger individuals' lives. Therefore, predicting this rejection is of great importance, and due to the complexity of the human body's immune system, it is a complicated task. Machine learning has been widely used in disease diagnoses such as graft rejection. One of the common issues in disease diagnosis is the class imbalance problem. The predictive model is likely to misclassify all of the minority class samples in such cases. This paper proposes a method that can accurately predict kidney rejection in an imbalanced dataset. We have adopted kidney transplantation data from 378 patients collected from 1994 to 2011. Our main contribution is to develop a novel clustering method using Red Deer Algorithm (RDA), which is later utilized in proposing a three-stage clustering-based undersampling approach to handle the class imbalance problem so that the final predictive models can classify the given data more accurately. The undersampling method includes denoising, RDA clustering, and sample selection. Moreover, the proposed clustering method is also used to reduce the data dimensionality. Subsequently, five different classification algorithms such as Support Vector Machine (SVM), Artificial Neural Network (ANN), K-Nearest Neighbor (KNN), Decision Tree (DT), and an ensemble method are used to predict kidney graft rejection. Then, these classifiers are compared in terms of five performance evaluation metrics: accuracy, sensitivity, specificity, F1 score, and area under the Receiver Operating Characteristic (ROC) curve (AUC). The obtained results indicate that the decision tree model outperformed other algorithms and achieved 0.96, 0.94, 0.97, 0.95, and 0.95 for accuracy, sensitivity, specificity, F1 score, and AUC, respectively. Hence, it is highly recommended to use the proposed method as a decision support system for clinical experts to predict kidney transplantation failure. [ABSTRACT FROM AUTHOR]

Predictive Maintenance has become an important component in modern industrial scenarios, as a way to minimize down-times and fault rate for different equipment. In this sense, while machine learning and deep learning approaches are promising due to their accurate predictive abilities, their data-heavy requirements make them significantly limited in real world applications. Since one of the main issues to overcome is lack of consistent training data, recent work has explored the possibility of adapting well-known deep-learning models for image recognition, by exploiting techniques to encode time series as images. In this paper, we propose a framework for evaluating some of the best known time series encoding techniques, together with Convolutional Neural Network -based image classifiers applied to predictive maintenance tasks. We conduct an extensive empirical evaluation of these approaches for the failure prediction task on two real-world datasets (PAKDD2020 Alibaba AI OPS Competition and NASA bearings), also comparing their performances with respect to the state-of-the-art approaches. We further discuss advantages and limitation of the exploited models when coupled with proper data augmentation techniques. • We analyze Predictive Maintenance task aiming to minimize down-times of an equipment. • We use time series encoding techniques with CNN-based models in different scenarios. • An extensive evaluation has been performed for HDD and bearings failure prediction. • Performance has been evaluated w.r.t. other benchmarking neural network models. [ABSTRACT FROM AUTHOR]

• Gear shift strategy. • Mine truck. • Machine learning. • Bi-LSTM network. As mine trucks face severe and changeable driving conditions in mining area, it is important to formulate a general automatic gear shift strategy for reducing the intensity and difficulty of the work. Therefore, gear shift strategy performs a significant role in automatic transmission studies. However, it is difficult to devise an automatic shift strategy due to the fact that it relies on tremendous historical experiment data which is the highest classified commercial secret of the automobile manufacture. Since the artificial neural network can build self-learning model by inputting multiple parameters, adjusting weights and automatically outputting results, many automatic gear shift strategy studies are reported based on it. However, the gear shift prediction accuracy by traditional neural network is limited due to the non-existence of feedback connection, lacking of associative memory function and missing implicit connection between continuous data. Hence, this paper proposes a novel method based on Bi-directional Long Short-Term Memory (Bi-LSTM) network, which controls gear shifting with nine parameters and updates the traditional mine truck transmission system from manual to automatic. Truck status data while manual driving are captured through the Controller Area Network-bus (CAN-bus) and converted to the proper sequence data format. Some sequence data are used to train the Bi-LSTM network for gear prediction, while the rest is used for verification. In order to evaluate the proposed machine learning model performance, it is compared with the Back Propagation Neural Network (BPNN), Recurrent Neural Network (RNN) and Long-Short Term Memory (LSTM) networks with two and nine parameters, respectively. The experimental results demonstrate that machine learning networks with nine parameters reach higher gear shifting prediction accuracy than those with dual-parameter, except BPNN. In addition, the highest prediction accuracy is 95.8 %, which is achieved by the Bi-LSTM network with nine parameters. This proves that Bi-LSTM network outperforms the traditional neural networks, such as BPNN with 81.5 %, and other recurrent neural networks, such as RNN and LSTM with 87 % and 91.5 %, respectively. As for the average time consumption, the Bi-LSTM network spends 1.86 ms per prediction calculation which is longer than LSTM and RNN respectively having 1.58 ms and 1.06 ms. Thus, it still fits the requirement of real-time processing on mine truck industry. In general, the Bi-LSTM network shows its advantage in processing sequential data within limited time, and has the potential to be widely applied in the field of prediction based on data with temporal characteristic. [ABSTRACT FROM AUTHOR]

• Computing on edge end improves the efficiency and reliability of data analysis. • A lightweight CNN model is efficient for real-time crowd density estimation on edge. • Better crowd density inference speed with a slight increase in estimation accuracy. • Equip the model in an IoT device to monitor the crowd density in a Subway Station. Crowd stampedes and incidents are critical threats to public security that have caused countless deaths during the past few decades. To avoid crowd stampedes, real-time crowd density estimation can help monitor crowd movements, and thus support a timely evacuation strategy development. In previous studies, scholars and engineers developed multiple video-based crowd density estimation algorithms based on deep neural networks. The excessive computational complexity of deep learning algorithms exacerbated the algorithm's efficiency, causing unacceptable real-time performance. In the Internet of Things era, deploying the crowd density estimation task with edge computing is an advanced strategy to maintain the real-time performance of the entire system. Considering the limited computational resources on the edge devices, deep learning-based crowd density estimation algorithms normally cannot be handled. To fulfill the deployment on the edge device, the algorithms need to be optimized with a smaller model size. Therefore, this paper proposes a lightweight Convolutional Neural Networks (CNN) based crowd density estimation model by combining the modified MobileNetv2 and the dilated convolution. Public crowd image data sets are used to conduct experiments for evaluating the performance of the proposed algorithm in terms of accuracy and inference speed. The results show that our model achieves much better inference speed accompanied by a slight increase in accuracy. The proposed method of this study can enhance the performance of the crowd monitoring system, and therefore help avoid crowd stampedes and incidents. [ABSTRACT FROM AUTHOR]

• The resource-constrained devices in Cyber-Physical Systems are considered. • The KD-TCNN model is utilized with knowledge distillation and metric learning. • A neural network training method called K-fold cross training is proposed. • The proposed model is tested using benchmark intrusion detection datasets. • Proposed mothed outperforms many state-of-the-art models. With the rapid development of technology and science, machine learning approaches and deep learning methods have been widely applied in industrial Cyber-Physical Systems. However, there are still some challenging issues for anomaly detection to classify various attacks in industrial CPS to ensure the cyber security, especially when dealing with resource-constrained IoT devices. In this paper, we propose a Knowledge Distillation model based on Triplet Convolution Neural Network to improve the model performance and greatly enhance the speed of anomaly detection for industrial CPS as well as reduce the complexity of the model. Specifically, during the training process, we design a robust model loss function to improve the training stability of the model. A new neural network training method called K-fold cross training is also proposed to enhance the accuracy of anomaly detection. A lot of experimental results demonstrate that the performance metrics of KD-TCNN on the benchmark datasets NSL-KDD and CIC IDS2017 have significant advantages over traditional deep learning approaches and the recent state-of-the-art models. Furthermore, when compared to the original model, our model's computational cost and size are both reduced by roughly 86% with just 0.4% accuracy loss. [ABSTRACT FROM AUTHOR]

• A novel dataset including combinations of RIR and BAR individuals has been created. • 20 characteristic egg features have been used to classify egg quality and HU. • Eggs have been classified as excellent quality and very good quality by ML models. • HU at the 24th week and the 32nd week have been classified by ML models. With the increase in the world population, the nutritional needs of people have been increased. The demand for eggs which is one of the most important food sources has been increased over years. Therefore, it is very important to inform people about egg quality in order to be prepared for adverse situations such as substitution, mislabeling, and fraud. In this study, it is aimed to specify egg quality without using haugh unit (HU). Besides, another aim is to find how much information HU carries about the specification of egg quality. A dataset including 20 features related to eggs taken from 438 chickens created by Poultry Research Institute (PRI) has been analyzed. An application that can classify egg qualities as very good and excellent using machine learning (ML) models like decision tree (DT), linear discriminant analysis (LDA), logistic regression (LR), naïve bayes (NB), support vector machines (SVM), K-nearest neighboring (KNN), random forest (RF) and artificial neural networks (ANN) has been developed in this study. In addition to that, HU at the 24th week and the 32nd week which are the most important classifier to determine egg qualities have been classified as low informative, medium informative, and high informative by the developed application. Egg quality has the best been classified by LR model based on accuracy and Matthews correlation coefficient (MCC) values as 98.6% and 0.96, respectively. HU at the 24th week has the best been classified by RF based on accuracy and MCC as 96.8% and 0.93, respectively. HU at the 32nd week has the best been classified by RF based on accuracy and MCC as 95.1% and 0.92, respectively. This paper mainly focuses on the classification of egg quality based on not only HU but also egg characteristic features and the importance of the informative feature of HU to classify egg quality. [ABSTRACT FROM AUTHOR]

High-accuracy temperature sensors usually come at a high cost, making them cost-ineffective solutions for mass-produced commercial products. The overall product cost can be one of the decisive factors for its success (e.g., commercial freezers). On the other hand, low-cost temperature sensors are viable options, but they suffer from accuracy issues. This paper presents a system aimed at improving the accuracy of a low-cost temperature sensor using a hybrid machine learning algorithm. The system utilizes a low-cost single-board computer connected to a series of sensors to collect data. These data are then used as an input vector to the proposed linear regression heterogeneous hybrid extreme learning machine (LR-HE-HyELM), a supervised learning algorithm responsible for increasing the sensors' accuracy. The proposed linear regression heterogeneous extreme learning machine (LR-HE-HyELM) combines linear regression with the self-adaptive heterogeneous hybrid extreme learning machine (SA-He-HyELM) algorithm. SA-He-HyELM is an extreme learning machine (ELM) based algorithm that utilizes a genetic algorithm (GA) for the creation of an optimized single-layer neural network (SLNN) having different combinations of custom-created neurons at its hidden layer. The proposed training algorithm has all the advantages of SA-He-HyELM. These advantages include utilizing low-order and higher-order neurons in the same network while retaining ELM's simplicity by making all GA parameters self-adaptive. The utilization of low-order and higher-order unit types in the same network resulted in optimized neural networks with better generalization ability than those with the same neuron types in their hidden layer. The creation of SA-He-HyELM was nature-inspired since the human brain contains different neuron types. The proposed system is tested with eight existing methods, with most of them having different neuron types in the hidden layer. The experimental results involved using a custom-created regression dataset having measurements taken from a commercial fridge. The proposed method managed to get the lowest mean square error (MSE) compared to the other eight methods. • The proposed approach retains extreme learning machine's simplicity. • The use of software means improves temperature sensors' accuracy in fridges. • The proposed algorithm makes all genetic algorithm parameters self-adaptive. [ABSTRACT FROM AUTHOR]

• Developed a novel neural network for CVD mortality prediction with clinical data. • Proposed a more time-efficient feature-based artificial neural network (ANN). • Investigated feature representation to improve ANN for practical guideline. • Boosted calibration of neural networks and enabled model to make less mistakes. • Made model updates in clinical context more accessible, flexible and reliable. Cardiovascular (CVD) is the leading cause of death worldwide and a significant public health concern. Therefore, its mortality prediction is crucial to many existing treatment guidelines. Medical claims data can be used to accurately foresee the health outcomes of patients contracting to a variety of diseases. Many machine learning algorithms, especially deep learning artificial neural networks, can predict mortality rate among patients with CVD using clinical data. Calibration of probabilistic prediction is essential for precise medical interventions as it indicates how well a model's output matches the probability of the event. However, deep learning neural networks are poorly calibrated. Through experiments, we observe that feature representation is an important factor influencing calibration. This paper proposes a novel feature-based deep learning neural network framework to predict the mortality rate among patients with CVD. Our focus is to present a comprehensive study to achieve advanced performance calibration of mortality prediction on CVD in leveraging deep learning architecture and feature representations. Our study demonstrates that the proposed feature-based neural network framework integrated with Principal Component Analysis or Autoencoders significantly reduces training time and boosts calibration, making model updates in clinical context more flexible and decision-making in medical prevention more reliable. [ABSTRACT FROM AUTHOR]

In general insurance companies, a correct estimation of liabilities plays a key role due to its impact on management and investing decisions. Since the Financial Crisis of 2007–2008 and the strengthening of regulation, the focus is not only on the total reserve but also on its variability, which is an indicator of the risk assumed by the company. Thus, measures that relate profitability with risk are crucial in order to understand the financial position of insurance firms. Taking advantage of the increasing computational power, this paper introduces a stochastic reserving model whose aim is to improve the performance of the traditional Mack's reserving model by applying an ensemble of Recurrent Neural Networks. The results demonstrate that blending traditional reserving models with deep and machine learning techniques leads to a more accurate assessment of general insurance liabilities. • The proposed Mack-Net model leads to a more accurate reserve prediction. • Mack-Net model generates more appropriate risk measures than Mack's model. • Mack-Net model does not increase the variance to generate an appropriate VaR. • Neural Networks estimate more accurate parameters than usual reserving models. [ABSTRACT FROM AUTHOR]

• European fishing ships are controlled by a vessel monitoring system (VMS). • FAMI system exceeds in reliability and accuracy the current VMS. • The mobile' sensors can be used by learning machines to predict fishing events. • Multilayer perceptrons identified correctly 96.26% of the fishing events. Knowing the activity of fishing vessels accurately and in real time means a leap in quality in the management of fishing activity. This paper presents the development of a new fishing activity monitoring integral system (FAMIS) that can complement and overcome the limitations of current fishing vessel monitoring systems (VMS). FAMIS is developed on the basis of a low-cost mobile device with GPS sensors, accelerometer, gyroscope and magnetic field and integrates different statistical methods (discriminant functions) and heuristics (artificial neural networks and vectorial support machines) as techniques to classify the information recorded by the sensors of a mobile device during fishing activity. The results obtained with FAMIS indicate that, in general, heuristics have a high degree of discrimination of each of the phases of fishing operation and that, in particular, multilayer perceptrons (MLPs) are capable of correctly identifying 96.3% of towing phases using only GPS and gyro sensors. [ABSTRACT FROM AUTHOR]

• Probabilistic Physics-guided Neural Network (PPgNN) is proposed for fatigue data. • PPgNN is flexible and does not impose restrictions on function types. • PPgNN includes known physics/knowledge constraints in the machine learning model. • PPgNN produces both accurate and physically consistent results. • Extensive experiments for fatigue P - S - N curves are conducted for model validation. A Probabilistic Physics-guided Neural Network (PPgNN) is proposed in this paper for probabilistic fatigue S - N curve estimation. The proposed model overcomes the limitations in existing parametric regression models and classical machine learning models for fatigue data analysis. Compared with explicit regression-type models (such as power law fitting), the PPgNN is flexible and does not impose restrictions on function types at different stress levels, mean stresses, or other factors. One unique benefit is that the proposed method includes the known physics/knowledge constraints in the machine learning model; the method can produce both accurate and physically consistent results compared with the classical machine learning model, such as neural network models. In addition, the PPgNN uses both failure and runout data in the training process, which encodes the runout data using a new proposed loss function, and is beneficial when compared with some existing models using only numerical point value data. A mathematical formulation is derived to include different types of physics constraints, which can deal with mean value, variance, and derivative/curvature constraints. Several data sets from open literature for fatigue S - N curve testing are used for model demonstration and model validation. Next, the proposed network architecture is extended to include multi-factor (e.g., mean stress, corrosion, frequency effect, etc.) fatigue data analysis. It is shown that the proposed PPgNN can serve as a flexible and robust model for general fitting and uncertainty quantification of fatigue data. This paper provides a feasible way to incorporate known physics/knowledge in neural network-based machine learning. This is achieved by properly designing the network topology and constraining the neural network's biases and weights. The benefits for the proposed physics-guided learning for fatigue data analysis are illustrated by comparing results from neural network models with and without physics guidance. The neural network model, without physics guidance, produces results contradictory to the common knowledge, such as a monotonic decrease of S - N curve slope and a monotonic increase of fatigue life variance as the stress level decreases. This problem can be avoided using the physics-guided learning model with encoded prior physics knowledge. [ABSTRACT FROM AUTHOR]

• An ANN-based chart with novel run rules is proposed for monitoring linear profiles. • A heuristic procedure is introduced for the design of the proposed chart. • The proposed chart outperforms other competing methods in most of the cases. • Adding run rules significantly improves the efficiency of ANN-based chart. • The proposed run rules is more effective than traditional Western Electric rules. In some applications, a relation between a response variable and one or more explanatory variables (referred as a "profile") characterizes the quality of a process. Profile monitoring is commonly performed through statistical methods, while machine learning schemes have not received much attention in this regard. In this paper, a control chart based on Artificial Neural Networks (ANN) is proposed to monitor linear profiles in phase II. In the proposed control chart, some novel run rules as the major contribution of this paper are also used to enhance the efficiency of the control chart and for faster detection of shifts. Simulation results revealed a good performance of the proposed control chart based on average run length (ARL) criterion. Further, a systematic ANN-based diagnostic procedure was proposed to identify which parameter has changed in the process. Finally, the implementation of the proposed scheme was illustrated through a real calibration example from the field of chemical engineering. [ABSTRACT FROM AUTHOR]

• Saffron adulterants detected and classified using a deep learning algorithm. • RGB photos took under uncontrolled/unstructured conditions. • Five CNN models were proposed for classification. • The best-proposed model can process an image in 36.26 ms with 99.67% accuracy. Saffron is the most expensive spice in the world which is traded internationally. Hence, Saffron adulterants is always a challenging issue. It should be noted that traditional methods that are being used to solve this problem, are costly and time-consuming. In this study, for the first time, Saffron adulterants was detected using a deep neural network and RGB photos taken under uncontrolled/unstructured conditions. There were five classes of fake Saffron include mixed saffron stamen and dyed straw, mixed stigma and stamens, dyed orange blossom, Safflower and dyed stamens, and two classes of genuine Saffron consisting of freeze-drying and microwave-drying saffron used to design, develop and optimize a Convolutional Neural Network (CNN). 60% of the images were used to train and optimize the neural network model. For preventing over-training, the number of images used in the training phase increased by applying the augmentation method. The general structure of the model to detect saffron adulterants consisted of multiple convolutional (input layer), pooling layers and fully-connected layers, respectively. In this study, five structures (S1, S2, S3, S4 and S5) were proposed for CNN using Batch Normalization methods. Additionally, this paper compared these five structures with Popular Convolutional Networks consisted of VGG11, ResNet18, ResNet50, Inception V3 and DarkNet53. According to the results, the best configuration was S5 that achieved the accuracy of 99.8% and the cross-entropy loss of 0.019. Although the time process of this structure was 36.26 ms, the shortest time process was 19.38 ms related to the S1. This number was more than 300 ms for Popular CNN. Based on the results of this study, imaging without restrictions with CNN can detect Saffron adulterants. [ABSTRACT FROM AUTHOR]

• A logic-guided neural network is presented to predict steel-concrete interfacial behaviors. • Logic and scientific principles are utilized in the logic-guided neural network. • Strategies are presented to utilize unstructured data and datasets with incomplete data. • The bond stress-slip curves of steel-concrete interface are predicted. • The logic-guided neural network achieves a desired performance of prediction. The interfacial behaviors play significant roles in various composite materials and structures. This paper presents a logic-guided neural network to seamlessly integrate data-driven methods and scientific knowledge in predicting interfacial properties of steel-concrete composites. The investigated properties include the bond strength, interface slip, and bond-slip relationship. Three methods are proposed to conform to logic and scientific principles: (1) logic data are generated to supplement experimental data; (2) a logic loss function is presented to guide the learning process; and (3) unstructured data and incomplete data are utilized to enlarge the dataset. The performance of the presented method is compared with four representative machine learning methods, which are artificial neural network, tree boosting, random forest, and epsilon-support vector regression. The results indicated that the proposed method achieved the highest accuracy. The coefficients of determination of the bond stress and compressive strength are higher than 0.95, and the predicted bond-slip behaviors conform to prior knowledge. The proposed method is useful for prediction of properties for composite materials and structures. [ABSTRACT FROM AUTHOR]

The Artificial Neural Network (ANN) is an important machine learning tool used in medical data classification for disease diagnosis. The learning algorithm in ANN training plays a crucial role in classification performance. Various approaches have been successfully applied as a learning algorithm for ANN training. This paper performs an experimental study that investigates the performance of different metaheuristics as learning algorithms to train the ANN for medical data classification tasks. The experiments are carried out on 15 well-known medical datasets. A comparative study is conducted with the classical Levenberg–Marquardt (LM) and other thirteen recent and relevant metaheuristics. Different evaluation criteria such as accuracy, sensitivity, specificity, precision, Geometric Mean, F-Measure, false-positive rate (FPR) are considered for performance estimation. The classification results are analyzed using Multi-Criteria Decision Making (MCDM) method, and the results with analysis establish that the Equilibrium Optimizer algorithm outperforms all the other algorithms included in the comparative study. • Artificial Neural Network training using metaheuristic algorithms. • An experimental study in medical data classification. • Performance analysis using multi-criteria decision making. • Equilibrium Optimizer shows superior performance over the competitive algorithms. [ABSTRACT FROM AUTHOR]

One in every eight women is susceptible to breast cancer, at some point of time in her life. Early detection and effective treatment is the only rescue to reduce breast cancer mortality. Accurate classification of a breast cancer tumor is an important task in medical diagnosis. Machine learning techniques are gaining importance in medical diagnosis because of their classification capability. In this paper, we propose a new, Genetically Optimized Neural Network (GONN) algorithm, for solving classification problems. We evolve a neural network genetically to optimize its architecture (structure and weight) for classification. We introduce new crossover and mutation operators which differ from standard crossover and mutation operators to reduce the destructive nature of these operators. We use the GONN algorithm to classify breast cancer tumors as benign or malignant. To demonstrate our results, we had taken the WBCD database from UCI Machine Learning repository and compared the classification accuracy, sensitivity, specificity, confusion matrix, ROC curves and AUC under ROC curves of GONN with classical model and classical back propagation model. Our algorithm gives classification accuracy of 98.24%, 99.63% and 100% for 50–50, 60–40, 70–30 training–testing partition respectively and 100% for 10 fold cross validation. The results show that our approach works well with the breast cancer database and can be a good alternative to the well-known machine learning methods. [ABSTRACT FROM AUTHOR]

The production of a motion picture is an expensive, risky endeavor. During the five-year period from 2008 through 2012, approximately 90 films were released in the United States with production budgets in excess of $100 million. The majority of these films failed to recoup their production costs via gross domestic box office revenues. Existing decision support systems for pre-production analysis and green-lighting decisions lack sufficient accuracy to meaningfully assist decision makers in the film industry. Established models focus primarily upon post-release and post-production forecasts. These models often rely upon opening weekend data and are reasonably accurate but only if data up until the moment of release is included. A forecast made immediately prior to the debut of a film, however, is of limited value to stakeholders because it can only influence late-stage adjustments to advertising or distribution strategies and little else. In this paper we present the development of a model based upon a dynamic artificial neural network (DAN2) for the forecasting of movie revenues during the pre-production period. We first demonstrate the effectiveness of DAN2 and show that DAN2 improves box-office revenue forecasting accuracy by 32.8% over existing models. Subsequently, we offer an alternative modeling strategy by adding production budgets, pre-release advertising expenditures, runtime, and seasonality to the predictive variables. This alternative model produces excellent forecasting accuracy values of 94.1%. [ABSTRACT FROM AUTHOR]

This paper addresses problem of predicting direction of movement of stock and stock price index for Indian stock markets. The study compares four prediction models, Artificial Neural Network (ANN), Support Vector Machine (SVM), random forest and naive-Bayes with two approaches for input to these models. The first approach for input data involves computation of ten technical parameters using stock trading data (open, high, low & close prices) while the second approach focuses on representing these technical parameters as trend deterministic data. Accuracy of each of the prediction models for each of the two input approaches is evaluated. Evaluation is carried out on 10 years of historical data from 2003 to 2012 of two stocks namely Reliance Industries and Infosys Ltd. and two stock price indices CNX Nifty and S&P Bombay Stock Exchange (BSE) Sensex. The experimental results suggest that for the first approach of input data where ten technical parameters are represented as continuous values, random forest outperforms other three prediction models on overall performance. Experimental results also show that the performance of all the prediction models improve when these technical parameters are represented as trend deterministic data. [ABSTRACT FROM AUTHOR]

Time series prediction has been widely used in a variety of applications in science, engineering, finance, etc. There are two different modeling options for constructing forecasting models in time series prediction. Global modeling constructs a model which is independent from user queries. On the contrary, local modeling constructs a local model for each different query from the user. In this paper, we propose a local modeling strategy and investigate the effectiveness of incorporating local modeling with three popular machine learning based forecasting methods, Neural Network (NN), Adaptive Neuro-Fuzzy Inference System (ANFIS), and Least Squares Support Vector Machine (LS-SVM), for time series prediction. Given a series of historical data, a local context of the user query is located and an appropriate number of lags are selected. Then forecasting models are constructed by applying NN, ANFIS, and LS-SVM, respectively. A number of experiments are conducted and the results show that local modeling can enhance the estimation performance of a forecasting method for time series prediction. [ABSTRACT FROM AUTHOR]

• Simple, efficient and low cost accident detection system for PTW. • DSS takes very less time to detect the accident with severity class. • Irrespective of type of rider and vehicle design, the designed system will work well. • Novel accident detection algorithm is devised using ASW technique. • The system could discriminate the real fall state from false-positive cases. The two-wheeler accidents in most populated and developing countries have become vulnerable and six accidents happen every hour on average. This paper proposes an efficient automatic accident detection system that attempts to detect the occurrences of the accidents in powered two-wheelers (PTW) automatically using vehicle-dependent parameters and the physiological parameters of the rider in real-time. The proposed system builds an accident detection system in PTW using three steps namely, critical event detection system, accident detection system, and severity assessment system. The critical event detection system reads the accelerometer sensor values from the On-Board Diagnostic (OBD) unit mounted on the PTW and classifies the state of the vehicle as normal, fall-like, and fall through the enhanced decision tree algorithm. The enhanced decision tree algorithm uses a tanh function to calculate entropy values. The rules are extracted to fix the threshold by pruning the decision tree to identify the fall of the vehicle and the rider. Due to the unstable nature of PTW and the rider, a novel Adaptive Sequence Window algorithm (ASW) is proposed to substantiate and validate the occurrence of accidents based on the sequence of states identified. Once the accident is detected, the Decision Support System (DSS) running on the OBD mounted on the PTW decides the severity of the accident by combining the three parameters namely fall of the vehicle, fall of the rider, and pulse rate of the rider using the first-order predicate logic rules. The enhanced decision tree algorithm outperforms the other classifiers such as naïve Bayes, artificial neural network, and recurrent neural network with an accuracy of 99.8%. The OBD unit mounted on the PTW and the rider's helmet is used to detect the occurrence of accidents automatically along with its severity with less time. The ASW algorithm enables the system to detect the fall of the vehicle and rider within five minutes and prevents false positives. Further, the information can be communicated based on the severity of the accident to the emergency medical service for quick response. [ABSTRACT FROM AUTHOR]

Deep learning can play a pivotal role in early fault detection in squirrel cage induction motors (SCIMs) and achieving Industry 4.0. SCIM finds application in industries like mining, textile, manufacturing, and many more. Early fault detection in SCIM can significantly reduce downtime and optimize productivity. This paper proposes a novel fault detection technique for bearing faults and broken rotor bar detection in SCIM using the dilated convolutional neural network-based model. A simple 1-D signal to image conversion technique is also proposed for transforming the 1-D vibration signal acquired from multiple accelerometers to images. The proposed method provides an end-to-end learning solution for fault detection. The propounded approach has accomplished an average accuracy of more than 99.50%. A comparison has also been made between different convolutional neural network (CNN) models and conventional machine learning models to show the proposed method's efficiency. The complete experimental work has been carried out on a 5 kW, 3-phase, 415 V, 50 Hz SCIM. The dilated CNN model development has been done using python software, and the packages used are Keras and TensorFlow. • Dilated convolutional neural network-based model is used for fault detection. • Developed model expedites high accuracy with less number of layers. • Proposed model provides faster convergence owing to dilated convolutions. • Model facilitates self and automatic fault feature learning. [ABSTRACT FROM AUTHOR]

• Thorough analysis of wearable Fall Detection Systems (FDS) based on neural networks. • Existing research has only focused on optimizing the recognition metrics. • The study reveals significant methodological flaws in the evaluation of FDSs. • Important implementability issues are normally neglected by the literature. • Operational and social aspects are also ignored by the related literature. As a result of the challenge posed by demographic aging and the important personal and medical costs associated to falls of elderly people, the study of automatic Fall Detection Systems (FDS) has gained significant research attention during the last decade. Due to the intrinsic difficulty of identifying falls with a deterministic or a simple threshold-based method, Artificial Neural Networks (ANNs) and other machine learning strategies have been largely employed by the literature as the detection algorithms. This paper thoroughly revises and analyzes those papers in the literature that have proposed a FDS that is partly or entirely based on the use of an ANN. The study, which is focused on wearable devices (as they normally impose higher battery and computing restrictions), discusses the most recent trends in the application of neural architectures to the problem of automatic detection. In this regard, the analysis reveals that the related literature has normally emphasized the maximization of the detection performance, while disregarding some methodological, operative implementation-related issues of utmost importance for the assessment of the actual feasibility of these applications. In particular, the investigation shows that cross validating the detection methods with public benchmarking datasets is gaining importance in recent works. Conversely, the practical problems related to the use of wearables (consumption of power and computing resources, configuration and limitations of the sensing units and the wireless transmissions, etc.) are still normally disregarded. [ABSTRACT FROM AUTHOR]

• Deep learning algorithm for identification of type and flight mode of detected drones. • Novel deep learning algorithm for detection of multiple drones in radio frequency domain. • Results of three indoor experiments concerning collecting and analyzing radio signals from drones. • Public available database with radio signals of drones from two different radio frequency bands. Unmanned aerial systems, especially drones have gone through remarkable improvement and expansion in recent years. Drones have been widely utilized in many applications and scenarios, due to their low price and ease of use. However, in some applications drones can pose a malicious threat. To diminish risks to public security and personal privacy, it is necessary to deploy an effective and affordable anti-drone system in sensitive areas to detect, localize, identify, and defend against intruding malicious drones. This research article presents a new publicly available radio frequency drone dataset and investigates detection and identification methodologies to detect single or multiple drones and identify a single detected drone's type. Moreover, special attention in this paper has been underlined to examine the possibility of using deep learning algorithms, particularly fully connected deep neural networks as an anti-drone solution within two different radio frequency bands. We proposed a supervised deep learning algorithm with fully-connected deep neural network models that use raw drone signals rather than features. Regarding the research results, the proposed algorithm shows a lot of potentials. The probability of detecting a single drone is 99.8%, and the probability of type identification is 96.1%. Moreover, the results of multiple drones detection demonstrate an average accuracy of 97.3%. There have not been such comprehensive publications, to this time, in the open literature that have presented and enlightened the problem of multiple drones detection in the radio frequency domain. [ABSTRACT FROM AUTHOR]