Search

Abstract: This article explores the application of image processing techniques in recyclable waste paper sorting. In recycling, waste papers are segregated into various grades as they are subjected to different recycling processes. Highly sorted paper streams facilitate high quality end products and save processing chemicals and energy. From 1932 to 2009, different mechanical and optical paper sorting methods have been developed to fill the paper sorting demand. Still, in many countries including Malaysia, waste papers are sorted into different grades using a manual sorting system. Because of inadequate throughput and some major drawbacks of mechanical paper sorting systems, the popularity of optical paper sorting systems has increased. Automated paper sorting systems offer significant advantages over human inspection in terms of worker fatigue, throughput, speed, and accuracy. This research attempts to develop a smart vision sensing system that is able to separate the different grades of paper using first-order features. To construct a template database, a statistical approach with intra-class and inter-class variation techniques are applied to the feature selection process. Finally, the K-nearest neighbor (KNN) algorithm is applied for paper object grade identification. The remarkable achievement obtained with the method is the accurate identification and dynamic sorting of all grades of papers using simple image processing techniques. [Copyright &y& Elsevier]

Nowadays, there has been a rapidly increasing number of scientific submissions in multiple research domains. A large number of journals have various acceptance rates, impact factors, and rankings in different publishers. It becomes time-consuming for many researchers to select the most suitable journal to submit their work with the highest acceptance rate. A paper submission recommendation system is more critical for the research community and publishers as it gives scientists another support to complete their submission conveniently. This paper investigates the submission recommendation system for two main research topics: computer science and applied mathematics. Unlike the previous works (Wang et al., 2018; Son et al., 2020) that extract TF–IDF and statistical features as well as utilize numerous machine learning algorithms (logistics regression and multiple perceptrons) for building the recommendation engine, we present an efficient paper submission recommendation algorithm by using different bidirectional transformer encoders and the Mixture of Transformer Encoders technique. We compare the performance between our methodology and other approaches by one dataset from Wang et al. (2018) with 14012 papers in computer science and another dataset collected by us with 223,782 articles in 178 Springer applied mathematics journals in terms of top K accuracy (K = 1 , 3 , 5 , 10). The experimental results show that our proposed method extensively outperforms other state-of-the-art techniques with a significant margin in all top K accuracy for both two datasets. We publish all datasets collected and our implementation codes for further references. 1 1 https://github.com/BinhMisfit/PSRMTE. • Bidirectional transformer encoders can improve the performance of the paper submission recommendation system. • The Mixture of Transformer Encoders framework shows the efficiency in the paper submission recommendation problem. • Proposed techniques can surpass other recent techniques on two datasets related. [ABSTRACT FROM AUTHOR]

The emergence of online social networks (OSN) platforms removes communication barriers that are essential to human life, catalyzing social networking growth. However, this emergence has given rise to a negative impact when someone abuses the platform to commit cyberbullying activities. Hence, it is crucial to work on automated cyberbullying-related classification to mitigate the societal phenomena in OSN. The research on the automated classification model for cyberbullying was pioneered over the last decade with growing interest among researchers. It is helpful to track its growth over the decades to elucidate the state-of-arts techniques applied in this field. This paper presents a large amount of literature germane to cyberbullying classification from past to present to provide a comprehensive review. A total of 126 papers were reviewed. This paper emphasizes text-based cyberbullying and multi-modal cyberbullying. The review was presented around the machine learning workflow, encompassing four core sections: dataset analysis, pre-processing analysis, feature analysis, and technique analysis. Based on the critical analysis, limitations are addressed along with the future works that can be conducted to fill the gap in previous research. Furthermore, the review also examined the ethical implications associated with the implementation of these techniques. This review paper is expected to assist readers in fully comprehending the current trend, architecture, and techniques applied to the field. [ABSTRACT FROM AUTHOR]

Truck drivers are essential to modern supply chains. However, the companies that employ them are having trouble keeping them. At many trucking firms, annual turnover rates approach 100 percent or higher. This paper develops a method for predicting individual truck driver turnover events before they happen by applying supervised machine learning classifiers to a new source of operational truck driver data. Our paper uses Electronic Logging Device (ELD) data, which comprise newly federally-mandated, time-stamped work logs collected from approximately 1200 American long-haul truck drivers over 3 years. We train three supervised machine learning classifiers (logistic regression, random forests, and support vector machines) on this data and achieve 60 to 70 percent prediction accuracy and 50 to 60 percent recall across two 5-fold cross-validated experiments. We observe that the quantity and consistency of week-day driving assignments explain most of our models' predictive power. We offer these results as both a new technical tool, as well as novel managerial insights for improving global truck driver retention problems for the benefit of supply chains worldwide. [Display omitted] • Newly available operational data from long haul truck drivers is analyzed. • Three machine learning classifiers are trained on the operational data. • Predictive models achieve 60–70 percent accuracy, 50–60 percent recall. • Models show potential to enable interventions to retain truck drivers. [ABSTRACT FROM AUTHOR]

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

Multiple Instance Learning (MIL) is a learning paradigm, where training instances are arranged in sets, called bags, and only bag-level labels are available during training. This learning paradigm has been successfully applied in various real-world scenarios, including medical image analysis, object detection, image classification, drug activity prediction, and many others. This survey paper presents a comprehensive analysis of MIL, highlighting its significance, recent advancements, methodologies, applications, and evolving trends across diverse domains. The survey begins by explaining the core principles that form the basis of MIL and how it differs from traditional learning approaches. This sets the foundation for comprehending the distinct challenges and techniques of solving MIL problems. Next, we discuss how supervised learning algorithms are tailored to support MIL and combine this discussion with a review of seminal MIL algorithms as well as the latest innovations that incorporate neural networks, deep learning architectures, and attention techniques. This comprehensive analysis helps to understand the strengths, limitations, and adaptability of these methods across diverse data modalities, complexities, and applications. In summary, this survey paper provides an essential resource for researchers, practitioners, and enthusiasts seeking a comprehensive understanding of Multiple Instance Learning. It covers foundational concepts, traditional methods, recent advancements, and future directions. By providing a holistic view of MIL's dynamic landscape, this paper aims to inspire further innovation and exploration in this ever-evolving field. • A survey on the current state of the Multiple Instance Learning. • We provide applications of Multiple instance learning in various domains. • We discuss how existing supervised learning algorithms are modified to support MIL. • Publicly available datasets and open research challenges in MIL are discussed. [ABSTRACT FROM AUTHOR]

This paper proposes physics-informed meta-learning-based surrogate modeling (PI-MLSM), a novel approach that combines meta-learning and physics-informed deep learning to train surrogate models with limited labeled data. PI-MLSM consists of two stages: meta-learning and physics-informed task adaptation. The proposed approach is demonstrated to outperform other methods in four numerical examples while reducing errors in prediction and reliability analysis, exhibiting robustness, and requiring less labeled data during optimization. Moreover, compared to other approaches, the proposed approach exhibits better performance in solving out-of-distribution tasks. Although this paper acknowledges certain limitations and challenges, such as the subjective nature of physical information, it highlights the key contributions of PI-MLSM, including its effectiveness in solving a wide range of tasks and its ability in handling situations wherein physical laws are not explicitly known. Overall, PI-MLSM demonstrates potential as a powerful and versatile approach for surrogate modeling. • A proposed approach uses meta-learning and PIDL to train surrogates with small data. • A model-agnostic meta-learning model is introduced to learn surrogate model weights. • A PIDL approach is introduced to adapt to target tasks with meta-learned weights. • Our approach outperformed other methods in numerical examples given limited data. • Proposed approach showed robustness in solving out-of-distribution tasks. [ABSTRACT FROM AUTHOR]

Automatic humor detection has compelling use cases in modern technologies, such as humanoid robots, chatbots, and virtual assistants. In this paper, we propose a novel approach for detecting and rating humor in short texts based on a popular linguistic theory of humor. The proposed technical method initiates by separating sentences of the given text and utilizing the BERT model to generate embeddings for each one. The embeddings are fed to a neural network as parallel lines of hidden layers in order to determine the congruity and other latent relationships between the sentences, and eventually, predict humor in the text. We accompany the paper with a novel dataset consisting of 200,000 short texts, labeled for the binary task of humor detection. In addition to evaluating our work on the novel dataset, we participated in a live machine-learning competition to rate humor in Spanish tweets. The proposed model obtained F1 scores of 0.982 and 0.869 in the performed experiments which outperform general and state-of-the-art models. The evaluation results confirm the model's strength and robustness and suggest two important factors in achieving high accuracy in the current task: (1) usage of sentence embeddings and (2) utilizing the linguistic structure of humor in designing the proposed model. • A novel method for humor detection and rating based on a general linguistic theory of humor. • Introduced a very large novel dataset with 200k short texts for humor detection. • Achieved 98% accuracy and outperformed five strong baselines on the new dataset. • Demonstrated accuracy and robustness in a data science competition for Spanish texts. • The novel approach and dataset contributed to tens of research projects. [ABSTRACT FROM AUTHOR]

• Treat a problem of the imbalanced customer churn classification. • A new multi-strategy collaborative processing method IADASYN-FLCatBoost is proposed. • The traditional ADASYN algorithm is improved and an IADASYN algorithm is proposed. • The Focal Loss function is embedded into the CatBoost to form a new FLCatBoost. • Empirical analysis shows the proposed method performs better than similar methods. The rapid advancement of big data and artificial intelligence heralds a dual-edged era of opportunities and challenges for the banking sector. Indeed, enhancing a model's capability to accurately classify imbalanced datasets represents a critical challenge within the field of customer churn prediction (CCP). In this paper, to address the challenges presented by the problem of imbalanced customer classification, a new multi-strategy collaborative processing method named IADASYN-FLCatBoost is proposed from dual perspectives: data and algorithm. At the data level, the traditional Adaptive Synthetic (ADASYN) sampling is improved, that is, the LOF (Local Outlier Factor) algorithm is introduced to eliminate outliers, and the classification features are specially processed to synthesize new minority class samples, thus an improved ADASYN (IADASYN) algorithm is obtained. At the algorithm level, the Focal Loss is embedded into the CatBoost ensemble learning framework to form a new Focal Loss-CatBoost (FLCatBoost) to make a focal-aware, cost-sensitive version of imbalanced customer churn prediction. Moreover, the empirical analysis is conducted in conjunction with the credit card customer dataset obtained from the Kaggle platform. The results of the staged comparison experiments show that the proposed method IADASYN-FLCatBoost in this paper shows the best prediction performance. Comparing the proposed method with 5 other imbalanced classification algorithms and 20 classifiers composed of classical sampling methods and ensemble learning algorithms, it is verified that the classification effect of the proposed method performs best, and the values of Recall , F 1 score, G-mean and Area under Precision-Recall curve (AUPRC) have been significantly improved. In addition, further verification of the model also proves that the proposed method has certain generalizability and is still valid for other banks and customer churn datasets of other industries. [ABSTRACT FROM AUTHOR]

• Newbie study using standard ML techniques with HAR Application and Discussions. • Activities found in Literature with the corresponding reference. • Co-occurrences between activities and sensors with the corresponding reference. • Summary and comparison among the different datasets found in Literature. • Summary of the experimentation settings with performance scores found in Literature. Human Activity Recognition (HAR) is an essential area of research related to the ability of smartphones to retrieve information through embedded sensors and recognize the activity that humans are performing. Researchers have recognized people's activities by processing the data received from the sensors with Machine Learning Models. This work is intended to be a hands-on survey with practical's tables capable of guiding the reader through the sensors used in modern smartphones and highly cited developed machine learning models that perform human activity recognition. Several papers in the literature have been studied, paying attention to the preprocessing, feature extraction, feature selection, and classification techniques of the HAR system. In addition, several summary tables illustrating HAR approaches have been provided: most popular human activities in the literature with paper references, the most popular datasets available for download (Analyzing their characteristics, such as the number of subjects involved, the activities recorded, and the sensors with online-availability), co-occurrences between activities and sensors, and a summary table showing the performance obtained by researchers. =The paper's goal is to recommend, through the discussion phase and thanks to the tables, the current state of the art on this topic. [ABSTRACT FROM AUTHOR]

Roundwood sorting is still a manual process in many Swiss sawmills, requiring employees to visually inspect and categorize thousands of logs per day. The heavy workload can be both physically and mentally taxing and can lead to increased rates of human error. State-of-the-art automation systems like X-ray log scanners are expensive and difficult to integrate into existing process lines. This paper proposes a novel recommendation system that leverages recent advances in image classification to automate roundwood classification by quality and species. The system integrates a camera to capture cross-sectional images of logs and record numerical data, such as length, taper, and diameter. The analysis of the resulting dataset highlights the challenges of data imbalance and noise, which makes classification difficult and, in some cases, impossible. However, by using selected datasets with reduced noise, state-of-the-art Convolutional Neural Networks (CNNs) can extract quality and species features. Quality models learn from a manually selected and simplified dataset, featuring samples that experts can clearly classify based on the image's information. Species models are trained on a label-noise-reduced dataset, reflecting real-world complexity. The accuracy on the selected dataset for three quality classes is 80%. The species determination is less challenging and reaches 91% accuracy on a synchronized dataset for the main species spruce and fir. Overall, this paper highlights the potential of Machine Learning in augmenting the roundwood sorting processes and presents a novel system that can improve the efficiency and accuracy of the process. [Display omitted] • Automation of roundwood sorting: Replaces manual sorting with image-based AI. • Integrated camera system in roundwood sorting to collect labeled dataset. • Species prediction: 91% accuracy in spruce–fir distinction. • Quality prediction on complexity reduced dataset: 80% accuracy between three main quality levels. • Efficient, adaptable & scalable system which is easy to integrate into existing process lines. [ABSTRACT FROM AUTHOR]

In this paper the performance of different Machine Learning and Deep Learning approaches is evaluated in problems related to green mobility in big cities. Specifically, the forecasting of bike sharing demand in Madrid and Barcelona (Spain) is approached, for different prediction time-horizons, and also a problem of cable car demand forecasting in Madrid city. An important number of predictive variables are considered, which are grouped into four different sets (categorical/calendrical, persistence-based, meteorological and, as a novelty of the paper, information about analogue past instances), whose relevance is studied for all cases. A feature selection mechanism is also incorporated in order to improve the prediction accuracy of the proposed algorithms. A total of 12 different multivariate regression techniques are implemented, covering from Machine Learning methods to time-series Deep Learning approaches. Excellent results in all the prediction problems approached are reported. Finally, the consequences of obtaining accurate prediction in these three problem of green mobility in big cities are discussed. In addition, it is studied how the results could be exported to other similar cases in more general urban mobility studies. Novelties of the work include: (1) Addressing the forecast problem of passenger flow on a cable car using ML and DL multivariate techniques; (2) using the demand of analogous past instances as an additional feature to solve the demand prediction problems; and (3) the extraction of global conclusions about feature relevance when addressing a demand forecasting problem in green mobility. [ABSTRACT FROM AUTHOR]

The paper presents a machine learning based Smart Intrusion and Fault Identification (SIFI) method to identify the cyber-physical abnormalities in an inverter-based cyber-physical microgrid (CPM). The SIFI method utilizes an ensemble classifier (EC) to assimilate the decisions from three distinct classifiers (C4.5 decision tree, random forest (RF), and forest by penalizing attributes (FPA)). The proposed method employs a voting mechanism to classify and localize abnormal events for improved accuracy. The paper investigates the effects of cyberattacks (Denial-of-Service (DoS) and Malicious Data Injection (MDI) attacks) and physical abnormalities caused by line faults in microgrids. To train the classifiers, SIFI employs dataset with statistical attributes extracted from measurements. The renewable alternative power systems simulation (RAPSim) software tool is utilized to model the proposed system. The effectiveness of the presented model is assessed with respect to mean value error (MVE). The efficiency of the classifier is demonstrated by comparing its performance with other classifiers in terms of the MVE. For physical abnormalities identification and localization, the approach provides enhanced outputs with the MVE of 0.157 % and 0.162 % correspondingly. For identifying MDI anomalies, the model provides better results with a lower error rate. For DoS anomaly detection, the model provides better classification performance with 0.136 % error. The extensive empirical analysis proves that the anticipated ensemble classifier-based SIFI model yields the minimum MVE for identifying physical faults and cyberattacks. Furthermore, the effectiveness of the proposed method is identified by considering a case study in which the SIFI method outperforms well for the MDI and DoS cyber-attacks in the CPM. [ABSTRACT FROM AUTHOR]

In recent years, the application of the internet of things (IoT) in areas such as intelligent transportation, smart cities, and the industrial internet has become increasingly widespread. As a crucial supporting infrastructure, IoT devices are utilized in various fields to construct IoT networks. However, due to the inherent limitations of IoT devices, such as limited computing resources and low memory capacity, security concerns have become increasingly prominent. Among these concerns are Denial-of-Service (DoS) and botnet attacks, which are difficult to prevent due to their large-scale and covert nature. To address these challenges, this paper proposes a Hybrid DoS Attack Intrusion Detection System (HDA-IDS) that combines signature-based detection with anomaly-based detection to effectively identify both known and unknown DoS/botnet attacks. Additionally, this paper introduces a novel anomaly-based detection model called CL-GAN. It integrates CNN-LSTM with GAN to establish a baseline for normal behavior and detect malicious traffic. In contrast to other semi-supervised models, the CL-GAN exhibits superior accuracy, as well as shorter training and testing times, in detecting DoS and botnet attacks. In addition, experimental results demonstrate that the HDA-IDS outperforms other IDSs in detecting DoS and botnet attacks. When tested on datasets such as NSL-KDD, CICIDS2018, and Bot-IoT, the HDA-IDS achieved an average of 5% overall improvement superior performance in terms of accuracy, precision, recall, and F1-Score compared to other works. These results highlight the effectiveness of the proposed system in addressing security issues in IoT networks, and presents a general framework that addresses the challenge of large-scale attacks constructed through the dissemination of false information. [ABSTRACT FROM AUTHOR]

In the era of rapid paper publications in various venues, automatic citation recommendations would be highly useful to researchers when they write papers. Local citation recommendation aims to recommend possible papers to cite given local citation contexts. Previous work mainly computes the similarity score between citation contexts and cited papers on a one-to-one basis, which is quite time-consuming. We train a pair of neural network encoders that map citation contexts and all possible cited papers to the same vector space, respectively. After that, we index the positions of all cited papers in the vector space. This makes our process for searching recommended papers considerably faster. On the other hand, existing methods tend to recommend papers that are highly similar to each other, which makes recommendations lack diversity. Therefore, we extend our algorithm to perform multi-topic recommendations. We generate multi-topic training examples based on the index we mentioned earlier. Furthermore, we specially design a multi-group contrastive learning method to train our model so that it can distinguish different topics. Empirical experiments show that our model outperforms previous methods by a wide margin. Our model is also light weighted and has been deployed online so that researchers can use it to obtain recommended citations for their own paper in real-time. • Proposed FLCR algorithm with sentence-transformer & k-d tree. • Introduced multi-topic citation recommendation for diverse contexts. • Developed large-scale dataset with 1.7 million citation contexts for evaluation. • Demonstrated significant performance improvement over previous methods. • Deployed demo for real-time citation recommendations. [ABSTRACT FROM AUTHOR]

Constructing a deformation prediction model for dams that can accurately capture deformation trends is crucial to ensure their operational safety. The accurate monitoring data are critical to the presentation of the variation patterns of the monitored quantities. However, due to various disturbances, there can be a lot of noise in the raw deformation monitoring data acquired by the sensors. Therefore, the reduction of the noise level in the data sequence is necessary for the improvement of the accuracy and reliability of the prediction model. This paper presents data denoising techniques for the removal of outliers from data sequences, including complete ensemble empirical mode decomposition with adaptive noise, garrote threshold functions, and sample entropy. The ensemble learning algorithm was then employed to complete the deformation prediction of concrete dams. Through example analysis and comparison with a benchmark model, the proposed model's performance was validated, highlighting its effectiveness and superiority. Additionally, this paper enhances the model's interpretability by visualizing the optimization process. The degree of influence of the input factors on the deformation prediction results is also analyzed using the feature importance metric. The proposed model is expected to be applied to deformation monitoring of other civil and hydraulic structures, as it can effectively reduce the impact of noise on prediction. [ABSTRACT FROM AUTHOR]

Cardiovascular diseases are currently the major causes of death globally. Among the strategies to prevent cardiovascular issues, the automated classification of heart sound abnormalities is an efficient way to detect early signs of cardiac conditions leading to heart failure or other, even asymptomatic, complications, quite effective for timely interventions. Despite the significant improvements in this field, there are still limitations due to the lack of solutions, available data-sets and poor (mainly binary — normal vs abnormal) classification models and algorithms. This paper presents a Medical Cyber–Physical System (MCPS) for the automatic classification of heart valve diseases onsite, in a timely manner. The proposed MCPS, indeed, can be deployed into personal and mobile devices, addressing the limitations of existing solutions for patients, healthcare practitioners, and researchers, through an efficient and easy accessible tool. It combines different neural network models trained on a new Italian dataset of 132 adult patients covering 9 heart sound categories (1 normal and 8 abnormal), also validated against two main open-access (Physionet/CinC Challenge 2016 and Korean) datasets. The overall MCPS performance (time, processing and energy resource utilization) and the high accuracy of the models (up to 98%) demonstrated the feasibility of the proposed solution, even with few data. The dataset supporting the findings of this paper is available upon request to the authors. • DB of 132 heart sounds of Italian patients in 9 categories, available upon request. • MCPS for automated HVD classification via hierarchical ML model structures. • Compared six ML classifiers (RF, SVM, NB, LR, DNN, LSTM) for HVD and majority voting. • MCPS validated on two open access datasets (Physionet/CinC 2016 and Korean). • Time, processing and energy efficiency of MCPS validated, with usage guidelines. [ABSTRACT FROM AUTHOR]

Data-driven models have been recognized as powerful tools to support Structural Health Monitoring (SHM). This paper contributes to the literature by exploring two data-driven approaches to detect damage through changes in a set of variables that assess the condition of the structure, and accommodates the challenge that may arise due to the influence of environmental and operational variabilities. This influence is reflected in the response of the structure and can reduce the probability of detecting damage in a structure or increase the probability of signaling false positives. This paper conducts a comparative study between a machine learning detection approach (supported by linear regression, random forest, support vector machine, and neural networks) and a cointegration approach, with the aim of detecting damage as early as possible. This study also contributes to the literature by evaluating the merits of the damage detection methods using real data collected from a small-scale structure. The structure is analyzed in a reference state and a perturbed state in which damage is emulated. The results show that both approaches are able to detect damage within the first 24 h, without ever signaling false positives. The cointegration based approach can notably detect damage after 10 h and 15 minutes, while the machine learning approach takes 20 h 30 m to detect damage. • Comparison of Machine Learning and Cointegration in SHM. • Consideration of damages and false alarms. • Usage of a reduced scale structure to reproduce the behavior of a bridge column. • Cointegration is the fastest to detect damage. [ABSTRACT FROM AUTHOR]

Citation counts is a crucial factor in evaluating the quality of research papers. Therefore, it is vital to accurately predict citation counts and explore the mechanisms underlying citations. In this study, we focus on predicting the citation counts in the field of statistics. We collect 55,024 academic papers published in 43 statistics journals between 2001 and 2018. Furthermore, we collect and clean a high-quality dataset and then construct multi-layer networks from different perspectives, including journal networks, author citation networks, co-citation networks, co-authorship networks, and keyword co-occurrence networks. Additionally, we extract 77 factors for citation counts prediction, including 22 traditional and 55 network-related factors. To address the issues of zero-inflated and over-dispersed citation counts, a neural network model is designed to achieve high prediction accuracy. Furthermore, we adopt a leave-one-feature-out approach to investigate the importance of these factors. The proposed neural network model achieves an MAE value of 7.352, which outperforms other machine learning models in the comparison. Thus, this study provides a useful guide for researchers to predict citation counts and can be easily extended to other research fields. • Extracting factors based on multi-layer academic networks. • Extracting research topics using keyword co-occurrence network. • Using the neural network model to predict the citation counts. • Indicating network-related factors possess strong predictive. performance. • Measuring factor importance using the leave-one-feature-out method. [ABSTRACT FROM AUTHOR]

Intelligent Fault-Tolerant Control (IFTC) refers to the applications of machine learning algorithms for fault diagnosis and design of Fault-Tolerant Control (FTC). The overall goal of the FTC is to accommodate defects in the system components while they are in use and maintain stability with little to no performance reduction. These systems are crucial for mission-critical and safety-related applications where the safety of people is at stake and service continuity is crucial. In this review paper, a systematic study has been done for the development of FTC with machine learning, deep learning, and transfer learning algorithms. The challenges and limitations faced with their possible solutions through machine learning theories for the IFTC model are lined up. This paper guides researchers on the different possible types of machine learning algorithms and their advanced forms like deep learning and transfer learning. The differences among these are highlighted by the challenges and limitations of each. The paper is significant such that most of the important literature references from the Scopus database particularly related to important electrical and mechanical industrial problems have been discussed to guide the researchers who want to apply IFTC for specific industrial problems, being the research gap. Finally, future research directions for the development of IFTC are highlighted. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

Flight diversions due to adverse weather have a significant negative impact on airspace users, airports, and the environment. Previous work proposed a tree-based model that learned which flights are more likely to be diverted due to adverse weather conditions at the destination airport from historical traffic and weather data. This model was trained using confident learning to exclude from training unpredictable flight diversions caused by reasons other than weather, such as medical emergencies or technical problems. Aggregated performance metrics computed on the test set demonstrated high precision and moderate recall, and inspection of the Shapley values unravelled the most important features. Initial human observations made during the live trial, however, have raised a potential connection between the quality of predictions and the specific weather conditions that lead to flight diversions. This study aims to provide empirical evidence to support this hypothesis. To accomplish this, a data-driven methodology should be used to characterise the main conditions that cause flight diversions. Concurrently, the effectiveness of the model in each of these conditions should be evaluated. The dataset under consideration, however, presents a challenge because it contains a plethora of high-cardinality categorical features (e.g., destination airport and aircraft operator) as well as skewed numerical features with a wide range of scales (e.g., visibility and wind gust). This scenario significantly increases the difficulty of characterising the main conditions that cause flight diversions and conducting subsequent analyses even with established methods like factor analysis of mixed data. To address this challenge, this paper proposes to utilise supervised clustering, a method that combines state-of-the-art feature attribution, dimensionality reduction, and clustering algorithms to identify the most representative decisions of a model. Following that, the distinctive characteristics of the various decisions (e.g., extreme obscuration at the airport), each associated with a cluster, are determined by examining them. Finally, conventional classification and uncertainty metrics are computed for each cluster to assess the performance of the model. This paper shows that supervised clustering is effective in characterising flight diversions due to weather, and provides valuable insights into the performance of the model in different conditions, highlighting situations where predictions require careful consideration. • Flight diversions due to adverse weather conditions could be predicted with machine learning. • Supervised clustering effectively identifies the common weather conditions causing flight diversions. • Flight diversions in low visibility and windy conditions are highly predictable. [ABSTRACT FROM AUTHOR]

• Pinch force strength is accurately predicted using sEMG. • Different pinch movements are accurately classified using sEMG. • Similarity of EMG features is evaluated. • An EMG feature-to-image method is proposed. The movement of intelligent EMG-driven prosthesis mainly relies on the synergy of different fingers to achieve function of grasping objects. The paper proposes a novel scheme for force prediction and movement classification about pinch between different fingers based on surface electromyography (sEMG) using machine learning. The pinch force and sEMG signals are recorded synchronously by a data acquisition device. Eight features are extracted, which are proven to have better performance in the estimation of sEMG-to-force. We present a novel feature selection method that uses the one-dimensional time series similarity assessment based on Manhattan distance to eliminate the repetitive information between features. Three optimal features carrying less repetitive information are retained. Seven machine learning algorithms are used to predict force strength. The results show that the Long Short Term Memory (LSTM) has the best performance of force prediction, achieving a R 2 of 0.9517 and RMSE of 3.2723. The paper proposes a novel method of converting EMG feature sequences to the normalized gray image in order to classify the finger movement. Five classifiers based on image feature extraction and the Convolutional Neural Network (CNN) are developed respectively. The experimental results indicate that the CNN performs best, achieving an accuracy of 97.66%. In this way, it not only realizes the accurate force prediction, but also realizes the movement classification between different fingers. The proposed methodology has the potential to realize simultaneous force and movement control of prosthetic hand. [ABSTRACT FROM AUTHOR]

In this paper we present SurvLIMEpy, an open-source Python package that implements the SurvLIME algorithm. This method allows to compute local feature importance for machine learning algorithms designed for modelling Survival Analysis data. The presented implementation uses a matrix-wise formulation, which allows to speed up the execution time. Additionally, SurvLIMEpy assists the user with visualisation tools to better understand the result of the algorithm. The package supports a wide variety of survival models, from the Cox Proportional Hazards Model to deep learning models such as DeepHit or DeepSurv. Two types of experiments are presented in this paper. First, by means of simulated data, we study the ability of the algorithm to capture the importance of the features. Second, we use three open source survival datasets together with a set of survival algorithms in order to demonstrate how SurvLIMEpy behaves when applied to different models. • Python package implementing SurvLIME algorithm for computing feature importance. • It supports a wide variety of survival models, including deep learning models. • Fast and efficient implementation due to matrix-wise optimisation problems. • An open-source implementation available on GitHub. • Stable release provided to PyPI. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a distributed machine learning method to protect users' privacy and security. It currently faces the following two problems (McMahan et al., 2017): (1) The performance of the global model degrades when dealing with Non-Independent Identically Distributed (Non-IID) data. Other existing classical methods do not have a rigorous theory based on error processing ; (2) In the process of global model aggregation, the classical FL algorithm either directly averages local models or solely considers the proportion of local data to assign weights to the models, without accounting for the discrepancies between the local models. In this paper, a federated aggregation algorithm Federated Maximum Mean Discrepancy (FedMMD) is proposed to address the deviation between the local models and Non-IID. First of all, this paper utilizes the Dilated Convolution Meet Transformer (DCMT) model for local model feature extraction. This approach aims to capture more feature information and minimize the impact of Non-IID scenarios. Secondly, the learning stability of the local model data participating in the aggregation is compared with Maximum Mean Discrepancy (MMD). The weights of the local models involved in the aggregation are determined using the SKNQ (Student–Keuls–Newman-Q) method and the entropy weight method. The SKNQ method calculates and compares the MMD across multiple clients, while the entropy weight method assigns different weights to clients based on their deviations. On the standard dataset, the FedAvg algorithm (McMahan et al., 2017), the FedProx algorithm (Li et al., 2020) and the FedMMD algorithm are compared. The experimental results demonstrate that the FedMMD algorithm, used in training the global model, not only enhances the accuracy of learning IID and Non-IID data but also improves the generalization capability of the global model. • An DCMT model is proposed to alleviate the model fluctuations caused by NonIID data. • An MMD method is proposed to compared horizontally before the model is aggregated. • Entropy weight method and the SKNQ method are combined to allocate local models' weight when they are fused into global model. [ABSTRACT FROM AUTHOR]

With the wide application of the shield tunneling method in tunnel engineering, the untimely and incorrect attitude control of shield systems has become an essential factor affecting the quality of shield tunneling. The use of multi-agent deep reinforcement learning (MADRL) algorithm to learn correct and timely shield attitude correction policies is beneficial for handling shield attitude control tasks which are complex and separable. However, the sensing data of the shield systems are complicated and widely distributed, which affects the learning efficiency of the agents. To resolve this problem, this paper proposes a MADRL framework based on state classification and assignment (SCA) for adaptive shield attitude control. SCA-MADRL uses a clustering algorithm to classify states and uses a planning model to make the agent's learning space a consistent state space so that each agent can learn efficiently in its own learning space. The state classification algorithm in the framework can improve the learning efficiency of the agent, and the proposed assignment algorithm can further enhance the learning effect when reusing the agent. This paper verifies the necessity and effectiveness of using task decomposition to implement MADRL for equipment control and provides a practical framework for automated and intelligent shield systems. [ABSTRACT FROM AUTHOR]

Federated learning (FL) has emerged as a promising privacy-preserving solution, which facilitates collaborative learning. However, FL is also vulnerable to poisoning attacks, as it has no control over the participant's behavior. Machine learning (ML) models are heavily trained for low generalization errors. Generative models learn the patterns in the input data to discover out-of-distribution samples, which can be used to poison the model, thereby degrading its performance. This paper proposes a novel approach to generate poisoned (adversarial) samples using hyperdimensional computing (HDC), projecting an input sample to a large HD space and perturbing it in the vicinity of the target class HDC model. This perturbation preserves the semantics of the original samples and adds hidden backdoor/noise into it. It generates a large set of adversarial samples equal to the HD space. It is observed that a trained ML model successfully misclassifies 60%–70% of the generated samples. The adversary uses these samples to frame data poisoning attacks, called hyperdimensional data poisoning attacks (HDPA). HDPA increases the attack impact by 5–10 × than existing poisoning attacks against the byzantine-robust defenses. Furthermore, we propose a hyperdimension-based confidence metric (HDBC) to check the model's conformance, which requires neither access to an ML model nor any additional calibration. • We frame a data poisoning attack in FL, called HDPA. • HDPA achieves 5–10× higher attack impact than existing attacks. • This paper presents an adversarial sample generation approach using the HDC model. • It does not even require the black-box access of the trained ML model. • A hyperdimension-based confidence metric is proposed to evaluate a model confidence. [ABSTRACT FROM AUTHOR]

Android has been a significant target of malware applications due to the exponential growth of mobile devices. This may result in severe threats to Android users such as financial loss, information leakage etc. The security of smartphones has encouraged the researchers in applying machine learning techniques to enhance the Android malware detection rate as the traditional approaches are not effective in distinguishing unknown malware. In recent years, many malicious apps have been detected on Android devices using machine learning based approaches. The problem with these detection systems is that they give a high false positive and false negative rate. Therefore, it is imperative to develop methods for better identifying and classifying malware. The main contribution of this paper is to improve the classification accuracy by fusing the base classifiers. This paper proposes a fusion approach named as MalDetect that enables an effective combination of machine learning techniques to improve the identification rate of malware. The proposed approach is based on the base classifiers and a set of ranking algorithms defined on their predictive error rate. The purpose of this method is to improve the detection rate of Android malware. Two datasets i.e. Drebin and AndroMD are used to illustrate the effectiveness of the proposed approach. Experimental results show that the proposed approach performs better than the individual base classifiers and traditional ensemble learning techniques for identifying Android malware. Further, the proposed approach is compared with the renowned Stacking ensemble technique that uses a meta -classifier. From the experimental outcomes, it is found that the weighted F-measure obtained from the Stacking for Drebin and AndroMD datasets is 0.9811 and 0.9775 respectively whereas the weighted F-measure acquired from the proposed approach is 0.9857 and 0.9790 respectively. [ABSTRACT FROM AUTHOR]

Predictions of the oil price are critical to support intelligent decision-making for individual investors, governments, and corporations, but a challenging task since there are complex nonlinear and random fluctuations involved in series. Additionally, most existing methods mainly focus on the data-layer fitting, and inherit uninterpretable design flowcharts and unclear layer functions. Based on the fact that living things evolve numerous advanced cognition processes after long-term interactions with environments, it is a promising problem to design bionic deep prediction networks. In this paper, an a uditory-inspired m ulti- f eature c ollaboration network (AMFC-Net) is explored to predict the oil price, and includes the feature extraction, brain memory, and comprehensive prediction blocks. Specifically, through imitating that complex sounds are captured by auditory canals and converted into electrical impulses, the first block adopts four typical activation functions to extract multi-group complementary features for detecting nonlinear changes. Through imitating that electrical impulses are transformed to the left and right hemispheres for handling and analyzing, the second block realizes gating and cooperation mechanisms to learn long short-term dependencies and highlight core information. Through imitating that information is sent to higher cerebral cortices for sensing the environment, the third block maps relationships from features to targets for producing final predictions. In summary, multi-channel convolution operations establish the imitations of both form (clear functional layer) and spirit (layer-by-layer collaboration), which not only ensure the prediction effectiveness but also improve the AMFC-Net interpretability. Three experiments (comparison with 12 machine learning methods), three experiments (comparison with 8 hybrid methods under decomposition plus ensemble framework), and seven discussions under two real-world oil price datasets all indicate that the proposed AMFC-Net is superior and suitable to predict the oil price. [ABSTRACT FROM AUTHOR]

• A scheme to recognize the distraction level was proposed. • MLP-AutoEncoder and DPC algorithms were used to define distraction-level labels. • Factors affecting driver distraction level were explored with mixed-model analyses. • A distraction-level prediction model with stacking ensemble learning was made. Driver distraction-level recognition while performing secondary tasks in full-touch in-vehicle information systems (FTIVISs) is essential for the harmonious co-driving of human and intelligent vehicle systems. However, there has been little research on this topic. To respond to this issue, this paper proposes a distraction-level recognition framework with a combination of semi-supervised learning, unsupervised learning, and supervised learning. First, unsupervised learning and semi-supervised is used to divide the collected unlabeled samples of driving distraction behavior into three categories of distraction levels: high, medium, and low. Second, the factors influencing the distraction level are explored through a mixed model analysis. Finally, a stacking-based ensemble learning model is proposed to recognize the driver distraction level by supervised learning, with the influencing factors of the distraction level used as model input parameters. To improve the recognition performance of the stacking model, four heterogeneous classifiers selected as the base classifiers, and principal component analysis (PCA) is introduced in the base classifier layer, which improves the ability of the meta -classifier to process raw features based on overfitting resistance. We conducted a real road experiment under different road and FTIVIS task conditions, and the proposed model performed better than traditional machine learning models. In addition, the model exhibited the greatest advantage when the deep neural network (DNN) algorithm was used as the meta -classifier of the model, with a recognition accuracy of 92.5%. The study findings are significant for developing a human–machine co-driving control strategy and improving vehicle driving safety. [ABSTRACT FROM AUTHOR]

Class imbalance (CI) in classification problems arises when the number of observations belonging to one class is lower than the other. Ensemble learning combines multiple models to obtain a robust model and has been prominently used with data augmentation methods to address class imbalance problems. In the last decade, a number of strategies have been added to enhance ensemble learning and data augmentation methods, along with new methods such as generative adversarial networks (GANs). A combination of these has been applied in many studies, and the evaluation of different combinations would enable a better understanding and guidance for different application domains. In this paper, we present a computational study to evaluate data augmentation and ensemble learning methods used to address prominent benchmark CI problems. We present a general framework that evaluates 9 data augmentation and 9 ensemble learning methods for CI problems. Our objective is to identify the most effective combination for improving classification performance on imbalanced datasets. The results indicate that combinations of data augmentation methods with ensemble learning can significantly improve classification performance on imbalanced datasets. We find that traditional data augmentation methods such as the synthetic minority oversampling technique (SMOTE) and random oversampling (ROS) are not only better in performance for selected CI problems, but also computationally less expensive than GANs. Our study is vital for the development of novel models for handling imbalanced datasets. • Class imbalance (CI) in classification problems arises one class is lower than the other classes. • We present a computational review to evaluate data augmentation and ensemble learning methods for CI problems. • We propose a general framework that evaluates 10 data augmentation and 10 ensemble learning methods for CI problems. • Our objective is to identify the most effective combination for improving classification performance on imbalanced datasets. • The results indicate that the combinations can significantly improve classification performance on imbalanced datasets. [ABSTRACT FROM AUTHOR]

For the purpose of managing financial risk and making investment decisions, interval stock price forecasting is essential. Currently, decomposition integration frameworks are widely used in point-valued stock price forecasting studies, mainly focusing on mining internal information. However, point forecasts are difficult to adequately capture price uncertainty and may suffer from loss of volatile information. Therefore, this paper proposes an enhanced interval-valued decomposition integration model for stock price prediction based on comprehensive feature extraction and optimized deep learning. Firstly, the interval variational modal decomposition with feedback mechanism (FIVMD) is proposed to extract internal features and can decompose interval values into interval trend and residual. FIVMD not only solves the interval decomposition challenge, but also helps to improve the internal feature extraction capability. Secondly, while considering the influencing factors more comprehensively, appropriate feature selection and compression techniques can effectively achieve external feature extraction, obtain the best influencing factors, and improve the modeling capability of high-dimensional data. Finally, the final prediction results are obtained by modeling the interval trend and residuals separately through the optimization algorithm and deep learning model to improve the prediction accuracy. The results of the empirical analysis reveal that the proposed interval decomposition integrated model has the smallest of the three evaluation metrics, where the values of interval mean average percentage errors (IMAPE) are 1.8188%, 1.1244%, 1.9001%, and 2.1542% respectively. This shows that the model is significantly more accurate and stable than the other comparative models, and it is a successful model for predicting interval-valued stock prices. [ABSTRACT FROM AUTHOR]

Personalized federated learning (PFL) is an exciting approach that allows machine learning (ML) models to be trained on diverse and decentralized sources of data, while maintaining client privacy and autonomy. However, PFL faces several challenges that can deteriorate the performance and effectiveness of the learning process. These challenges include data heterogeneity, communication overhead, model privacy, model drift, client heterogeneity, label noise and imbalance, federated optimization challenges, and client participation and engagement. To address these challenges, researchers are exploring innovative techniques and algorithms that can enable efficient and effective PFL. These techniques include several optimization algorithms. This research survey provides an overview of the challenges and motivations related to the model optimization strategies for PFL, as well as the state-of-the-art (SOTA) methods and algorithms which seek to provide solutions of these challenges. Overall, this survey can be a valuable resource for researchers who are interested in the emerging field of PFL as well as its potential for personalized machine learning in a federated environment. • A Comprehensive Review of Contemporary Approaches in Personalized Federated Learning (PFL). • Categorization and classification of various methods and solutions to provide comprehensive taxonomies. • A diverse array of literature that addresses emerging challenges and cutting-edge solutions in the field. • This paper outlines and proposes new research directions that can be pursued by the interested community. [ABSTRACT FROM AUTHOR]

In 2016, Google proposed Federated Learning (FL) as a novel paradigm to train Machine Learning (ML) models across the participants of a federation while preserving data privacy. Since its birth, Centralized FL (CFL) has been the most used approach, where a central entity aggregates participants' models to create a global one. However, CFL presents limitations such as communication bottlenecks, single point of failure, and reliance on a central server. Decentralized Federated Learning (DFL) addresses these issues by enabling decentralized model aggregation and minimizing dependency on a central entity. Despite these advances, current platforms training DFL models struggle with key issues such as managing heterogeneous federation network topologies, adapting the FL process to virtualized or physical deployments, and using a limited number of metrics to evaluate different federation scenarios for efficient implementation. To overcome these challenges, this paper presents Fedstellar, a novel platform designed to train FL models in a decentralized, semi-decentralized, and centralized fashion across diverse federations of physical or virtualized devices. Fedstellar allows users to create federations by customizing parameters like the number and type of devices training FL models, the network topology connecting them, the machine and deep learning algorithms, or the datasets of each participant, among others. Additionally, it offers real-time monitoring of model and network performance. The Fedstellar implementation encompasses a web application with an interactive graphical interface, a controller for deploying federations of nodes using physical or virtual devices, and a core deployed on each device, which provides the logic needed to train, aggregate, and communicate in the network. The effectiveness of the platform has been demonstrated in two scenarios: a physical deployment involving single-board devices such as Raspberry Pis for detecting cyberattacks and a virtualized deployment comparing various FL approaches in a controlled environment using MNIST and CIFAR-10 datasets. In both scenarios, Fedstellar demonstrated consistent performance and adaptability, achieving F 1 s c o r e s of 91%, 98%, and 91.2% using DFL for detecting cyberattacks and classifying MNIST and CIFAR-10, respectively, reducing training time by 32% compared to centralized approaches. • Open-source platform for training FL models in a decentralized fashion. • Support for diverse FL architectures, network topologies, datasets, and ML/DL models. • User-friendly setup and efficient monitoring of FL processes. • Successful deployment across diverse physical and virtualized devices. • Promising performance in tasks like cyberattack detection and image classification. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) presents a decentralized approach to model training in the agri-food sector and offers the potential for improved machine learning performance, while ensuring the safety and privacy of individual farms or data silos. However, the conventional FL approach has two major limitations. First, the heterogeneous data on individual silos can cause the global model to perform well for some clients but not all, as the update direction on some clients may hinder others after they are aggregated. Second, it is lacking with respect to the efficiency perspective concerning communication costs during FL and large model sizes. This paper proposes a new technical solution that utilizes network pruning on client models and aggregates the pruned models. This method enables local models to be tailored to their respective data distribution and mitigate the data heterogeneity present in agri-food data. Moreover, it allows for more compact models that consume less data during transmission. We experiment with a soybean yield forecasting dataset and find that this approach can improve inference performance by 15.5% to 20% compared to FedAvg, while reducing local model sizes by up to 84% and the data volume communicated between the clients and the server by 57.1% to 64.7%. Our method demonstrates the potential to use efficient models that are more environmentally friendly to support the agri-food sector's transition to net zero. Future enhancements of this method could further optimize distributed learning in agri-food, enhancing sustainability and applicability. • We propose a new federated pruning learning method for yield forecasting. • Our method can be used to improve local inference performance. • The method reduces communication costs during training and model sizes. • Our method facilitates the on-edge implementation of ML models using sparse tensor. • The method is generalizable and works with different pruning policies and schedules. [ABSTRACT FROM AUTHOR]

Deep learning has emerged as a powerful tool in various domains, revolutionising machine learning research. However, one persistent challenge is the scarcity of labelled training data, which hampers the performance and generalisation of deep learning models. To address this limitation, researchers have developed innovative methods to overcome data scarcity and enhance deep model learning capabilities. Two prevalent techniques that have gained significant attention are transfer learning and self-supervised learning. Transfer learning leverages knowledge learned from pre-training on a large-scale dataset, such as ImageNet, and applies it to a target task with limited labelled data. This approach allows models to benefit from the learned representations and effectively transfer knowledge to new tasks, resulting in improved learning performance and generalisation. On the other hand, self-supervised learning focuses on training models using pretext tasks that do not require manual annotation, allowing them to learn valuable representations from large amounts of unlabelled data. These learned representations can then be fine-tuned for downstream tasks, mitigating the need for extensive labelled data. In recent years, transfer and self-supervised learning have found applications in various fields, including medical image processing, video recognition, and natural language processing. These approaches have demonstrated remarkable achievements, enabling breakthroughs in areas such as disease diagnosis, object recognition, and language understanding. However, while these methods offer numerous advantages, they also have limitations. For example, transfer learning may face domain mismatch issues between the pre-training and target domains, while self-supervised learning requires careful design of pretext tasks to ensure meaningful representations. This review paper explores the recent applications of these pre-training methods in various fields within the past three years. It delves into the advantages and limitations of each approach, assesses the performance of models employing these techniques, and identifies potential directions for future research. By providing a comprehensive review of current pre-training methods, this article offers guidance for selecting the best technique for specific deep learning applications to address the data scarcity issue. • Differences between Transfer Learning and Self-Supervised Learning. • Applications of Transfer Learning and Self-Supervised Learning. • Advantages and Limitations of Transfer Learning and Self-Supervised Learning. • Recommendations for the Use of Each Approach in Specific Applications. [ABSTRACT FROM AUTHOR]

Seepage pressure data analysis is considered one of the most direct ways to evaluate seepage safety in dam engineering. Therefore, accurate and consistent prediction of seepage pressure is essential. However, satisfactory predictive performance is rugged due to the irregular, nonlinear, and non-stationary nature of seepage pressure monitoring data. This paper proposes a combined optimization prediction model for seepage pressure prediction to resolve the deficiency. The model is coupled with a data decomposition technology, multi-machine learning fusion technology, adaptive variable weight combined strategy, and an improved optimization algorithm. The interaction between high and low frequency components is eliminated by improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) technique to achieve the smoothing of non-stationary data. Multi-machine learning models are integrated by the combined strategy to compensate for the shortcomings of single models. Combined weights are adaptively changed by the kernel extreme learning machine (KELM) variable weighting strategy to overcome the limitations of traditional fixed weights. For the challenge of determining KELM parameters, the artificial electric field algorithm is improved to optimize KELM parameters to accelerate the determination of adaptive weights. The application of the proposed model to an earth-rock dam demonstrates enhanced predictive accuracy of seepage pressure by implementing strategies. The model is an effective tool for the seepage pressure prediction of dams and is conducive to dam operation and maintenance management. [ABSTRACT FROM AUTHOR]

• MMODE is developed as a new hybrid gene selection method for tumor identification. • MMODE combines multi-tasking and multi-objective frameworks. • MMODE uses a new elite strategy and a new guidance strategy. • MMODE selects a few genes and achieves high classification accuracy. A key preprocessing step in tumor recognition based on microarray expression profile data and machine learning is to identify tumor marker genes. Gene selection aims to select the most relevant gene subset from the original ultra-high dimensional microarray expression profile data to improve tumor identification performance. Inspired by evolutionary multitasking (EMT) and multi-objective optimization, this paper puts forward a novel multitasking multi-objective differential evolution gene selection algorithm (MMODE) which uses new elite and guidance strategies to select the best gene subsets. MMODE initializes two different populations according to different filtering criteria to increase the diversity of the search. These two populations guide their respective populations to search in the optimal direction through knowledge transfer in the evolutionary process. In addition, MMODE employs new elite and guidance strategies that enables individuals to narrow the search range and jump out of local optima. The proposed algorithm is validated on 13 publicly available microarray expression datasets in comparison with state-of-the-art gene selection algorithms. The experimental results show that MMODE can find smaller gene subsets and achieve higher classification accuracy compared with other algorithms. [ABSTRACT FROM AUTHOR]

The focus of this paper is to improve short-term load forecasting for electric power. To achieve this goal, the study explores and evaluates hybrid models, specifically using the CatBoost and XGBoost algorithms, which are optimized with different optimizers. The study incorporates hourly electricity load data and also includes temperature data to enhance the precision of the forecasting models. Statistical metrics are then used to assess the performance of these models. The study evaluates the performance of the hybrid models on both training and testing datasets. It finds that the CatBoost-Arithmetic Optimization Algorithm hybrid model outperforms the other models in the training dataset. However, in the testing dataset, the XGBoost- Arithmetic Optimization Algorithm hybrid model demonstrates superior performance compared to the CatBoost models. The study conducts an importance and sensitivity analysis to understand which variables have the most significant impact on the target variable, which is likely electricity load. The results of this analysis reveal that temperature is the most influential variable affecting the target variable. Additionally, the month variable is identified as having a notable impact on load forecasting. These findings suggest that employing hybrid models, particularly those optimized with appropriate algorithms, can significantly improve the accuracy of short-term load forecasting. Moreover, the study highlights the importance of incorporating temperature data into these models, as temperature is a key driver of electricity load patterns. [ABSTRACT FROM AUTHOR]

Traditional machine learning algorithms face challenges in assessing flood susceptibility reliably due to their low robustness and the inherent 'black-box' nature. This paper utilizes five hyperparameter optimization algoirthms (HPO), namely grid search (GS), random search (RS), gauss process (GP), tree-structured parzen estimator (TPE) and simulated annealing (SA), to tune the traditional random forest's (RF) hyperparameters to improve the robustness of flood hazard mapping (FHM) models at Ningxiang City Hunan Province, China. Additionally, SHapley Additive exPlanations (SHAP) method were used to interpret the decision-mechanisms of these flood hazard models. This study considers 19 pluvial flood influencing factors and 2064 flood locations to create a geospatial database. The performance of each hybrid model was evaluated by area under the receiver operating characteristic (ROC) curve (AUC) and several validation methods. The results demonstrate that the developed hybrid models demonstrated good performance, with RF-TPE achieving the highest AUC (0.9660), followed by RF-GP (0.9648), RF-SA (0.9624), RF-GS (0.9612), RF-RS (0.9600), and RF (0.9539). The RF-TPE model exhibits superior robustness than other models, and the FHM constructed using it is more reliable. HPO is an effective approach to improve the predictive accuracy and robustness of FHM models. When considering limited computational resources, Bayesian optimization (TPE) should be prioritized for optimizing FHM models, followed by metaheuristic algorithms and model-free algorithms. Moreover, the study revealed that distance from river, peak rainfall intensity, continuous rainfall, antecedent effective rainfall, and terrain relief, are the most significant for pluvial FHM modeling in this region. [ABSTRACT FROM AUTHOR]

In recent years, the enormous development of Machine Learning, especially Deep Learning, has led to the widespread adoption of Artificial Intelligence (AI) systems in a large variety of contexts. Many of these systems provide excellent results but act as black-boxes. This can be accepted in various contexts, but there are others (e.g., medical ones) where a result returned by a system cannot be accepted without an explanation on how it was obtained. Explainable AI (XAI) is an area of AI well suited to explain the behavior of AI systems that act as black-boxes. In this paper, we propose a model-agnostic XAI framework to explain the behavior of classifiers. Our framework is based on network theory; thus, it is able to make use of the enormous amount of results that researchers in this area have discovered over time. Being network-based, our framework is completely different from the other model-agnostic XAI approaches. Furthermore, it is parameter-free and is able to handle heterogeneous features that may not even be independent of each other. Finally, it introduces the notion of dyscrasia that allows us to detect not only which features are important in a particular task but also how they interact with each other. • Performing model-agnostic Explainable AI on classifiers. • Introducing the concept of dyscrasia of feature occurrences. • Assessing the relevance of a feature during classification. [ABSTRACT FROM AUTHOR]

This paper presents the FlowTransformer framework, a novel approach for implementing transformer-based Network Intrusion Detection Systems (NIDSs). FlowTransformer leverages the strengths of transformer models in identifying the long-term behaviour and characteristics of networks, which are often overlooked by most existing NIDSs. By capturing these complex patterns in network traffic, FlowTransformer offers a flexible and efficient tool for researchers and practitioners in the cybersecurity community who are seeking to implement NIDSs using transformer-based models. FlowTransformer allows the direct substitution of various transformer components, including the input encoding, transformer, classification head, and the evaluation of these across any flow-based network dataset. To demonstrate the effectiveness and efficiency of the FlowTransformer framework, we utilise it to provide an extensive evaluation of various common transformer architectures, such as GPT 2.0 and BERT, on three commonly used public NIDS benchmark datasets. We provide results for accuracy, model size and speed. A key finding of our evaluation is that the choice of classification head has the most significant impact on the model performance. Surprisingly, Global Average Pooling, which is commonly used in text classification, performs very poorly in the context of NIDS. In addition, we show that model size can be reduced by over 50%, and inference and training times improved, with no loss of accuracy, by making specific choices of input encoding and classification head instead of other commonly used alternatives. • A framework to tackle challenges of implementing transformer-based NIDSs. • Providing the implementation (Code) for public use. • Evaluation of transformers for NIDS, vs components, speed, and accuracy. • Exploring various trade-offs in model size, performance, and inference time. • An optimal transformer architecture for transformer-based NIDS research. [ABSTRACT FROM AUTHOR]

Thunderstorms can be a large source of disruption for European air-traffic management causing a chaotic state of operation within the airspace system. In current practice, air-traffic managers are provided with imprecise forecasts which limit their ability to plan strategically. As a result, weather mitigation is performed using tactical measures with a time horizon of three hours. Increasing the lead time of thunderstorm predictions to the day before operations could help air-traffic managers plan around weather and improve the efficiency of air-traffic-management operations. Emerging techniques based on machine learning have provided promising results, partly attributed to reduced human bias and improved capacity in predicting thunderstorms purely from numerical weather prediction data. In this paper, we expand on our previous work on thunderstorm forecasting, by applying convolutional neural networks (CNNs) to exploit the spatial characteristics embedded in the weather data. The learning task of predicting convection is formulated as a binary-classification problem based on satellite data. The performance of multiple CNN-based architectures, including a fully-convolutional neural network (FCN), a CNN-based encoder–decoder, a U-Net, and a pyramid-scene parsing network (PSPNet) are compared against a multi-layer-perceptron (MLP) network. Our work indicates that CNN-based architectures improve the performance of point-prediction models, with a fully-convolutional neural-network architecture having the best performance. Results show that CNN-based architectures can be used to increase the prediction lead time of thunderstorms. Lastly, a case study illustrating the applications of convection-prediction models in an air-traffic-management setting is presented. • CNNs are applied to numerical weather prediction data to predict thunderstorms. • CNNs outperform MLP architectures in the task of predicting thunderstorms. • ML-based weather forecasts allow for better planning of air traffic operations. [ABSTRACT FROM AUTHOR]

Automatic modulation recognition (AMR) has garnered significant attention in both civilian and military domains, with applications ranging from spectrum sensing and cognitive radio (CR) to the deterrence of adversary communication. Index modulation (IM) represents an innovative digital modulation technique that exploits the indices of parameters of communication systems to transmit extra information bits. This paper aims to examine the performance of a convolutional neural network (CNN)-based AMR across various IM systems, including spatial modulation (SM), quadrature spatial modulation (QSM), and generalized spatial modulation (GSM) with eight digital modulation schemes. In this study, we leverage confusion matrices, receiver operating characteristic (ROC) curves, and F1 scores to illustrate the recognition model's outputs. • Modulation recognition models exploit deep learning algorithms effectively. • Index modulation is a novel technique for conveying additional information. • Three datasets are created for the recognition of eight digital modulation schemes. • Three modulation recognition scenarios analyze the system's performance. • Quadrature amplitude modulation types are more robust than phase shift keying types. [ABSTRACT FROM AUTHOR]

Amidst the robust development of the service economy and information technology, the information age has significantly transformed consumption concepts and service demands. The requirements for logistics services from customers have become increasingly stringent. Beyond price and speed, quality has steadily emerged as the most crucial factor. When evaluating express services, experts primarily choose indicators from an enterprise perspective, mainly overlooking customer perception. The purpose of this paper is to establish a key index system for assessing the quality of express services aimed at improving the service quality of express deliveries within the development mode of intelligent logistics. In terms of methodology, we proposed a quantitative and qualitative e-commerce logistics service evaluation system. Specifically, we developed a key index system for assessing the quality of express services based on the theory of the six senses and the selection from an index library at first. And then, we ascertained the weight of each index in the evaluation by analyzing the proportion of customer comments. We employed analysis methods of machine learning, text data processing, mathematical statistics, and big data technology to determine the evaluation index and evaluation model of express service. Finally, our primary findings from the empirical verification and analysis of a real express enterprise indicate that this evaluation model can effectively assess the quality of express services. Our model can evaluate the current service quality and predict the quality of future services. We have derived some interpretations and conclusions based on these analyses and findings. Firstly, the level of customer perception is of great importance in the evaluation of express service quality. Secondly, our model can provide valuable feedback for express enterprises to improve their services. Lastly, we proposed corresponding improvement strategies to enhance the service quality of express enterprises under the intelligent logistics development mode. • Customer perception, not just enterprise indicators, is critical for comprehensive express service quality evaluation. • A machine learning and big data driven model is developed to evaluate and predict express service quality. • The model is empirically shown to be effective and can aid service quality improvement for intelligent logistics. [ABSTRACT FROM AUTHOR]

The high dimensionality of data hinders the learning ability of machine learning algorithms. Feature selection techniques can be used to reduce dimensionality, which is an important step for processing high-dimensional data. Feature selection solves this problem by removing irrelevant and redundant information, which can improve learning models, reduce calculation time, and improve learning accuracy. In this paper, a novel filter in mixed-attribute datasets for feature selection is proposed. The independent attributes are mixed or heterogeneous in the sense that both numerical and categorical attribute types may appear together in the same dataset. Based on the preordonnances theory, we use a new concept to quantify the relevance and redundancy of features even if there are heterogeneous (mixed-type) data. The technique for order preference by similarity to the ideal solution is one of the well-known multicriteria decision-making methods; it is utilized as a weighting and informative feature selection filter. To assess the effectiveness of the proposed method, several experiments, both simulated and real, are performed, including a comparison to other well-known filter methods. The experimental results show that, in most cases, the method yielded competitive results in comparison to other methods. • Introducing new criterions for relevance and complementarity. • Dealing with mixed-attribute datasets without requiring a preprocessing step. • The multi-criteria decision method, namely TOPSIS, is used to score the explanatory features. • Investigate the performance of the proposed filter in high-dimensional data by using simulated and real data sets. [ABSTRACT FROM AUTHOR]

Over the years, the study on logic mining approach has increased exponentially. However, most logic mining models disregarded any efforts in expanding the search space which led to poor generalizability property of the retrieved induced logic. In light of this gap, this paper initiated the hybridization of logic mining approach with a multi-objective training algorithm namely Modified Niche Genetic Algorithm. The core impetus of this algorithm is to ensure optimal production of multiple superstrings via Wan Abdullah method resulting in multiple units associative memory feature of the Discrete Hopfield Neural Network. Therefore, the storage capacity of DHNN increases which directed towards larger search space of locating optimal induced logic. Additionally, several modifications were imposed to counter other issues such as, rigid logical rule, outdated quality of best logic, and high dependency on the supervised attributes selection method. Experimentation was done on 20 repository datasets from reputable machine learning repositories. Results showed that the proposed model outperformed all baseline methods in terms of accuracy = 0.8727, precision = 0.9845, specificity = 0.9988, and Matthew's correlation coefficient = 0.5815. [ABSTRACT FROM AUTHOR]

Financial fraud is a persistent problem in the finance industry that may have severe consequences for individuals, businesses, and economies. Graph Neural Networks (GNNs) are a class of deep learning models designed to operate on graph data structures that consist of nodes and edges connecting them. GNNs have emerged as a powerful tool for detecting fraudulent activities in complex financial systems because they can analyze the network structure of financial transactions, capturing patterns and anomalies that traditional rule-based and machine learning methods might miss. The objective of this systematic review is to provide a comprehensive overview of the current state-of-the-art technologies in using Graph Neural Networks (GNNs) for financial fraud detection, identify the gaps and limitations in the existing research, and suggest potential directions for future research. We searched five academic databases, including Web of Science, Scopus, IEEE Xplore, ACM, and science direct using specific keywords and search strings related to graph neural networks, financial areas, and anomaly detection to identify relevant publications, resulting in a total of 388 unique articles. We selected the relevant publications based on the inclusion, exclusion, and quality assessment criteria, and 33 articles were included in the review. In addition, forward snowballing was used to identify relevant papers that were not captured in the initial search. Data was extracted from the selected articles, then analyzed and summarized to identify current state, gaps, and trends in the literature. Our review presents a new taxonomy of GNNs applied in financial fraud detection and identifies potential research directions in this field. We find that GNNs applied to financial fraud detection have mostly been employed in a supervised or semi-supervised manner, with limited exploration of unsupervised approaches. In addition to financial areas, we explore the different types of graphs such as homogeneous, heterogenous, static, temporal, and dynamic graphs, and investigate the various learning mechanisms and anomaly types studied. We also note a lack of research on edge-level and graph-level anomaly detection commonly employed in financial domain. [ABSTRACT FROM AUTHOR]

Computer system failures can cause significant financial and reputational damage, making it critical to detect and prevent them before they occur. In this article, we propose a novel approach for predicting system failures based on log file analysis, called MABFailPred. Our approach leverages a Blockchain-based Multi-Agent architecture to analyze execution traces and detect early warning signals for system failure prediction during execution. Moreover, we introduce a new method for extracting log keys called HCLPars. The MABFailPred approach comprises four agents that work together to gather log keys, categorize data, develop a failure model, and make predictions. The use of blockchain ensures that agent and administrator predictions are synchronized, and agent collaboration is coordinated. We validate our approach by applying it to a real-world distributed system and accurately predicting problems before they occur. Our proposed approach represents a significant advancement in the field of system failure prediction, with the potential to enhance the reliability and security of computer systems. By employing a machine learning algorithm, our approach can predict anomalies in real-time and places a strong emphasis on the continuous monitoring of system behavior. The paper includes a validation of the approach using the Hadoop Distributed File System (HDFS) log dataset. According to our findings, the proposed approach achieves superior performance compared to other methods on the same dataset, with a precision of 0.998. This result establishes a new state-of-the-art performance level for the dataset. • Failure prediction based on system log files using Multi agent system and Blockchain. • Implement and evaluate MABFailPred based on supervised learning algorithms (rule-based classification, decision tree algorithm). • Implement and evaluate a Hierarchical Clustering Log Parsing method (HCLPars), for extracting log keys. [ABSTRACT FROM AUTHOR]

Heart rate variability serves as a valuable indicator and biomarker for stress detection and monitoring. Feature selection, which aims to identify relevant features from a large set of variables, is a crucial preprocessing step towards this. However, this task becomes challenging due to high dimensionality and the presence of irrelevant and redundant attributes. The Minimum Redundancy and Maximum Relevance (mRMR) feature selection method addresses this challenge by selecting relevant features while controlling redundancy. This paper presents extensions and evaluated versions of the mRMR feature selection methods for stress detection using Heart Rate Variability (HRV) measures. The proposed feature selection methods extend the traditional mRMR by replacing the Pearson correlation redundancy with non-linear feature redundancy measures capable of capturing more complex relationships between variables. An extensive empirical evaluation is conducted on the proposed mRMR extensions, comparing them with four other baseline feature selection methods using three publicly available datasets. The experimental results demonstrate the effectiveness of incorporating the non-linear feature redundancy measure into the feature selection process. [ABSTRACT FROM AUTHOR]