Search

Self-supervised learning is a new machine learning method that does not rely on manually labeled data, and learns from rich unlabeled data itself by designing agent tasks using the input data as supervision to obtain a more generalized representation for application in downstream tasks. However, the current self-supervised learning suffers from the problem of relying on the selection and number of negative samples and the problem of sample bias phenomenon after graph data augmentation. In this paper, we investigate the above problems and propose a corresponding solution, proposing a graph contrastive learning algorithm without negative samples. The model uses matrix sketching in the implicit space for feature augmentation to reduce sample bias and iteratively trains the mutual correlation matrix of two viewpoints by drawing closer to the distance of the constant matrix as the objective function. This method does not require techniques such as negative samples, gradient stopping, and momentum updating to prevent self-supervised model collapse. This method is compared with 10 graph representation learning algorithms on four datasets for node classification tasks, and the experimental results show that the algorithm proposed in this paper achieves good results. [ABSTRACT FROM AUTHOR]

The level of difficulty of mathematical test items is a critical aspect for evaluating test quality and educational outcomes. Accurately predicting item difficulty during test creation is thus significantly important for producing effective test papers. This study used more than ten years of content and score data from China's Henan Provincial College Entrance Examination in Mathematics as an evaluation criterion for test difficulty, and all data were obtained from the Henan Provincial Department of Education. Based on the framework established by the National Center for Education Statistics (NCES) for test item assessment methodology, this paper proposes a new framework containing eight features considering the uniqueness of mathematics. Next, this paper proposes an XGBoost-based SHAP model for analyzing the difficulty of mathematics tests. By coupling the XGBoost method with the SHAP method, the model not only evaluates the difficulty of mathematics tests but also analyzes the contribution of specific features to item difficulty, thereby increasing transparency and mitigating the "black box" nature of machine learning models. The model has a high prediction accuracy of 0.99 for the training set and 0.806 for the test set. With the model, we found that parameter-level features and reasoning-level features are significant factors influencing the difficulty of subjective items in the exam. In addition, we divided senior secondary mathematics knowledge into nine units based on Chinese curriculum standards and found significant differences in the distribution of the eight features across these different knowledge units, which can help teachers place different emphasis on different units during the teaching process. In summary, our proposed approach significantly improves the accuracy of item difficulty prediction, which is crucial for intelligent educational applications such as knowledge tracking, automatic test item generation, and intelligent paper generation. These results provide tools that are better aligned with and responsive to students' learning needs, thus effectively informing educational practice. [ABSTRACT FROM AUTHOR]

Pattern recognition systems always misclassify anomalies, which can be dangerous for uninformed users. Therefore, anomalies must be filtered out from each classification. The main challenge for the anomaly filter design is the huge number of possible anomaly samples compared with the number of samples in the training set. Tailoring the filter for the given classifier is just the first step in this reduction. Paper tests the hypothesis that the filter trained in avoiding "near" anomalies will also refuse the "far" anomalies, and the anomaly detector is then just a classifier distinguishing between "far real" and "near anomaly" samples. As a "far real" samples generator was used, a Generative Adversarial Network (GAN) fake generator that transforms normally distributed random seeds into fakes similar to the training samples. The paper proves the assumption that seeds unused in fake training will generate anomalies. These seeds are distinguished according to their Chebyshev norms. While the fakes have seeds within the hypersphere with a given radius, the near anomalies have seeds within the sphere near cover. Experiments with various anomaly test sets have shown that GAN-based anomaly detectors create a reliable anti-anomaly shield using the abovementioned assumptions. The proposed anomaly detector is tailored to the given classifier, but its limitation is due to the need for the availability of the database on which the classifier was trained. [ABSTRACT FROM AUTHOR]

Accurately predicting the price of carbon is an effective way of ensuring the stability of the carbon trading market and reducing carbon emissions. Aiming at the non-smooth and non-linear characteristics of carbon price, this paper proposes a novel hybrid prediction model based on improved feature extraction and non-linear integration, which is built on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), fuzzy entropy (FuzzyEn), improved random forest using particle swarm optimisation (PSORF), extreme learning machine (ELM), long short-term memory (LSTM), non-linear integration based on multiple linear regression (MLR) and random forest (MLRRF), and error correction with the autoregressive integrated moving average model (ARIMA), named CEEMDAN-FuzzyEn-PSORF-ELM-LSTM-MLRRF-ARIMA. Firstly, CEEMDAN is combined with FuzzyEn in the feature selection process to improve extraction efficiency and reliability. Secondly, at the critical prediction stage, PSORF, ELM, and LSTM are selected to predict high, medium, and low complexity sequences, respectively. Thirdly, the reconstructed sequences are assembled by applying MLRRF, which can effectively improve the prediction accuracy and generalisation ability. Finally, error correction is conducted using ARIMA to obtain the final forecasting results, and the Diebold–Mariano test (DM test) is introduced for a comprehensive evaluation of the models. With respect to carbon prices in the pilot regions of Shenzhen and Hubei, the results indicate that the proposed model has higher prediction accuracy and robustness. The main contributions of this paper are the improved feature extraction and the innovative combination of multiple linear regression and random forests into a non-linear integrated framework for carbon price forecasting. However, further optimisation is still a work in progress. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence (AI), deep learning is widely used in various industries. At CRYPTO 2019, researchers used deep learning to analyze the block cipher for the first time and constructed a differential neural network distinguisher to meet a certain accuracy. In this paper, a mixture differential neural network distinguisher using ResNet is proposed to further improve the accuracy by exploring the mixture differential properties. Experiments are conducted on SIMON32/64, and the accuracy of the 8-round mixture differential neural network distinguisher is improved from 74.7% to 92.3%, compared with that of the previous differential neural network distinguisher. The prediction accuracy of the differential neural network distinguisher is susceptible to the choice of the specified input differentials, whereas the mixture differential neural network distinguisher is less affected by the input difference and has greater robustness. Furthermore, by combining the probabilistic expansion of rounds and the neutral bit, the obtained mixture differential neural network distinguisher is extended to 11 rounds, which can realize the 12-round actual key recovery attack on SIMON32/64. With an appropriate increase in the time complexity and data complexity, the key recovery accuracy of the mixture differential neural network distinguisher can be improved to 55% as compared to 52% of the differential neural network distinguisher. The mixture differential neural network distinguisher proposed in this paper can also be applied to other lightweight block ciphers. [ABSTRACT FROM AUTHOR]

Recent research has revealed that subtle imperceptible perturbations can deceive well-trained neural network models, leading to inaccurate outcomes. These instances, known as adversarial examples, pose significant threats to the secure application of machine learning techniques in safety-critical systems. In this paper, we delve into the study of one-pixel attacks in deep neural networks, recently reported as a kind of adversarial examples. To identify such one-pixel attacks, most existing methodologies rely on the differential evolution method, which utilizes random selection from the current population to escape local optima. However, the differential evolution technique might waste search time and overlook good solutions if the number of iterations is insufficient. Hence, in this paper, we propose a gradient ascent with momentum approach to efficiently discover good solutions for the one-pixel attack problem. As our method takes a more direct route to the goal compared to existing methods relying on blind random walks, it can effectively identify one-pixel attacks. Our experiments conducted on popular CNNs demonstrate that, in comparison with existing methodologies, our technique can detect one-pixel attacks significantly faster. [ABSTRACT FROM AUTHOR]

This correction notice provides updates and corrections to a research paper on deep machine learning model-based cyber-attacks detection in smart power systems. The paper proposes a deep learning-based attack detection model that addresses system disturbances caused by natural events and cyber-attacks. It discusses the use of principal component analysis (PCA) for feature selection and the potential of deep learning-based and decision tree classifiers for detecting cyber-attacks in intelligent energy grids. The notice also includes updates to references, clarifications on the experimental data set, and feature selection based on PCA. The study evaluates the effectiveness of the proposed model using various assessment indexes and compares it to conventional methods, concluding that the proposed model is effective in detecting cyber-attacks and demonstrates good performance in detecting destructive attacks. [Extracted from the article]

The problem of a stabilisation system synthesis for the motion of a control object along a given spatial trajectory is considered. The complexity of the problem is that the preset trajectory is defined in the state subspace and not in time. This paper describes a stabilisation system synthesis for motion along a trajectory specified in time and along a trajectory specified in the form of a manifold in a state space. In order to construct a stabilisation system, it is necessary to determine a distance between an object and the given trajectory at each moment in time. For trajectories that are not given in time, the determination of this distance can be ambiguous. An object may be exactly on a trajectory but at a different time. This paper proposes some approaches to solve the problem. One of the approaches is to transform a given trajectory in a state subspace into a trajectory given in time. A description of a universal method to perform this transformation is presented. In order to solve the synthesis problem automatically, without having to analyse the mathematical model of the control object, it is suggested that machine learning control by symbolic regression is used. In computational experiments, examples of stabilisation system syntheses for quadcopter motion along a given spatial trajectory are presented. [ABSTRACT FROM AUTHOR]

This paper presents a study of fork–join systems. The fork–join system breaks down each customer into numerous tasks and processes them on separate servers. Once all tasks are finished, the customer is considered completed. This design enables the efficient handling of customers. The customers enter the system in a MAP flow. This helps create a more realistic and flexible representation of how customers arrive. It is important for modeling various real-life scenarios. Customers are divided into K ≥ 2 tasks and assigned to different subsystems. The number of tasks matches the number of subsystems. Each subsystem has a server that processes tasks, and a buffer that temporarily stores tasks waiting to be processed. The service time of a task by the k-th server follows a PH (phase-type) distribution with an irreducible representation ( β k , S k ), 1 ≤ k ≤ K . An analytical solution was derived for the case of K = 2 when the input MAP flow and service time follow a PH distribution. We have efficient algorithms to calculate the stationary distribution and performance characteristics of the fork–join system for this case. In general cases, this paper suggests using a combination of Monte Carlo and machine learning methods to study the performance of fork–join systems. In this paper, we present the results of our numerical experiments. [ABSTRACT FROM AUTHOR]

This paper describes the results from a study designed to illustrate the use of machine learning analytical techniques from a household consumer perspective. The outcome of interest in this study is a household's degree of financial preparedness as indicated by the presence of an emergency fund. In this study, six machine learning algorithms were evaluated and then compared to predictions made using a conventional regression technique. The selected ML algorithms showed better prediction performance. Among the six ML algorithms, Gradient Boosting, kNN, and SVM were found to provide the most robust degree of prediction and classification. This paper contributes to the methodological literature in consumer studies as it relates to household financial behavior by showing that when prediction is the main purpose of a study, machine learning techniques provide detailed yet nuanced insights into behavior beyond traditional analytic methods. [ABSTRACT FROM AUTHOR]

Wireless communication technologies have profoundly impacted the interconnectivity of mobile users and terminals. Nevertheless, the exponential increase in the number of users poses significant challenges, particularly in interference management, which is a major concern in wireless communication. Machine learning (ML) approaches have emerged as powerful tools for solving various problems in this domain. However, existing studies have not fully addressed the problem of interference management for wireless communication using ML techniques. In this paper, we explore the application of recurrent neural network (RNN) approaches to address co-channel interference in wireless communication. Specifically, we investigate the effectiveness of long short-term memory (LSTM), bidirectional LSTM (Bi-LSTM), and gated recurrent unit (GRU) network architectures in two different network settings. The first network comprises 10 connected devices, while the second network involves 20 devices. Our experimental results demonstrate that Bi-LSTM outperforms LSTM and GRU in terms of mean squared error, normalized mean squared error, and sum rate. While LSTM and GRU produce similar results, LSTM exhibits a marginal advantage over GRU. In addition, a combined RNN approach is also studied, and it can provide better results in dense networks. [ABSTRACT FROM AUTHOR]

As a type of financial derivative, the price fluctuation of futures is influenced by a multitude of factors, including macroeconomic conditions, policy changes, and market sentiment. The interaction of these factors makes the future trend become complex and difficult to predict. However, for investors, the ability to accurately predict the future trend of stock index futures price is directly related to the correctness of investment decisions and investment returns. Therefore, predicting the stock index futures market remains a leading and critical issue in the field of finance. To improve the accuracy of predicting stock index futures price, this paper introduces an innovative forecasting method by combining the strengths of Long Short-Term Memory (LSTM) networks and various Generalized Autoregressive Conditional Heteroskedasticity (GARCH)-family models namely, MULTI-GARCH-LSTM. This integrated approach is specifically designed to tackle the challenges posed by the nonstationary and nonlinear characteristics of stock index futures price series. This synergy not only enhances the model's ability to capture a wide range of market behaviors but also significantly improves the precision of future price predictions, catering to the intricate nature of financial time series data. Initially, we extract insights into the volatility characteristics, such as the aggregation of volatility in futures closing prices, by formulating a model from the GARCH family. Subsequently, the LSTM model decodes the complex nonlinear relationships inherent in the futures price series and incorporates assimilated volatility characteristics to predict future prices. The efficacy of this model is validated by applying it to an authentic dataset of gold futures. The empirical findings demonstrate that the performance of our proposed MULTI-GARCH-LSTM hybrid model consistently surpasses that of the individual models, thereby confirming the model's effectiveness and superior predictive capability. [ABSTRACT FROM AUTHOR]

Quantum machine learning stands out as one of the most promising applications of quantum computing, widely believed to possess potential quantum advantages. In the era of noisy intermediate-scale quantum, the scale and quality of quantum computers are limited, and quantum algorithms based on fault-tolerant quantum computing paradigms cannot be experimentally verified in the short term. The variational quantum algorithm design paradigm can better adapt to the practical characteristics of noisy quantum hardware and is currently one of the most promising solutions. However, variational quantum algorithms, due to their highly entangled nature, encounter the phenomenon known as the "barren plateau" during the optimization and training processes, making effective optimization challenging. This paper addresses this challenging issue by researching a variational quantum neural network optimization method based on collective intelligence algorithms. The aim is to overcome optimization difficulties encountered by traditional methods such as gradient descent. We study two typical applications of using quantum neural networks: random 2D Hamiltonian ground state solving and quantum phase recognition. We find that the collective intelligence algorithm shows a better optimization compared to gradient descent. The solution accuracy of ground energy and phase classification is enhanced, and the optimization iterations are also reduced. We highlight that the collective intelligence algorithm has great potential in tackling the optimization of variational quantum algorithms. [ABSTRACT FROM AUTHOR]

In order to solve the problems of traditional mental stress detection in college students that are time-consuming, random, and subjective, this paper proposes an intelligent perception-driven mental stress assessment method for college students. First, we analyze the factors in SRQ and SCL-90, which can be measured by intelligent sensing methods, including sleep, exercise, social interaction, and environment, and then perform feature extraction. Secondly, we use machine learning methods to build a mental stress assessment model. The Shapley additive explanations (SHAP) model is used to explain the training results. Experimental results show that the model proposed in this article can effectively assess the mental stress state of college students. This means that the collection of intelligent perception data based on the mental stress scale can effectively evaluate the mental stress state of college students and provide a new research idea for further developing a non-intrusive and real-time mental stress assessment for college students. [ABSTRACT FROM AUTHOR]

Positive and unlabeled learning (PU learning) is a significant binary classification task in machine learning; it focuses on training accurate classifiers using positive data and unlabeled data. Most of the works in this area are based on a two-step strategy: the first step is to identify reliable negative examples from unlabeled examples, and the second step is to construct the classifiers based on the positive examples and the identified reliable negative examples using supervised learning methods. However, these methods always underutilize the remaining unlabeled data, which limits the performance of PU learning. Furthermore, many methods require the iterative solution of the formulated quadratic programming problems to obtain the final classifier, resulting in a large computational cost. In this paper, we propose a new method called the absolute value inequality support vector machine, which applies the concept of eccentricity to select reliable negative examples from unlabeled data and then constructs a classifier based on the positive examples, the selected negative examples, and the remaining unlabeled data. In addition, we apply a hyperparameter optimization technique to automatically search and select the optimal parameter values in the proposed algorithm. Numerical experimental results on ten real-world datasets demonstrate that our method is better than the other three benchmark algorithms. [ABSTRACT FROM AUTHOR]

The proliferation of Internet of Things (IoT) devices and their integration into critical infrastructure and business operations has rendered them susceptible to malware and cyber-attacks. Such malware presents a threat to the availability and reliability of IoT devices, and a failure to address it can have far-reaching impacts. Due to the limited resources of IoT devices, traditional rule-based detection systems are often ineffective against sophisticated attackers. This paper addressed these issues by designing a new framework that uses a machine learning (ML) algorithm for the detection of malware. Additionally, it also employed sequential detection architecture and evaluated eight malware datasets. The design framework is lightweight and effective in data processing and feature selection algorithms. Moreover, this work proposed a classification model that utilizes one support vector machine (SVM) algorithm and is individually tuned with three different optimization algorithms. The employed optimization algorithms are Nuclear Reactor Optimization (NRO), Artificial Rabbits Optimization (ARO), and Particle Swarm Optimization (PSO). These algorithms are used to explore a diverse search space and ensure robustness in optimizing the SVM for malware detection. After extensive simulations, our proposed framework achieved the desired accuracy among eleven existing ML algorithms and three proposed ensemblers (i.e., NRO_SVM, ARO_SVM, and PSO_SVM). Among all algorithms, NRO_SVM outperforms the others with an accuracy rate of 97.8%, an F1 score of 97%, and a recall of 99%, and has fewer false positives and false negatives. In addition, our model successfully identified and prevented malware-induced attacks with a high probability of recognizing new evolving threats. [ABSTRACT FROM AUTHOR]

This paper examines the distribution of eigenvalues for a 2 × 2 random confusion matrix used in machine learning evaluation. We also analyze the distributions of the matrix's trace and the difference between the traces of random confusion matrices. Furthermore, we demonstrate how these distributions can be applied to calculate the superiority probability of machine learning models. By way of example, we use the superiority probability to compare the accuracy of four disease outcomes machine learning prediction tasks. [ABSTRACT FROM AUTHOR]

This study investigates the integration of quantum circuits with classical neural networks for enhancing credit scoring for small- and medium-sized enterprises (SMEs). We introduce a hybrid quantum–classical model, focusing on the synergy between quantum and classical rather than comparing the performance of separate quantum and classical models. Our model incorporates a quantum layer into a traditional neural network, achieving notable reductions in training time. We apply this innovative framework to a binary classification task with a proprietary real-world classical credit default dataset for SMEs in Singapore. The results indicate that our hybrid model achieves efficient training, requiring significantly fewer epochs (350) compared to its classical counterpart (3500) for a similar predictive accuracy. However, we observed a decrease in performance when expanding the model beyond 12 qubits or when adding additional quantum classifier blocks. This paper also considers practical challenges faced when deploying such models on quantum simulators and actual quantum computers. Overall, our quantum–classical hybrid model for credit scoring reveals its potential in industry, despite encountering certain scalability limitations and practical challenges. [ABSTRACT FROM AUTHOR]

To make wind power more competitive, it is necessary to reduce turbine downtime and reduce costs associated with wind turbine operation and maintenance (O&M). Incorporating machine learning in the development of condition-based predictive maintenance methodologies for wind turbines can enhance their efficiency and reliability. This paper presents a monitoring method that utilizes Density Based Support Vector Machines (DBSVM) and the evolutionary Fourier spectra of vibrations. This method allows for the smart monitoring of the function evolution of the turbine. A complex optimal function (FO) for 5-degree order has been developed that will be the boundary function of the DBSVM to be timely determined from the Fourier spectrum through the magnitude–frequency and place of the failure occurring in the wind turbine drivetrains. The trend of the failure was constructed with the maximal values of the optimal frequency function for both yesthe cases of the upwind and downwind parts of the gearbox. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive study on fall recognition and forecasting for reconfigurable stair-accessing robots by leveraging deep learning techniques. The proposed framework integrates machine learning algorithms and recurrent neural networks (RNNs), specifically Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM), for fall detection of service robots on staircases. The reconfigurable stair-accessing robot sTetro serves as the platform, and the fall data required for training models are generated in a simulation environment. The two machine learning algorithms are compared and their effectiveness on the fall recognition task is reported. The results indicate that the BiLSTM model effectively classifies falls with a median categorical accuracy of 94.10% in simulation and 90.02% with limited experiments. Additionally, the BiLSTM model can be used for forecasting, which is practically valuable for making decisions well before the onset of a free fall. This study contributes insights into the design and implementation of fall detection systems for service robots used to navigate staircases through deep learning approaches. Our experimental and simulation data, along with the simulation steps, are available for reference and analysis via the shared link. [ABSTRACT FROM AUTHOR]

Trajectory planning plays a crucial role in achieving unmanned excavator operations. The quality of trajectory planning results heavily relies on the level of rules extracted from objects such as scenes and optimization objectives, using traditional theoretical methods. To address this issue, this study focuses on professional operators and employs machine learning methods for job trajectory planning, thereby obtaining planned trajectories which exhibit excellent characteristics similar to those of professional operators. Under typical working conditions, data collection and analysis are conducted on the job trajectories of professional operators, with key points being extracted. Machine learning is then utilized to train models under different parameters in order to obtain the optimal model. To ensure sufficient samples for machine learning training, the bootstrap method is employed to adequately expand the sample size. Compared with the traditional spline curve method, the trajectories generated by machine learning models reduce the maximum speeds of excavator boom arm, dipper stick, bucket, and swing joint by 8.64 deg/s, 10.24 deg/s, 18.33 deg/s, and 1.6 deg/s, respectively; moreover, the error does not exceed 2.99 deg when compared with curves drawn by professional operators; and, finally, the trajectories generated by this model are continuously differentiable without position or velocity discontinuities, and their overall performance surpasses that of those generated by the traditional spline curve method. This paper proposes a trajectory generation method that combines excellent operators with machine learning and establishes a machine learning-based trajectory-planning model that eliminates the need for manually establishing complex rules. It is applicable to motion path planning in various working conditions of unmanned excavators. [ABSTRACT FROM AUTHOR]

The economic operation of hydropower stations has the potential to increase water use efficiency. However, there are some challenges, such as the fixed and unchangeable flow characteristic curve of the hydraulic turbines, and the large number of variables in optimal load distribution, which limit the progress of research. In this paper, we propose a new optimal method of the economic operation of hydropower stations based on improved Long Short-Term Memory neural network (I-LSTM) and Random Forest (RF) algorithm. Firstly, in order to accurately estimate the water consumption, the LSTM model's hyperparameters are optimized using improved particle swarm optimization, and the I-LSTM method is proposed to fit the flow characteristic curve of the hydraulic turbines. Secondly, the Random Forest machine learning algorithm is introduced to establish a load-distribution model with its powerful feature extraction and learning ability. To improve the accuracy of the load-distribution model, we use the K-means algorithm to cluster the historical data and optimize the parameters of the Random Forest model. A Hydropower Station in China is selected for a case study. It is shown that (1) the I-LSTM method fits the operating characteristics under various working conditions and actual operating characteristics of hydraulic turbines, ensuring that they are closest to the actual operating state; (2) the I-LSTM method is compared with Support Vector Machine (SVM), Extreme Learning Machine (ELM) and Long Short-Term Memory neural network (LSTM). The prediction results of SVM have a large error, but compared with ELM and LSTM, MSE is reduced by about 46% and 38% respectively. MAE is reduced by about 25% and 21%, respectively. RMSE is reduced by about 27% and 24%, respectively; (3) the RF algorithm performs better than the traditional dynamic programming algorithm in load distribution. With the passage of time and the increase in training samples, the prediction accuracy of the Random Forest model has steadily improved, which helps to achieve optimal operation of the units, reducing their average total water consumption by 1.24%. This study provides strong support for the application of intelligent low-consumption optimization strategies in hydropower fields, which can bring higher economic benefits and resource savings to renewable energy production. [ABSTRACT FROM AUTHOR]

To address the cost and efficiency issues of manually analysing requirement dependency in requirements engineering, a requirement dependency extraction method based on part-of-speech features and an improved stacking ensemble learning model (P-Stacking) is proposed. Firstly, to overcome the problem of singularity in the feature extraction process, this paper integrates part-of-speech features, TF-IDF features, and Word2Vec features during the feature selection stage. The particle swarm optimization algorithm is used to allocate weights to part-of-speech tags, which enhances the significance of crucial information in requirement texts. Secondly, to overcome the performance limitations of standalone machine learning models, an improved stacking model is proposed. The Low Correlation Algorithm and Grid Search Algorithms are utilized in P-stacking to automatically select the optimal combination of the base models, which reduces manual intervention and improves prediction performance. The experimental results show that compared with the method based on TF-IDF features, the highest F1 scores of a standalone machine learning model in the three datasets were improved by 3.89%, 10.68%, and 21.4%, respectively, after integrating part-of-speech features and Word2Vec features. Compared with the method based on a standalone machine learning model, the improved stacking ensemble machine learning model improved F1 scores by 2.29%, 5.18%, and 7.47% in the testing and evaluation of three datasets, respectively. [ABSTRACT FROM AUTHOR]

Improving the estimation of origin–destination (O-D) travel times poses a formidable challenge due to the intricate nature of transportation dynamics. Current deep learning models often require an overwhelming amount of data, both in terms of data points and variables, thereby limiting their applicability. Furthermore, there is a scarcity of models capable of predicting travel times with basic trip information such as origin, destination, and starting time. This paper introduces novel models rooted in the k-nearest neighbor (KNN) algorithm to tackle O-D travel time estimation with limited data. These models represent innovative adaptations of weighted KNN techniques, integrating the haversine distance of neighboring trips and incorporating correction factors to mitigate prediction biases, thereby enhancing the accuracy of travel time estimations for a given trip. Moreover, our models incorporate an adaptive heuristic to partition the time of day, identifying time blocks characterized by similar travel-time observations. These time blocks facilitate a more nuanced understanding of traffic patterns, enabling more precise predictions. To validate the effectiveness of our proposed models, extensive testing was conducted utilizing a comprehensive taxi trip dataset sourced from Santiago, Chile. The results demonstrate substantial improvements over existing state-of-the-art models (e.g., MAPE between 35 to 37% compared to 49 to 60% in other methods), underscoring the efficacy of our approach. Additionally, our models unveil previously unrecognized patterns in city traffic across various time blocks, shedding light on the underlying dynamics of urban mobility. [ABSTRACT FROM AUTHOR]

Stagnation at local optima represents a significant challenge in bio-inspired optimization algorithms, often leading to suboptimal solutions. This paper addresses this issue by proposing a hybrid model that combines the Orca predator algorithm with deep Q-learning. The Orca predator algorithm is an optimization technique that mimics the hunting behavior of orcas. It solves complex optimization problems by exploring and exploiting search spaces efficiently. Deep Q-learning is a reinforcement learning technique that combines Q-learning with deep neural networks. This integration aims to turn the stagnation problem into an opportunity for more focused and effective exploitation, enhancing the optimization technique's performance and accuracy. The proposed hybrid model leverages the biomimetic strengths of the Orca predator algorithm to identify promising regions nearby in the search space, complemented by the fine-tuning capabilities of deep Q-learning to navigate these areas precisely. The practical application of this approach is evaluated using the high-dimensional Heartbeat Categorization Dataset, focusing on the feature selection problem. This dataset, comprising complex electrocardiogram signals, provided a robust platform for testing the feature selection capabilities of our hybrid model. Our experimental results are encouraging, showcasing the hybrid strategy's capability to identify relevant features without significantly compromising the performance metrics of machine learning models. This analysis was performed by comparing the improved method of the Orca predator algorithm against its native version and a set of state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Accurate estimation of software effort and time in the software development process is a key activity to achieve the necessary product quality. However, underestimation or overestimation of effort has become a key challenge for software development. One of the main problems is the estimation with metrics from late stages, because the product must already be finished to make estimates. In this paper, the use of statistical models and machine learning approaches for software estimation are used in early stages such as software design, and a data set is presented with metric values of design artifacts with 37 software projects. As results, models for the estimation of development time and effort are proposed and validated through leave-one-out cross-validation. Further, machine learning techniques were employed in order to compare software projects estimations. Through the statistical tests, it was proven that the errors were not statistically different with the regression models for effort estimation. However, with Random Forest the best statistical results were obtained for estimating development time. [ABSTRACT FROM AUTHOR]

Many methods exist for solving the problem of evaluating efficiency in different processes. They are divided into two basic groups, parametric and non-parametric methods, which can have significant differences in the results. In this study, the authors consider the process of assessing the business climate depending on realized foreign investments. Due to the expected difference in efficiency assessment using different approaches, the goal of this paper is to create an optimization model of an ensemble for efficiency assessment that uses both types of methods with the aim of creating a symmetrical approach that achieves better results than each type of method individually. The proposed solution simultaneously analyzes the impact of different factors on foreign investments in order to determine the most important factors and thus enable each local government to ensure the best possible efficiency in this process. The innovative idea of this study is in the inclusion of classification and feature selection methods of machine learning to fulfill the set goal. Our research, focused on a specific case study in various cities across the Republic of Serbia, evaluated the effectiveness of that process. This study extends previous research and confirms the published results, highlighting the advantages of the newly proposed model. [ABSTRACT FROM AUTHOR]

The interpretability and explainability of machine learning (ML) approaches play a key role in the trustworthiness of ML models in various applications. The objective of this paper is to incorporate a logic-based reasoning in the ML model that is not only accurate but also interpretable and easily applied. More precisely, we propose a hybrid IBA-VNS approach based on interpolative Boolean algebra (IBA) and variable neighborhood search (VNS). IBA is chosen over traditional multi-valued and/or fuzzy logic techniques due to its consistency in preserving all Boolean axioms. The VNS heuristic is used for model training, i.e., determining the optimal logical aggregation function within the IBA framework for solving observed prediction problems. Obtained logic aggregation functions are easy to understand and may provide additional insight to the decision-maker. The proposed approach does not require any domain knowledge and is applicable in various domains. IBA-VNS is evaluated on several standard datasets. Further, IBA-VNS is applied to the real-world problem of predicting hospital length of stay (LOS), showing exceptional results in terms of interpretability and accuracy. In fact, the dataset is collected from the LabSerb program regarding colorectal surgeries in the period 2015–2023. The proposed approach extracted knowledge regarding the problem, i.e., the causal relations between the patient's health condition and LOS, along with achieving an MAE of 1.144 days. [ABSTRACT FROM AUTHOR]

In order to cope with ever-evolving and increasing cyber threats, intrusion detection systems have become a crucial component of cyber security. Compared with signature-based intrusion detection methods, anomaly-based methods typically employ machine learning techniques to train detection models and possess the capability to discover unknown attacks. However, intrusion detection methods face the challenge of low detection rates for minority class attacks due to imbalanced data distributions. Traditional intrusion detection algorithms address this issue by resampling or generating synthetic data. Additionally, reinforcement learning, as a machine learning method that interacts with the environment to obtain feedback and improve performance, is gradually being considered for application in the field of intrusion detection. This paper proposes a reinforcement-learning-based intrusion detection method that innovatively uses adaptive sample distribution dual-experience replay to enhance a reinforcement learning algorithm, aiming to effectively address the issue of imbalanced sample distribution. We have also developed a reinforcement learning environment specifically designed for intrusion detection tasks. Experimental results demonstrate that the proposed model achieves favorable performance on the NSL-KDD, AWID, and CICIoT2023 datasets, effectively dealing with imbalanced data and showing better classification performance in detecting minority attacks. [ABSTRACT FROM AUTHOR]

Credit risk scorecard models are utilized by lending institutions to optimize decisions on credit approvals. In recent years, ensemble learning has often been deployed to reduce misclassification costs in credit risk scorecards. In this paper, we compared the risk estimation of 26 widely used machine learning algorithms based on commonly used statistical metrics. The best-performing algorithms were then used for model selection in ensemble learning. For the first time, we proposed financial criteria that assess the impact of losses associated with both false positive and false negative predictions to identify optimal ensemble learning. The German Credit Dataset (GCD) is augmented with simulated financial information according to a hypothetical mortgage portfolio observed in UK, European and Australian banks to enable the assessment of losses arising from misclassification costs. The experimental results using the simulated GCD show that the best predictive individual algorithm with the accuracy of 0.87, Gini of 0.88 and Area Under the Receiver Operating Curve of 0.94 was the Generalized Additive Model (GAM). The ensemble learning method with the lowest misclassification cost was the combination of Random Forest (RF) and K-Nearest Neighbors (KNN), totaling USD 417 million in costs (USD 230 for default costs and USD 187 for opportunity costs) compared to the costs of the GAM (USD 487, USD 287 and USD 200). Implementing the proposed financial criteria has led to a significant USD 70 million reduction in misclassification costs derived from a small sample. Thus, the lending institutions' profit would considerably rise as the number of submitted credit applications for approval increases. [ABSTRACT FROM AUTHOR]

The power supply is crucial in the present day due to the negative impacts of poor power quality on the electric grid. In this research, we employed deep learning methods to investigate the power factor, which is a significant indicator of power quality. A multi-step forecast was developed for the power factor in the power supply installation of a hot rolling mill, extending beyond the horizontal line. This was conducted using data obtained from the respective electrical supply system. The forecast was developed via hybrid RNN (recurrent neural networks) incorporating LSTM (long short-term memory) and GRU (gated recurrent unit) layers. This research utilized hybrid recurrent neural network designs with deep learning methods to build several power factor models. These layers have advantages for time series forecasting. After conducting time series forecasting, qualitative indicators of the prediction were identified, including the sMAPE (Symmetric Mean Absolute Percentage Error) and regression coefficient. In this paper, the authors examined the quality of applied models and forecasts utilizing these indicators, both in the short term and long term. [ABSTRACT FROM AUTHOR]

Learning-based classifiers are found to be vulnerable to attacks by adversarial samples. Some works suggested that ensemble classifiers tend to be more robust than single classifiers against evasion attacks. However, recent studies have shown that this is not necessarily the case under more realistic settings of black-box attacks. In this paper, we propose a novel ensemble approach to improve the robustness of classifiers against evasion attacks by using diversified feature selection and a stochastic aggregation strategy. Our proposed scheme includes three stages. Firstly, the adversarial feature selection algorithm is used to select a feature each time that can trade-offbetween classification accuracy and robustness, and add it to the feature vector bank. Secondly, each feature vector in the bank is used to train a base classifier and is added to the base classifier bank. Finally, m classifiers from the classifier bank are randomly selected for decision-making. In this way, it can cause each classifier in the base classifier bank to have good performance in terms of classification accuracy and robustness, and it also makes it difficult to estimate the gradients of the ensemble accurately. Thus, the robustness of classifiers can be improved without reducing the classification accuracy. Experiments performed using both Linear and Kernel SVMs on genuine datasets for spam filtering, malware detection, and handwritten digit recognition demonstrate that our proposed approach significantly improves the classifiers' robustness against evasion attacks. [ABSTRACT FROM AUTHOR]

This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent's optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs). [ABSTRACT FROM AUTHOR]

Modern massive data with enormous sample size and tremendous dimensionality are usually impossible to process with a single machine. They are typically stored and processed in a distributed manner. In this paper, we propose a distributed bootstrap simultaneous inference for a high-dimensional quantile regression model using massive data. Meanwhile, a communication-efficient (CE) distributed learning algorithm is developed via the CE surrogate likelihood framework and ADMM procedure, which can handle the non-smoothness of the quantile regression loss and the Lasso penalty. We theoretically prove the convergence of the algorithm and establish a lower bound on the number of communication rounds ι min that warrant statistical accuracy and efficiency. The distributed bootstrap validity and efficiency are corroborated by an extensive simulation study. [ABSTRACT FROM AUTHOR]

Graph embedding plays an important role in the analysis and study of typical non-Euclidean data, such as graphs. Graph embedding aims to transform complex graph structures into vector representations for further machine learning or data mining tasks. It helps capture relationships and similarities between nodes, providing better representations for various tasks on graphs. Different orders of neighbors have different impacts on the generation of node embedding vectors. Therefore, this paper proposes a multi-order adjacency view encoder to fuse the feature information of neighbors at different orders. We generate different node views for different orders of neighbor information, consider different orders of neighbor information through different views, and then use attention mechanisms to integrate node embeddings from different views. Finally, we evaluate the effectiveness of our model through downstream tasks on the graph. Experimental results demonstrate that our model achieves improvements in attributed graph clustering and link prediction tasks compared to existing methods, indicating that the generated embedding representations have higher expressiveness. [ABSTRACT FROM AUTHOR]

This study introduces a framework that integrates business analytics into educational decision-making to improve learner engagement and performance in Massive Open Online Courses (MOOCs), focusing on learning environments in English as a Foreign Language (EFL). By examining three specific research questions, this paper delineates patterns in learner engagement, evaluates factors that affect these patterns, and examines the relationship between these factors and educational outcomes. The study provides an empirical analysis that elucidates the connection between learner behaviors and learning outcomes by employing machine learning, process mining, and statistical methods such as hierarchical clustering, process discovery, and the Mann–Kendall test. The analysis determines that learning patterns, characterized as single-phase or multi-phase, repetitive or non-repetitive, and sequential or self-regulated, are more closely associated with the nature of the educational content—such as books, series, or reading levels—than learner characteristics. Furthermore, it has been observed that learners exhibiting self-regulated learning patterns tend to achieve superior academic outcomes. The findings advocate for integrating analytics in educational practices, offer strategic insights for educational enhancements, and propose a new perspective on the connection between learner behavior and educational success. [ABSTRACT FROM AUTHOR]

Language barriers and the communication difficulties of individuals with developmental disabilities are two major causes of communication problems that societies worldwide encounter. A particularly challenging group is hearing-impaired people who have difficulties with communication, reading, writing, learning, and social interactions, which have a substantial impact on their quality of life. This article focuses on detailing a Serbian Sign Language alphabet database and the method for creating it in order to provide a foundation for answering the various societal challenges of persons who use the Serbian language. In front of a computer camera, 41 people performed Serbian Sign Language sign movements that replicated the Serbian alphabet for this study's aims. Hand and body key points were identified using the recorded video clips, and the numerical values of the identified key points were then stored in a database for further processing. In total, 8.346 video clips of people making recognized hand gestures were gathered, processed, classed, and archived. This paper provides a thorough technique that may be applied to comparable tasks and details the process of constructing a dataset based on Serbian Sign Language alphabet signs. This dataset was created using custom-made Python 3.11 software. Data regarding dynamic video clips that capture entire subject movement were incorporated into this dataset to fill in the gaps in other similar efforts based on static photographs. Thus, the purpose of this investigation is to employ innovative technology to support the community of hearing-impaired people in areas such as general inclusion, education, communication, and empowerment. [ABSTRACT FROM AUTHOR]

Imbalanced class data are commonly observed in pattern analysis, machine learning, and various real-world applications. Conventional approaches often resort to resampling techniques in order to address the imbalance, which inevitably alter the original data distribution. This paper proposes a novel classification method that leverages optimal transport for handling imbalanced data. Specifically, we establish a transport plan between training and testing data without modifying the original data distribution, drawing upon the principles of optimal transport theory. Additionally, we introduce a non-convex interclass regularization term to establish connections between testing samples and training samples with the same class labels. This regularization term forms the basis of a regularized discrete optimal transport model, which is employed to address imbalanced classification scenarios. Subsequently, in line with the concept of maximum minimization, a maximum minimization algorithm is introduced for regularized discrete optimal transport. Subsequent experiments on 17 Keel datasets with varying levels of imbalance demonstrate the superior performance of the proposed approach compared to 11 other widely used techniques for class-imbalanced classification. Additionally, the application of the proposed approach to water quality evaluation confirms its effectiveness. [ABSTRACT FROM AUTHOR]

This paper investigates the output feedback (OPFB) tracking control problem for discrete-time linear (DTL) systems with unknown dynamics. To solve this problem, we use an augmented system approach, which first transforms the tracking control problem into a regulation problem with a discounted performance function. The solution to this problem is derived using a Bellman equation, based on the Q-function. In order to overcome the challenges of unmeasurable system state variables, we employ a multistep Q-learning algorithm that surpasses the advantages of the policy iteration (PI) and value iteration (VI) techniques and state reconstruction methods for output feedback control. As such, the requirement for an initial stabilizing control policy for the PI method is removed and the convergence speed of the learning algorithm is improved. Finally, we demonstrate the effectiveness of the proposed scheme using a simulation example. [ABSTRACT FROM AUTHOR]

Sentiment analysis has increasingly gained significance in commercial settings, driven by the rising impact of reviews on purchase decision-making in recent years. This research conducts a thorough examination of the suitability of machine learning and deep learning approaches for sentiment analysis, using Romanian reviews as a case study, with the aim of gaining insights into their practical utility. A comprehensive, multi-level analysis is performed, covering the document, sentence, and aspect levels. The main contributions of the paper refer to the in-depth exploration of multiple sentiment analysis models at three different textual levels and the subsequent improvements brought with respect to these standard models. Additionally, a balanced dataset of Romanian reviews from twelve product categories is introduced. The results indicate that, at the document level, supervised deep learning techniques yield the best outcomes (specifically, a convolutional neural network model that obtains an AUC value of 0.93 for binary classification and a weighted average F1-score of 0.77 in a multi-class setting with 5 target classes), albeit with increased resource consumption. Favorable results are achieved at the sentence level, as well, despite the heightened complexity of sentiment identification. In this case, the best-performing model is logistic regression, for which a weighted average F1-score of 0.77 is obtained in a multi-class polarity classification task with three classes. Finally, at the aspect level, promising outcomes are observed in both aspect term extraction and aspect category detection tasks, in the form of coherent and easily interpretable word clusters, encouraging further exploration in the context of aspect-based sentiment analysis for the Romanian language. [ABSTRACT FROM AUTHOR]

Many families of binary nonlinear codes (e.g., Kerdock, Goethals, Delsarte–Goethals, Preparata) can be very simply constructed from linear codes over the Z 4 ring (ring of integers modulo 4), by applying the Gray map to the quaternary symbols. Generalized Kerdock codes represent an extension of classical Kerdock codes to the Z 2 S ring. In this paper, we develop two novel soft-input decoders, designed to exploit the unique structure of these codes. We introduce a novel soft-input ML decoding algorithm and a soft-input soft-output MAP decoding algorithm of generalized Kerdock codes, with a complexity of O (N S log 2 N) , where N is the length of the Z 2 S code, that is, the number of Z 2 S symbols in a codeword. Simulations show that our novel decoders outperform the classical lifting decoder in terms of error rate by some 5 dB. [ABSTRACT FROM AUTHOR]

This paper compares two statistical methods for parameter reconstruction (random drift and diffusion coefficients of the Itô stochastic differential equation, SDE) in the problem of stochastic modeling of air–sea heat flux increment evolution. The first method relates to a nonparametric estimation of the transition probabilities (wherein consistency is proven). The second approach is a semiparametric reconstruction based on the approximation of the SDE solution (in terms of distributions) by finite normal mixtures using the maximum likelihood estimates of the unknown parameters. This approach does not require any additional assumptions for the coefficients, with the exception of those guaranteeing the existence of the solution to the SDE itself. It is demonstrated that the corresponding conditions hold for the analyzed data. The comparison is carried out on the simulated samples, modeling the case where the SDE random coefficients are represented in trigonometric form, which is related to common climatic models, as well as on the ERA5 reanalysis data of the sensible and latent heat fluxes in the North Atlantic for 1979–2022. It is shown that the results of these two methods are close to each other in a quantitative sense, but differ somewhat in temporal variability and spatial localization. The differences during the observed period are analyzed, and their geophysical interpretations are presented. The semiparametric approach seems promising for physics-informed machine learning models. [ABSTRACT FROM AUTHOR]

An innovative coupling numerical algorithm is proposed in the current paper, the front-tracking method–lattice Boltzmann method–machine learning (FTM-LBM-ML) method, to precisely capture fluid flow phase interfaces at the mesoscale and accurately simulate dynamic processes. This method combines the distinctive abilities of the FTM to accurately capture phase interfaces and the advantages of the LBM for easy handling of mesoscopic multi-component flow fields. Taking a single vacuole rising as an example, the input and output sets of the machine learning model are constructed using the FTM's flow field, such as the velocity and position data from phase interface markers. Such datasets are used to train the Bayesian-Regularized Back Propagation Neural Network (BRBPNN) machine learning model to establish the corresponding relationship between the phase interface velocity and the position. Finally, the trained BRBPNN neural network is utilized within the multi-relaxation LBM pseudo potential model flow field to predict the phase interface position, which is compared with the FTM simulation. It was observed that the BRBPNN-predicted interface within the LBM exhibits a high degree of consistency with the FTM-predicted interface position, showing that the BRBPNN model is feasible and satisfies the accuracy requirements of the FT-LB coupling model. [ABSTRACT FROM AUTHOR]

Elevators have become an integral part of modern buildings, and technological advances have enabled the monitoring of their operational status through sensor technology. In response to the development of the elevator industry and the need for practical elevator operation risk evaluation, this paper proposes an elevator risk evaluation method based on fuzzy theory and machine learning methods. The method begins by establishing an elevator operation risk evaluation index system. The traditional fuzzy comprehensive evaluation method is then employed to evaluate the risk levels of the 50 elevators studied. The collected index data and labels (fuzzy comprehensive evaluation results) are used as inputs to train the support vector machine (SVM) model. To optimize the SVM model, the maximum information coefficient method, enhanced by the correlation-based feature selection (MIC-CFS) method, is employed to select features for the index input to the SVM model. The improved gray wolf algorithm (IGWO) method optimizes the SVM. Finally, the model's performance is verified using new index data. The experimental results demonstrate that introducing machine learning methods for elevator risk evaluation saves time and effort while providing good accuracy compared to the traditional expert evaluation method. The optimization of the SVM model by IGWO and feature selection by the MIC-CFS method results in a more concise SVM model that converges faster during training, exhibits better stability, and achieves higher accuracy. [ABSTRACT FROM AUTHOR]

Short-term traffic flow forecasting, an essential enabler for intelligent transportation systems, is a fundamental and challenging task for dramatically changing traffic flow over time. In this paper, we present a gravitational search optimized kernel extreme learning machine, named GSA-KELM, to avoid manually traversing all possible parameters to improve the potential performance. Furthermore, with the interference of heavy-tailed impulse noise, the performance of KELM may be seriously deteriorated. Based on the Kalman filter that cleverly combines observed data and estimated data to perform the closed-loop management of errors and limit the errors within a certain range, we propose a combined model, termed GSA-KELM-KF. The experimental results of two real-world datasets demonstrate that GSA-KELM-KF outperforms the state-of-the-art parametric and non-parametric models. [ABSTRACT FROM AUTHOR]

NONNEGATIVE matrix factorization (NMF) is an effective technique for dimensionality reduction of high-dimensional data for tasks such as machine learning and data visualization. However, for practical clustering tasks, traditional NMF ignores the manifold information of both the data space and feature space, as well as the discriminative information of the data. In this paper, we propose a semisupervised NMF called dual-graph-regularization-constrained nonnegative matrix factorization with label discrimination (DCNMFLD). DCNMFLD combines dual graph regularization and prior label information as additional constraints, making full use of the intrinsic geometric and discriminative structures of the data, and can efficiently enhance the discriminative and exclusionary nature of clustering and improve the clustering performance. The evaluation of the clustering experimental results on four benchmark datasets demonstrates the effectiveness of our new algorithm. [ABSTRACT FROM AUTHOR]

In practical applications, learning models that can perform well even when the data distribution is different from the training set are essential and meaningful. Such problems are often referred to as out-of-distribution (OOD) generalization problems. In this paper, we propose a method for OOD generalization based on causal inference. Unlike the prevalent OOD generalization methods, our approach does not require the environment labels associated with the data in the training set. We analyze the causes of distributional shifts in data from a causal modeling perspective and then propose a backdoor adjustment method based on variational inference. Finally, we constructed a unique network structure to simulate the variational inference process. The proposed variational backdoor adjustment (VBA) framework can be combined with any mainstream backbone network. In addition to theoretical derivation, we conduct experiments on different datasets to demonstrate that our method performs well in prediction accuracy and generalization gaps. Furthermore, by comparing the VBA framework with other mainstream OOD methods, we show that VBA performs better than mainstream methods. [ABSTRACT FROM AUTHOR]

Hybrid approaches combining machine learning with traditional inverse problem solution methods represent a promising direction for the further development of inverse modeling algorithms. The paper proposes an approach to emission source identification from measurement data for advection–diffusion–reaction models. The approach combines general-type source identification and post-processing refinement: first, emission source identification by measurement data is carried out by a sensitivity operator-based algorithm, and then refinement is done by incorporating a priori information about unknown sources. A general-type distributed emission source identified at the first stage is transformed into a localized source consisting of multiple point-wise sources. The second, refinement stage consists of two steps: point-wise source localization and emission rate estimation. Emission source localization is carried out using deep learning with convolutional neural networks. Training samples are generated using a sensitivity operator obtained at the source identification stage. The algorithm was tested in regional remote sensing emission source identification scenarios for the Lake Baikal region and was able to refine the emission source reconstruction results. Hence, the aggregates used in traditional inverse problem solution algorithms can be successfully applied within machine learning frameworks to produce hybrid algorithms. [ABSTRACT FROM AUTHOR]

In a class of large-scale distributed optimization, the calculation of RELM based on the Moore–Penrose inverse matrix is prohibitively expensive, which hinders the formulation of a computationally efficient optimization model. Attempting to improve the model's convergence performance, this paper proposes a low computing cost Alternating Direction Method of Multipliers (ADMM), where the original update in ADMM is solved inexactly with approximate curvature information. Based on quasi-Newton techniques, the ADMM approach allows us to solve convex optimization with reasonable accuracy and computational effort. By introducing this algorithm into the RELM model, the model fitting problem can be decomposed into a set of subproblems that can be executed in parallel to achieve efficient classification performance. To avoid the storage of expensive Hessian for large problems, BFGS with limited memory is proposed with computational efficiency. And the optimal parameter values of the step-size search method are obtained through Wolfe line search strategy. To demonstrate the superiority of our methods, numerical experiments are conducted on eight real-world datasets. Results on problems arising in machine learning suggest that the proposed method is competitive with other similar methods, both in terms of better computational efficiency as well as accuracy. [ABSTRACT FROM AUTHOR]