Search

The problem of a stabilisation system synthesis for the motion of a control object along a given spatial trajectory is considered. The complexity of the problem is that the preset trajectory is defined in the state subspace and not in time. This paper describes a stabilisation system synthesis for motion along a trajectory specified in time and along a trajectory specified in the form of a manifold in a state space. In order to construct a stabilisation system, it is necessary to determine a distance between an object and the given trajectory at each moment in time. For trajectories that are not given in time, the determination of this distance can be ambiguous. An object may be exactly on a trajectory but at a different time. This paper proposes some approaches to solve the problem. One of the approaches is to transform a given trajectory in a state subspace into a trajectory given in time. A description of a universal method to perform this transformation is presented. In order to solve the synthesis problem automatically, without having to analyse the mathematical model of the control object, it is suggested that machine learning control by symbolic regression is used. In computational experiments, examples of stabilisation system syntheses for quadcopter motion along a given spatial trajectory are presented. [ABSTRACT FROM AUTHOR]

Applying parallel-in-time algorithms to multiscale Hamiltonian systems to obtain stable long-time simulations is very challenging. In this paper, we present novel data-driven methods aimed at improving the standard parareal algorithm developed by Lions et al. in 2001, for multiscale Hamiltonian systems. The first method involves constructing a correction operator to improve a given inaccurate coarse solver through solving a Procrustes problem using data collected online along parareal trajectories. The second method involves constructing an efficient, high-fidelity solver by a neural network trained with offline generated data. For the second method, we address the issues of effective data generation and proper loss function design based on the Hamiltonian function. We show proof-of-concept by applying the proposed methods to a Fermi-Pasta-Ulam (FPU) problem. The numerical results demonstrate that the Procrustes parareal method is able to produce solutions that are more stable in energy compared to the standard parareal. The neural network solver can achieve comparable or better runtime performance compared to numerical solvers of similar accuracy. When combined with the standard parareal algorithm, the improved neural network solutions are slightly more stable in energy than the improved numerical coarse solutions. [ABSTRACT FROM AUTHOR]

The rapidly advancing fields of machine learning and mathematical modeling, greatly enhanced by the recent growth in artificial intelligence, are the focus of this special issue. This issue compiles extensively revised and improved versions of the top papers from the workshop on Mathematical Modeling and Problem Solving at PDPTA'23, the 29th International Conference on Parallel and Distributed Processing Techniques and Applications. Covering fundamental research in matrix operations and heuristic searches to real-world applications in computer vision and drug discovery, the issue underscores the crucial role of supercomputing and parallel and distributed computing infrastructure in research. Featuring nine key studies, this issue pushes forward computational technologies in mathematical modeling, refines techniques for analyzing images and time-series data, and introduces new methods in pharmaceutical and materials science, making significant contributions to these areas. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel approach to solving theorem proving problems without relying on any deduction method. We transform logical formulas into numbers, vectors and matrices, and feed the corresponding data into various machine learning algorithms to predict their satisfiability. Here, we introduce ProverX, a novel theorem prover that utilizes various binary classification algorithms, ranging from traditional machine learning to deep learning, to tackle the satisfiability (SAT) problem of propositional logic. Empirical experiments were conducted to evaluate the performance of ProverX using datasets we generated. ProverX achieved accuracy rates ranging from 81.8% to 98.7%, demonstrating a remarkable speedup of almost 180 times compared to CTL-RP, a highly efficient prover. These results demonstrate the feasibility of replacing deduction with learning in theorem proving, opening promising avenues for further exploration in more complex logics, e.g., Modal Logic, Coalition Logic, Propositional Linear-Time Temporal Logic, Computation Tree Logic, etc., provided that their resolution methods exist. We also present a detailed analysis of the best-performing machine learning approaches for this task and introduce a new algorithm, IPDA, which has the potential to further enhance performance. [ABSTRACT FROM AUTHOR]

Data-driven fault diagnosis method is widely used in the field of engine health management, which uses engine sensor data as input and engine faulty components as output for component-level fault diagnosis of the engine. The application premise of the general data-driven fault diagnosis method is that all data come from the same working conditions, that is, they belong to the same distribution. However, this assumption is not valid in the actual engine fault diagnosis, because the engine state will change with the increase of running time. In the meantime, collecting engine data is usually expensive, time-consuming, and laborious. To solve these problems, extreme learning machine (ELM)-based two transfer learning methods for fault diagnosis of turboshaft engines are proposed in this paper. One is joint solving ELM (JSELM), which regards the information of the target domain and source domain as similar and different parts, respectively, and knowledge is extracted from them at the same time. The other is model transfer-based ELM (MTELM), which uses the idea of pretraining. First, a general ELM classifier is trained with the source domain data and then fine-tuned with the target domain data. Both methods have a good real-time performance as the traditional ELM. When there are a few data in the target domain, they achieve much better classification accuracy than traditional ELM. Finally, experiments are carried out with turboshaft engine simulation data. The results show that both methods are effective, especially MTELM, which has better classification accuracy. [ABSTRACT FROM AUTHOR]

Network data is ubiquitous, such as telecommunication, transport systems, online social networks, protein-protein interactions, etc. Since the huge scale and the complexity of network data, former machine learning system tried to understand network data arduously. On the other hand, thought of multi-granular cognitive computation simulates the problem-solving process of human brains. It simplifies the complex problems and solves problems from the easier to harder. Therefore, the application of multi-granularity problem-solving ideas or methods to deal with network data mining is increasingly adopted by researchers either intentionally or unintentionally. This paper looks into the domain of network representation learning (NRL). It systematically combs the research work in this field in recent years. In this paper, it is discovered that in dealing with the complexity of the network and pursuing the efficiency of computing resources, the multi-granularity solution becomes an excellent path that is hard to go around. Although there are several papers about survey of NRL, to our best knowledge, we are the first to survey the NRL from the perspective of multi-granular computing. This paper proposes the challenges that NRL meets. Furthermore, the feasibility of solving the challenges of NRL with multi-granular computing methodologies is analyzed and discussed. Some potential key scientific problems are sorted out and prospected in applying multi-granular computing for NRL research. [ABSTRACT FROM AUTHOR]

This article introduces a special issue on machine learning (ML) applications in geophysical exploration and monitoring. ML techniques have shown promise in streamlining workflows, extracting insights from large datasets, and solving computational problems in geophysics. The special issue includes 17 papers covering a range of geophysical applications, such as seismic data processing, electromagnetic inversion, and well logging. These papers demonstrate the current state of ML applications in geophysics and highlight challenges and potential directions for future research. The hope is that this special issue will serve as a valuable resource for geophysicists interested in ML techniques and their applications in geophysics. [Extracted from the article]

Since 2007, methods that utilize side-channel data to detect hardware Trojan (HT) problems have been widely studied. Machine learning methods are widely used for hardware Trojan detection, but with the development of integrated circuits (ICs), better results are usually obtained using deep learning methods. In this paper, we propose an architecture inspired by Residual-Block and Dense-Block and combine it with SE Attention Mechanism, which we named the Res-Dense-SE-Net network. By combining residual connectivity, dense connectivity, and attention mechanism, the Res-Dense-SE-Net network can enjoy the advantages of these three network architectures at the same time, which can improve the expressiveness and performance of the model. The Res-Dense-SE-Net network can capture the key features in the image better, and it can solve the problems of gradient vanishing and feature transfer efficiently, which can in turn improve the classification accuracy and the generalization ability of the model. Based on the publicly available AES series of hardware Trojans from TrustHub and the publicly available hardware Trojan-side channel data by Faezi et al., we evaluate the effectiveness of the method proposed in this paper. The experimental results show that when a single Trojan exists, the method proposed in this paper has a high accuracy rate; and when multiple types of hardware Trojans exist at the same time and need to be categorized, the categories of hardware Trojans can also be effectively identified, and the categorization accuracy is high compared with the existing deep learning methods. [ABSTRACT FROM AUTHOR]

Many methods exist for solving the problem of evaluating efficiency in different processes. They are divided into two basic groups, parametric and non-parametric methods, which can have significant differences in the results. In this study, the authors consider the process of assessing the business climate depending on realized foreign investments. Due to the expected difference in efficiency assessment using different approaches, the goal of this paper is to create an optimization model of an ensemble for efficiency assessment that uses both types of methods with the aim of creating a symmetrical approach that achieves better results than each type of method individually. The proposed solution simultaneously analyzes the impact of different factors on foreign investments in order to determine the most important factors and thus enable each local government to ensure the best possible efficiency in this process. The innovative idea of this study is in the inclusion of classification and feature selection methods of machine learning to fulfill the set goal. Our research, focused on a specific case study in various cities across the Republic of Serbia, evaluated the effectiveness of that process. This study extends previous research and confirms the published results, highlighting the advantages of the newly proposed model. [ABSTRACT FROM AUTHOR]

Stochastic Gradient Descent (SGD) is one of the many iterative optimization methods that are widely used in solving machine learning problems. These methods display valuable properties and attract researchers and industrial machine learning engineers with their simplicity. However, one of the weaknesses of this type of methods is the necessity to tune learning rate (step-size) for every loss function and dataset combination to solve an optimization problem and get an efficient performance in a given time budget. Stochastic Gradient Descent with Polyak Step-size (SPS) is a method that offers an update rule that alleviates the need of fine-tuning the learning rate of an optimizer. In this paper, we propose an extension of SPS that employs preconditioning techniques, such as Hutchinson's method, Adam, and AdaGrad, to improve its performance on badly scaled and/or ill-conditioned datasets. [ABSTRACT FROM AUTHOR]

Due to advancements in data collection, storage, and processing techniques, machine learning has become a thriving and dominant paradigm. However, one of its main shortcomings is that the classical machine learning paradigm acts in isolation without utilizing the knowledge gained through learning from related tasks in the past. To circumvent this, the concept of Lifelong Machine Learning (LML) has been proposed, with the goal of mimicking how humans learn and acquire cognition. Human learning research has revealed that the brain connects previously learned information while learning new information from a single or small number of examples. Similarly, an LML system continually learns by storing and applying acquired information. Starting with an analysis of how the human brain learns, this paper shows that the LML framework shares a functional structure with the brain when it comes to solving new problems using previously learned information. It also provides a description of the LML framework, emphasizing its similarities to human brain learning. It also provides citation graph generation and scientometric analysis algorithms for the LML literatures, including information about the datasets and evaluation metrics that have been used in the empirical evaluation of LML systems. Finally, it presents outstanding issues and possible future research directions in the field of LML. [ABSTRACT FROM AUTHOR]

In recent years, semantic segmentation algorithms based on deep learning have been widely used in building extraction, which requires large sample data and does not consider the geometric features of the building, and the effect of the extraction is greatly affected by the data scene, while the traditional methods are difficult to extract the remote sensing buildings accurately because they only consider their greyscale features when extracting them. To solve this problem, we propose a method for building classification extraction from remote sensing images that combine over-pixel and maximum interclass variance. The method combines superpixel and maximum interclass variance (OTSU). First, a number of superpixel subregions with different shapes and sizes are generated based on the watershed transform. Then, the superpixels of buildings are merged using the spectral features of buildings, so the first extraction of buildings is achieved by this method.Then, the noise is suppressed with median filtering. Finally, the post-extraction of buildings is performed according to the OTSU algorithm. In this paper, seven images of buildings located in different landscapes were selected. The experimental results show that the algorithm is more advantageous than the classical algorithm and the deep learning algorithm.. [ABSTRACT FROM AUTHOR]

In numerous scientific disciplines and practical applications, addressing optimization challenges is a common imperative. Nature-inspired optimization algorithms represent a highly valuable and pragmatic approach to tackling these complexities. This paper introduces Dendritic Growth Optimization (DGO), a novel algorithm inspired by natural branching patterns. DGO offers a novel solution for intricate optimization problems and demonstrates its efficiency in exploring diverse solution spaces. The algorithm has been extensively tested with a suite of machine learning algorithms, deep learning algorithms, and metaheuristic algorithms, and the results, both before and after optimization, unequivocally support the proposed algorithm's feasibility, effectiveness, and generalizability. Through empirical validation using established datasets like diabetes and breast cancer, the algorithm consistently enhances model performance across various domains. Beyond its working and experimental analysis, DGO's wide-ranging applications in machine learning, logistics, and engineering for solving real-world problems have been highlighted. The study also considers the challenges and practical implications of implementing DGO in multiple scenarios. As optimization remains crucial in research and industry, DGO emerges as a promising avenue for innovation and problem solving. [ABSTRACT FROM AUTHOR]

This paper investigates the output feedback (OPFB) tracking control problem for discrete-time linear (DTL) systems with unknown dynamics. To solve this problem, we use an augmented system approach, which first transforms the tracking control problem into a regulation problem with a discounted performance function. The solution to this problem is derived using a Bellman equation, based on the Q-function. In order to overcome the challenges of unmeasurable system state variables, we employ a multistep Q-learning algorithm that surpasses the advantages of the policy iteration (PI) and value iteration (VI) techniques and state reconstruction methods for output feedback control. As such, the requirement for an initial stabilizing control policy for the PI method is removed and the convergence speed of the learning algorithm is improved. Finally, we demonstrate the effectiveness of the proposed scheme using a simulation example. [ABSTRACT FROM AUTHOR]

The moth-flame algorithm shows some shortcomings in solving the complex problem of optimization, such as insufficient population diversity and unbalanced search ability. In this paper, an IMFO (Improved Moth-Flame Optimization) algorithm is proposed to be applied in solving the optimization problem of function. First, cat chaotic mapping is used to generate the initial position of moth to improve the population diversity. Second, cosine inertia weight is introduced to balance the global and local search abilities of the algorithm. Third, the memory information in the particle swarm algorithm is introduced into the iterative process of the algorithm to speed up the convergence of the population. Finally, Gaussian mutation strategy is used in the current optimal solution to avoid the algorithm from falling into the local optimum. Simulation experiments are conducted on 11 benchmark test functions, compared with other improved MFO (Moth-Flame Optimization) algorithms and classical optimization algorithms. The results show that the IMFO has higher accuracy and stability in solving the above-mentioned test functions. The proposed algorithm is experimented and verified by optimizing the KELM (Kernel Extreme Learning Machine) in an engineering example and exhibits a better optimization performance. [ABSTRACT FROM AUTHOR]

As an important part of the bridge structure system, the underwater pile-pier structure usually occurs various defects on its surfaces due to its complex hydrological environment. The existing conventional defect detection approaches exist two aspects of problems: (1) insufficient definition and color distortion of the underwater images, and (2) low efficiency and error-prone. To solve these problems, this paper proposed the target defect detection model by integrating the image-fusion enhancement algorithm and the deep learning algorithm. Firstly, by analyzing the reasons for the degradation of the underwater images, the ACE (automatic color equalization) and CLAHE (contrast limited adaptive histogram equalization) algorithms are selected to enhance the image, respectively. Secondly, the two enhanced images are fused based on the point sharpness weight, and then the fusion results are further sharpened by the USM (unsharp mask) algorithm, thus obtaining the final fused images. Thirdly, 3,200 fused images are taken as the training set, by adopting the YOLOv3 algorithm to train the detection model, and then the training model is validated and tested by the other each 400 fused images, thus building up the target automatic detection model of underwater pile-pier surface defects. Finally, a series of comparison and discussion were conducted to validate the effectiveness of image-fusion and the robustness and effectiveness of the target detection model. The results found that the target detection model has excellent robustness against noise and effectiveness in the surface defect detection. This indicates that the image-fusion approach proposed in this paper can effectively enhance the image features, and the target detection model is feasible, robust, and effective in the automatic detection of surface defects on underwater pile-pier structures. [ABSTRACT FROM AUTHOR]

The novelty of additive manufacturing (AM) involves new requirements, restrictions and rules, that are considerably different to those of conventional manufacturing methods. Therefore, designers often lack experience and knowledge about identifying AM-suited components. However, to ensure profitability, it is essential to choose components, that are well suited for additive manufacturing. State-of-the-art user-support methods for identifying AM potential widely focus on either economic potential or manufacturability but fail to address both aspects. While machine learning solutions are considered highly efficient, the assessment outcome lacks process transparency, inhibiting troubleshooting, plausibility checks and problem-oriented considerations. This paper proposes a holistic, yet detailed and transparent approach to identify conventionally manufactured parts for AM from an existing product portfolio, enabling decision-making based on quantifiable results. It combines and advances state-of-the-art methods, considering manufacturability, economic profitability and socio-ecological aspects. Besides evaluating AM potential, the method additionally assesses the components' potential for re-design-based enhancement for AM suitability. Besides understanding product functions and present production processes, users are expected to have a basic understanding of company goals. The approach involves inquiries regarding company- and product-specific priorities, enabling a weighted assessment. The weights are determined based on individual company philosophies regarding AM value propositions such as differing stakeholder interests and priorities. Additionally, the approach allows users to investigate different development goals by weighting opportunistic and restrictive assessment. The method application is demonstrated via an assembly comprising 11 parts in a scenario focusing on serviceability, eventually determining suitability statements. [ABSTRACT FROM AUTHOR]

In the process of facial expression recognition, face detection is the prerequisite, image preprocessing is the foundation, facial expression feature extraction is the key, and facial expression classification is the target. Effective feature extraction in this process can improve the accuracy of facial expression classifications. On the other hand, traditional facial expression recognition methods are not only complicated in the feature extraction process, but also unable to obtain more in-depth high-semantic features and deep features from the original image. To solve the above problems, this paper proposes a facial expression recognition method based on multi-channel fusion and lightweight neural network. First, a cascade classifier based on Haar features is used to detect the face region of the facial expression image. Second, the local binary pattern (LBP) is used to extract the local texture features from the face region. Third, face edge features are simultaneously obtained by performing edge detection in the face region based on the Canny edge detection algorithm. Fourth, the obtained face image, LBP texture feature image, and edge detection Canny image are fused, and the fused image is input into the constructed lightweight neural network for training and recognition. Experiments are carried out on the public image databases Facial Expression Recognition 2013 (Fer2013) and extended Cohn–Kanade (CK +) using the hold-out cross-validation method. The experimental results show that the proposed method effectively extracts more complete image features by combining traditional feature extraction algorithms with deep learning feature extraction algorithms, improves the accuracy and robustness of facial expression recognition, and has better recognition rate and generalization ability compared to other mainstream methods. [ABSTRACT FROM AUTHOR]

Machine learning (ML) is evolving rapidly and has made many theoretical breakthroughs while widely applied in various fields. ML allows systems the ability to access data and use it to enable computers to execute cognitive processes such as learning and improving from previous experiences and solving complicated issues. Many first-order stochastic optimization methods have been used to solve the optimization model of ML. These algorithms adopt Barzilai-Borwein (BB) step size instead of fixed or diminishing step size to improve performance. However, the BB step size format involves fractional calculation, which inevitably leads to a zero denominator, especially when the objective function is non-convex. The BB technique will be violated if the denominator is near 0 or even negative. To improve the computation of the step size, a Positive Defined Stabilized Barzilai-Borwein (PDSBB) approach is introduced in this paper. Integrating PDSBB with the stochastic variance reduced gradient (SVRG) approach, a new method SVRG-PDSBB is proposed. Numerical experiments have shown that the new algorithm has stabilized the performance of the new step size, which successfully avoiding zero denominators and effectively solving the common problems in machine learning. The convergence of SVRG-PDSBB is theoretically and numerically proven, and the effectiveness of the new algorithm is shown by comparison with other algorithms. [ABSTRACT FROM AUTHOR]

Production problems have a significant impact on the on-time delivery of orders, resulting in deviations from planned scenarios. Therefore, it is crucial to predict interruptions during scheduling and to find optimal production sequencing solutions. This paper introduces a selflearning framework that integrates association rules and optimisation techniques to develop a scheduling algorithm capable of learning from past production experiences and anticipating future problems. Association rules identify factors that hinder the production process, while optimisation techniques use mathematical models to optimise the sequence of tasks and minimise execution time. In addition, association rules establish correlations between production parameters and success rates, allowing corrective factors for production quantity to be calculated based on confidence values and success rates. The proposed solution demonstrates robustness and flexibility, providing efficient solutions for Flow-Shop and Job-Shop scheduling problems with reduced calculation times. The article includes two Flow-Shop and Job-Shop examples where the framework is applied. [ABSTRACT FROM AUTHOR]

Imbalanced data classification has received much attention in machine learning, and many oversampling methods exist to solve this problem. However, these methods may suffer from insufficient noise filtering, overlap between synthetic and original samples, etc., resulting in degradation of classification performance. To this end, we propose a hybrid sampling with two-step noise filtering (HSNF) method in this paper, which consists of three modules. In the first module, HSNF denoises twice according to different noise discrimination mechanisms. Note that denoising mechanism is essentially based on the Euclidean distance between samples. Then in the second module, the minority class samples are divided into two categories, boundary samples and safe samples, respectively, and a portion of the boundary majority class samples are removed. In the third module, different oversampling methods are used to synthesize instances for boundary minority class samples and safe minority class samples. Experimental results on synthetic data and benchmark datasets demonstrate the effectiveness of HSNF in comparison with several popular methods. The code of HSNF will be released. [ABSTRACT FROM AUTHOR]

The interaction between machine learning and quantum physics has given rise to an emerging frontier of quantum machine learning research. In this line, quantum classifiers have received significant attention recently as a quantum device designed to solve the classification problem in machine learning. In this paper, we propose a new variational quantum multi-class classifier that uses l o g 2 N qubits to represent N labels, converts the labels into different quantum states, and optimizes the circuit parameters by the fidelity between the true label state and the output state. Our method effectively reduces the width of the circuit and lowers the number of auxiliary particles needed from N to l o g 2 N . We conducted simulation experiments on several datasets. On the MNIST handwritten digits dataset, we achieved 99.8% accuracy for 4 classifications and 97% for 8 classifications. On the CIFAR-10 dataset, we obtained 85.3% accuracy for 8 classifications. Finally, on the CIFAR-100 dataset, we reached 76% accuracy for 16 classifications. [ABSTRACT FROM AUTHOR]

Online portfolio optimization with transaction costs is a big challenge in large-scale intelligent computing community, since its undersample from rapidly-changing market and complexity from varying transaction costs. In this paper, we focus on this problem and solve it by machine learning system. Specifically, we reformulate the optimization problem with the minimization over simplex containing three items, which are negative expected return, the elastic net regularization of transaction costs controlled term and portfolio variable, respectively. We propose to apply linearized augmented Lagrangian method (LALM) and the alternating direction method of multipliers (ADMM) to solve the optimization model in a higher efficiency, meanwhile theoretically guarantee their convergence and deduce closed-form solutions of their subproblems in each iteration. Furthermore, we conduct extensive experiments on five benchmark datasets from real market to demonstrate that the proposed algorithms outperform compared state-of-the-art strategies in most cases in six dimensions. [ABSTRACT FROM AUTHOR]

The widespread use of the internet and the exponential growth in small hardware diversity enable the development of Internet of things (IoT)-based localization systems. We review machine-learning-based approaches for IoT localization systems in this paper. Because of their high prediction accuracy, machine learning methods are now being used to solve localization problems. The paper's main goal is to provide a review of how learning algorithms are used to solve IoT localization problems, as well as to address current challenges. We examine the existing literature for published papers released between 2020 and 2022. These studies are classified according to several criteria, including their learning algorithm, chosen environment, specific covered IoT protocol, and measurement technique. We also discuss the potential applications of learning algorithms in IoT localization, as well as future trends. [ABSTRACT FROM AUTHOR]

The autonomous underwater vehicle (AUV) has a problem with feature loss when recognizing small targets underwater. At present, algorithms usually use multi-scale feature extraction to solve the problem, but this method increases the computational effort of the algorithm. In addition, low underwater light and turbid water result in incomplete information on target features. This paper proposes an enhanced dilated convolution framework (EHDC) for underwater blurred target recognition. Firstly, this paper extracts small target features through hybrid dilated convolution networks, increasing the perceptive field of the algorithm without increasing the computational power of the algorithm. Secondly, the proposed algorithm learns spatial semantic features through an adaptive correlation matrix and compensates for the missing features of the target. Finally, this paper fuses spatial semantic features and visual features for the recognition of small underwater blurred targets. Experiments show that the proposed method improves the recognition accuracy by 1.04% compared to existing methods when recognizing small underwater blurred targets. [ABSTRACT FROM AUTHOR]

Clustering is widely used in data mining and machine learning. The possibilistic c-means clustering (PCM) method loosens the constraint of the fuzzy c-means clustering (FCM) method to solve the problem of noise sensitivity of FCM. But there is also a new problem: overlapping cluster centers are not suitable for clustering non-cluster distribution data. We propose a novel possibilistic c-means clustering method based on the nearest-neighbour isolation similarity in this paper. All samples are taken as the initial cluster centers in the proposed approach to obtain k sub-clusters iteratively. Then the first b samples farthest from the center of each sub-cluster are chosen to represent the sub-cluster. Afterward, sub-clusters are mapped to the distinguishable space by using these selected samples to calculate the nearest-neighbour isolation similarity of the sub-clusters. Then, adjacent sub-clusters can be merged according to the presented connecting strategy, and finally, C clusters are obtained. Our method proposed in this paper has been tested on 15 UCI benchmark datasets and a synthetic dataset. Experimental results show that our proposed method is suitable for clustering non-cluster distribution data, and the clustering results are better than those of the comparison methods with solid robustness. [ABSTRACT FROM AUTHOR]

In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application. [ABSTRACT FROM AUTHOR]

The traditional parallel Sparse matrix vector multiplication (SpMV) method has been optimized by an application-specific or compression format-specific. However, a single compression format cannot deal with all sparse matrix types effectively in practical applications. To solve this problem, an adaptive compression format based on Bagging ensemble learning algorithm is proposed in this paper. Experiments show that the adaptive compression format has higher prediction and computational performance on NVIDIA V100 and NVIDIA RTX 2080Ti. Compared with SpMV of the four compression formats, SpMV based on adaptive compression format reduces the execution time of 1.5×, 6.6×, 9× and 1.1×, respectively. [ABSTRACT FROM AUTHOR]

Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an ε -stationary point of the objective function in O ε - 2 (resp. O ε - 4 ) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an ε -stationary point of the objective function is bounded by O ε - 2 (resp., O ε - 4 ) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings. [ABSTRACT FROM AUTHOR]

Federated learning is gaining significant interests as it enables model training over a large volume of data that is distributedly stored over many users. However, Malicious or dishonest aggregator still possible to infer sensitive information and even restore private data from local model updates even destroy the process of training. To solve the problem, researchers have proposed many excellent methods based on privacy protection technologies, such as secure multiparty computation (MPC), homomorphic encryption (HE), and differential privacy. But these methods don't only ignore users' address and identity privacy, but also include nothing about a feasible scheme to trace malicious users and malicious gradients. In this paper, we propose a general decentralized byzantine-fault-tolerant federated learning protocol, named TFPA, which can integrate multiple learning algorithms. This protocol can not only ensure the accuracy of aggregation under the adversary setting of 4 f + 1 , but also provide user address privacy and identity privacy assurance. In addition, we also provide a heuristic malicious gradient discovery and tracking scheme to help participants better resist malicious gradients and ensure the fairness of aggregation to a certain extent. We evaluate our framework on Linear Regression, Logistic Regression, SVM, MLP and RNN, and attain good results both in accuracy and performance. Last but not least, we also simply prove the correction and security of TFPA. [ABSTRACT FROM AUTHOR]

Machine learning (ML) has been recently introduced to solving optimization problems, especially for combinatorial optimization (CO) tasks. In this paper, we survey the trend of leveraging ML to solve the mixed-integer programming problem (MIP). Theoretically, MIP is an NP-hard problem, and most CO problems can be formulated as MIP. Like other CO problems, the human-designed heuristic algorithms for MIP rely on good initial solutions and cost a lot of computational resources. Therefore, researchers consider applying machine learning methods to solve MIP since ML-enhanced approaches can provide the solution based on the typical patterns from the training data. Specifically, we first introduce the formulation and preliminaries of MIP and representative traditional solvers. Then, we show the integration of machine learning and MIP with detailed discussions on related learning-based methods, which can be further classified into exact and heuristic algorithms. Finally, we propose the outlook for learning-based MIP solvers, the direction toward more combinatorial optimization problems beyond MIP, and the mutual embrace of traditional solvers and ML components. We maintain a list of papers that utilize machine learning technologies to solve combinatorial optimization problems, which is available at https://github.com/Thinklab-SJTU/awesome-ml4co. [ABSTRACT FROM AUTHOR]

Originally, tangles were invented as an abstract tool in mathematical graph theory to prove the famous graph minor theorem. In this paper, we showcase the practical potential of tangles in machine learning applications. Given a collection of cuts of any dataset, tangles aggregate these cuts to point in the direction of a dense structure. As a result, a cluster is softly characterized by a set of consistent pointers. This highly flexible approach can solve clustering problems in various setups, ranging from questionnaires over community detection in graphs to clustering points in metric spaces. The output of our proposed framework is hierarchical and induces the notion of a soft dendrogram, which can help explore the cluster structure of a dataset. The computational complexity of aggregating the cuts is linear in the number of data points. Thus the bottleneck of the tangle approach is to generate the cuts, for which simple and fast algorithms form a sufficient basis. In our paper we construct the algorithmic framework for clustering with tangles, prove theoretical guarantees in various settings, and provide extensive simulations and use cases. Python code is available on github. [ABSTRACT FROM AUTHOR]

This paper provides a comprehensive review of the application of Reinforcement Learning (RL) in the domain of finance, shedding light on the groundbreaking progress achieved andthe challenges that lie ahead. We explore how RL, a subfield of machine learning, has been instrumental in solving complex financial problems by enabling decision-making processes that optimize long-term rewards. Reinforcement learning (RL) is a powerful machinelearning technique that can be used to train agents to make decisions in complex environments. In finance, RL has been used to solve a variety of problems, including optimal execution, portfolio optimization, option pricing and hedging, market making, smart order routing, and robo-advising. In this paper, we review the recent developments in RL for finance. We begin by introducing RL and Markov decision processes (MDPs), which is the mathematical framework for RL. We then discuss the various RL algorithms that have beenused in finance, with a focus on value-based and policy-based methods. We also discuss the use of neural networks in RL for finance. Finally, we discuss the results of recent studies that have used RL to solve financial problems. We conclude by discussing the challenges and opportunities for future research in RL for finance. [ABSTRACT FROM AUTHOR]

Engineers strive to realize the goal of control in the best possible way based on a given quality criterion when developing control system. The well-known optimal control problem requires finding a solution as a function of time. Such a function cannot be directly used to control a real object because its application corresponds to an open-loop control system and engineers usually complete it with a feedback stabilization system. Hence, there is an obvious need to reformulate the optimal control problem so that its solution can be directly applied to real objects. The paper presents a new extended statement of the optimal control problem. An additional requirement for the control function is introduced to give the system describing the control object properties that will ensure the stability of solutions. The desired control function must provide for the optimal trajectory given the properties of the attractor in the neighbourhood. The solution to the extended optimal control problem can be directly used to control a real object. The paper presents a computational machine learning approach to solving the extended problem of optimal control based on the application of a synthesized optimal control technique. Examples of the practical solution to the stated problem are given to illustrate the efficiency of the approach, where the solution to the conventional optimal control problem is compared with the proposed extended one in the presence of perturbations in models and initial conditions. [ABSTRACT FROM AUTHOR]

This survey is devoted to the task of network traffic classification, specifically, to the use of machine learning algorithms in this task. The survey begins with the description of the task, its different statements, and possible real-world applications. It then proceeds to the description of the methods historically used for network traffic classification, as well as their limitations and evolution of traffic making machine learning, which is the main way to solve the problem. The most popular machine learning algorithms used in this task are described and accompanied with examples of research papers that provide insight into their advantages and disadvantages. The problem of feature selection is discussed with subsequent consideration of a more global problem of acquiring a suitable dataset for network traffic classification; examples of popular datasets and their descriptions are provided. The paper concludes with an overview of some current problems in this field of research: model training and comparison, user data protection, and network traffic volatility. [ABSTRACT FROM AUTHOR]

Due to the heterogeneity of program language and natural language query, it is difficult to identify the semantic relationship between them, which leads to the low efficiency of code recommendation. In order to solve the problems of the above code recommendation technology, a code recommendation method based on joint embedded attention network (JEAN) is proposed in this paper. The method uses GRU Network to embed code snippets and describe queries into vector representation, which solves the problem of heterogeneous code snippets and natural language queries. The Attention mechanism is then used to distribute totally different weights to different components of every mode of the code snippet. The reason for the Attention mechanism is that different components of every mode of the code snippet contribute differently to the semantic vector of the final code snippet, making it interpretable. Finally, two commonly used evaluation indexes of information retrieval, SuccessRate@k and MRR, are used for experimental comparison with other baseline models. The experimental results show that the code recommendation method based on joint embedded attention network proposed in this paper can effectively recommend appropriate code snippets according to the needs of developers, and its performance is better than other baseline methods. [ABSTRACT FROM AUTHOR]

With the abundance of data, machine learning applications engaged increased attention in the last decade. An attractive approach to robustify the statistical analysis is to preprocess the data through clustering. This paper develops a low-complexity Riemannian optimization framework for solving optimization problems on the set of positive semidefinite stochastic matrices. The low-complexity feature of the proposed algorithms stems from the factorization of the optimization variable $\mathbf {X}=\mathbf {Y}\mathbf {Y}^{\mathrm{T}}$ X = Y Y T and deriving conditions on the number of columns of $\mathbf {Y}$ Y under which the factorization yields a satisfactory solution. The paper further investigates the embedded and quotient geometries of the resulting Riemannian manifolds. In particular, the paper explicitly derives the tangent space, Riemannian gradients and Hessians, and a retraction operator allowing the design of efficient first and second-order optimization methods for the graph-based clustering applications of interest. The numerical results reveal that the resulting algorithms present a clear complexity advantage as compared with state-of-the-art euclidean and Riemannian approaches for graph clustering applications. [ABSTRACT FROM AUTHOR]

Routing is one of the most important and time-consuming stages of physical design. As the process node of semiconductors keeps scaling down, the routing process faces increasing challenges, and the traditional solutions are not sufficiently efficient. In recent years, machine learning has aroused much interest in this context, and an increasing number of algorithms have introduced advanced machine learning techniques to help solve the routing problem. In this paper, we survey the recent development of machine learning-based routing algorithms. • This paper provides an overview of recent published machine learning-based routing algorithms. • Some trends on the development of machine learning-based routing algorithms are revealed. • The possible advantages and disadvantages on machine learning-based routing methods are discussed. [ABSTRACT FROM AUTHOR]

Facial expression recognition based on residual networks is important for technologies related to space human-robot interaction and collaboration but suffers from low accuracy and slow computation in complex network structures. To solve these problems, this paper proposes a multiscale feature fusion attention lightweight wide residual network. The network first uses an improved random erasing method to preprocess facial expression images, which improves the generalizability of the model. The use of a modified depthwise separable convolution in the feature extraction network reduces the computational effort associated with the network parameters and enhances the characterization of the extracted features through a channel shuffle operation. Then, an improved bottleneck block is used to reduce the dimensionality of the upper layer network feature map to further reduce the number of network parameters while enhancing the network feature extraction capability. Finally, an optimized multiscale feature lightweight attention mechanism module is embedded to further improve the feature extractability of the network for human facial expressions. The experimental results show that the accuracy of the model is 73.21%, 98.72%, and 95.21% on FER2013, CK+ and JAFFE, respectively, with a covariance of 10.14 M. Compared with other networks, the model proposed in this paper has faster computing speed and better accuracy at the same time. [ABSTRACT FROM AUTHOR]

This paper overviews and discusses the relationship between Reinforcement Learning (RL) and the recently developed Dual Control for Exploitation and Exploration (DCEE). It is argued that there are two related but quite distinctive approaches, namely, control and machine learning, in tackling intractability arising in optimal decision making/control problems. In the control approach, the original problems (of an infinite horizon) are approximated by finite horizon problems and solved online by taking advantage of the availability of computing power. In the machine learning approach, the optimal solutions are approximated through iterations, or (offline) training through trials when models are not available. When dealing with unknown environments, DCEE as a technique developed from the control approach could potentially solve similar problems as RL while offering a number of advantages, most notably, coping with uncertainty in environment/tasks, high efficiency in learning through balancing exploitation and exploration, and potential in establishing its formal properties like stability. The links between DCEE and other relevant methods like dual control, Model Predictive Control and particularly Active Inference in neuroscience are discussed. The latter provides a strong biological endorsement for DCEE. The methods and discussions are illustrated by autonomous source search using a robot. It is concluded that DCEE provides a promising, complementary approach to RL, and more research is required to develop it as a generic theory and fully realise its potential. The relationships revealed in this paper provide insights into these relevant methods and facilitate cross fertilisation between control, machine learning and neuroscience for developing autonomous control under uncertain environments. [ABSTRACT FROM AUTHOR]

Background: In many applications of bioinformatics, data stem from distinct heterogeneous sources. One of the well-known examples is the identification of drug–target interactions (DTIs), which is of significant importance in drug discovery. In this paper, we propose a novel framework, manifold optimization based kernel preserving embedding (MOKPE), to efficiently solve the problem of modeling heterogeneous data. Our model projects heterogeneous drug and target data into a unified embedding space by preserving drug–target interactions and drug–drug, target–target similarities simultaneously. Results: We performed ten replications of ten-fold cross validation on four different drug–target interaction network data sets for predicting DTIs for previously unseen drugs. The classification evaluation metrics showed better or comparable performance compared to previous similarity-based state-of-the-art methods. We also evaluated MOKPE on predicting unknown DTIs of a given network. Our implementation of the proposed algorithm in R together with the scripts that replicate the reported experiments is publicly available at https://github.com/ocbinatli/mokpe. [ABSTRACT FROM AUTHOR]

In the application of Bayesian networks to solve practical problems, it is likely to encounter the situation that the data set is expensive and difficult to obtain in large quantities and the small data set is easy to cause the inaccuracy of Bayesian network (BN) scoring functions, which affects the BN optimization results. Therefore, how to better learn Bayesian network structures under a small data set is an important problem we need to pay attention to and solve. This paper introduces the idea of parallel ensemble learning and proposes a new hybrid Bayesian network structure learning algorithm. The algorithm adopts the elite-based structure learner using genetic algorithm (ESL-GA) as the base learner. Firstly, the adjacency matrices of the network structures learned by ESL-GA are weighted and averaged. Then, according to the preset threshold, the edges between variables with weak dependence are filtered to obtain a fusion matrix. Finally, the fusion matrix is modified as the adjacency matrix of the integrated Bayesian network so as to obtain the final Bayesian network structure. Comparative experiments on the standard Bayesian network data sets show that the accuracy and reliability of the proposed algorithm are significantly better than other algorithms. [ABSTRACT FROM AUTHOR]

The application of neural network models to solve combinatorial optimization has recently drawn much attention and shown promising results in dealing with similar problems, like Travelling Salesman Problem. The neural network allows to learn solutions based on given problem instances, using reinforcement learning or supervised learning. In this paper, we present a novel end-to-end method to solve routing problems. In specific, we propose a gated cosine-based attention model (GCAM) to train policies, which accelerates the training process and the convergence of policy. Extensive experiments on different scale of routing problems show that the proposed method can achieve faster convergence of the training process than the state-of-the-art deep learning models while achieving solutions of the same quality. • We proposed a cosine-based neural network model based on the transformer architecture to improve node embeddings. • We use cosine decomposability to accelerate training speed of the proposed model. • We use a gating connection to stabilize training and get faster convergence, which leads to better overall performance. • We apply the proposed gated cosine-based attention model to solve several Routing Problems. [ABSTRACT FROM AUTHOR]

Getting stuck is an inevitable part of learning programming. Long-term stuck decreases the learner's motivation and learning efficiency. The current approach to supporting learning in lectures involves teachers finding students who are getting stuck, reviewing their source code, and solving the problems. However, it is difficult for teachers to grasp every learner's stuck situation and to distinguish stuck or deep thinking only by their source code. Teachers should advise learners only when there is no progress and they are psychologically stuck. This paper proposes a method for detecting when learners get stuck during programming by using multi-modal data, considering both their source code and psychological state measured by a heart rate sensor. The evaluation results of the proposed method show that it can detect more stuck situations than the method that uses only a single indicator. Furthermore, we implemented a system that aggregates the stuck situation detected by the proposed method and presents them to a teacher. In evaluations during the actual programming lecture, participants rated the notification timing of application as suitable and commented that the application was useful. The questionnaire survey showed that the application can detect situations where learners cannot find solutions to exercise problems or express them in programming. [ABSTRACT FROM AUTHOR]

Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our results show that convolutional neural networks (CNNs) outperform feed-forward neural networks in both loss and accuracy. The CNN models classified up to 97.53% of the image responses into the appropriate scoring category, which is comparable to, if not more accurate, than typical human raters. These findings were further strengthened by the observation that the most accurate CNN models correctly classified some image responses that had been incorrectly scored by the human raters. As an additional innovation, we outline a method to select human-rated responses for the training sample based on an application of the expected response function derived from item response theory. This paper argues that CNN-based automated scoring of image responses is a highly accurate procedure that could potentially replace the workload and cost of second human raters for international large-scale assessments (ILSAs), while improving the validity and comparability of scoring complex constructed-response items. [ABSTRACT FROM AUTHOR]

Multi-layer perceptron (MLP) in artificial neural networks (ANN) is one among the trained neural models which can hold several layers as a hidden layer for intensive training to obtain optimal results. On the other hand, the classification problem has a high level of attraction towards researchers to increase the accuracy in classification. In ANN, feedforward neural network (FNN) is one model that possesses the art of solving classification and regression problems. When input data is given to FNN, it will apply the sum of product rule and the activation function to map the input with its appropriate output. In the sum of product rule, a term called weights is to be chosen appropriately to map between the input and output. In standard FNN, the weights are chosen in a random way which may lead to slower convergence towards the optimal choice of weight values. In this paper, an effective optimization model is proposed to optimize the weights of MLP of FNN for effective classification problems. Four different datasets were chosen, and the results are interpreted with statistical performance measures. [ABSTRACT FROM AUTHOR]