Search

Convolutional neural networks (CNNs) play a prominent role in solving problems in various domains such as pattern recognition, image tasks, and natural language processing. In recent years, neural architecture search (NAS), which is the automatic design of neural network architectures as an optimization algorithm, has become a popular method to design CNN architectures against some requirements associated with the network function. However, many NAS algorithms are characterised by a complex search space which can negatively affect the efficiency of the search process. In other words, the representation of the neural network architecture and thus the encoding of the resulting search space plays a fundamental role in the designed CNN performance. In this paper, to make the search process more effective, we propose a novel compact representation of the search space by identifying network blocks as elementary units. The study in this paper focuses on a popular CNN called DenseNet. To perform the NAS, we use an ad-hoc implementation of the particle swarm optimization indicated as PSO-CNN. In addition, to reduce size of the final model, we propose a segmentation method to cut the blocks. We also transfer the final model to different datasets, thus demonstrating that our proposed algorithm has good transferable performance. The proposed PSO-CNN is compared with 11 state-of-the-art algorithms on CIFAR10 and CIFAR100. Numerical results show the competitiveness of our proposed algorithm in the aspect of accuracy and the number of parameters. [ABSTRACT FROM AUTHOR]

The problem of a stabilisation system synthesis for the motion of a control object along a given spatial trajectory is considered. The complexity of the problem is that the preset trajectory is defined in the state subspace and not in time. This paper describes a stabilisation system synthesis for motion along a trajectory specified in time and along a trajectory specified in the form of a manifold in a state space. In order to construct a stabilisation system, it is necessary to determine a distance between an object and the given trajectory at each moment in time. For trajectories that are not given in time, the determination of this distance can be ambiguous. An object may be exactly on a trajectory but at a different time. This paper proposes some approaches to solve the problem. One of the approaches is to transform a given trajectory in a state subspace into a trajectory given in time. A description of a universal method to perform this transformation is presented. In order to solve the synthesis problem automatically, without having to analyse the mathematical model of the control object, it is suggested that machine learning control by symbolic regression is used. In computational experiments, examples of stabilisation system syntheses for quadcopter motion along a given spatial trajectory are presented. [ABSTRACT FROM AUTHOR]

The rapidly advancing fields of machine learning and mathematical modeling, greatly enhanced by the recent growth in artificial intelligence, are the focus of this special issue. This issue compiles extensively revised and improved versions of the top papers from the workshop on Mathematical Modeling and Problem Solving at PDPTA'23, the 29th International Conference on Parallel and Distributed Processing Techniques and Applications. Covering fundamental research in matrix operations and heuristic searches to real-world applications in computer vision and drug discovery, the issue underscores the crucial role of supercomputing and parallel and distributed computing infrastructure in research. Featuring nine key studies, this issue pushes forward computational technologies in mathematical modeling, refines techniques for analyzing images and time-series data, and introduces new methods in pharmaceutical and materials science, making significant contributions to these areas. [ABSTRACT FROM AUTHOR]

Since 2007, methods that utilize side-channel data to detect hardware Trojan (HT) problems have been widely studied. Machine learning methods are widely used for hardware Trojan detection, but with the development of integrated circuits (ICs), better results are usually obtained using deep learning methods. In this paper, we propose an architecture inspired by Residual-Block and Dense-Block and combine it with SE Attention Mechanism, which we named the Res-Dense-SE-Net network. By combining residual connectivity, dense connectivity, and attention mechanism, the Res-Dense-SE-Net network can enjoy the advantages of these three network architectures at the same time, which can improve the expressiveness and performance of the model. The Res-Dense-SE-Net network can capture the key features in the image better, and it can solve the problems of gradient vanishing and feature transfer efficiently, which can in turn improve the classification accuracy and the generalization ability of the model. Based on the publicly available AES series of hardware Trojans from TrustHub and the publicly available hardware Trojan-side channel data by Faezi et al., we evaluate the effectiveness of the method proposed in this paper. The experimental results show that when a single Trojan exists, the method proposed in this paper has a high accuracy rate; and when multiple types of hardware Trojans exist at the same time and need to be categorized, the categories of hardware Trojans can also be effectively identified, and the categorization accuracy is high compared with the existing deep learning methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel approach to solving theorem proving problems without relying on any deduction method. We transform logical formulas into numbers, vectors and matrices, and feed the corresponding data into various machine learning algorithms to predict their satisfiability. Here, we introduce ProverX, a novel theorem prover that utilizes various binary classification algorithms, ranging from traditional machine learning to deep learning, to tackle the satisfiability (SAT) problem of propositional logic. Empirical experiments were conducted to evaluate the performance of ProverX using datasets we generated. ProverX achieved accuracy rates ranging from 81.8% to 98.7%, demonstrating a remarkable speedup of almost 180 times compared to CTL-RP, a highly efficient prover. These results demonstrate the feasibility of replacing deduction with learning in theorem proving, opening promising avenues for further exploration in more complex logics, e.g., Modal Logic, Coalition Logic, Propositional Linear-Time Temporal Logic, Computation Tree Logic, etc., provided that their resolution methods exist. We also present a detailed analysis of the best-performing machine learning approaches for this task and introduce a new algorithm, IPDA, which has the potential to further enhance performance. [ABSTRACT FROM AUTHOR]

Many methods exist for solving the problem of evaluating efficiency in different processes. They are divided into two basic groups, parametric and non-parametric methods, which can have significant differences in the results. In this study, the authors consider the process of assessing the business climate depending on realized foreign investments. Due to the expected difference in efficiency assessment using different approaches, the goal of this paper is to create an optimization model of an ensemble for efficiency assessment that uses both types of methods with the aim of creating a symmetrical approach that achieves better results than each type of method individually. The proposed solution simultaneously analyzes the impact of different factors on foreign investments in order to determine the most important factors and thus enable each local government to ensure the best possible efficiency in this process. The innovative idea of this study is in the inclusion of classification and feature selection methods of machine learning to fulfill the set goal. Our research, focused on a specific case study in various cities across the Republic of Serbia, evaluated the effectiveness of that process. This study extends previous research and confirms the published results, highlighting the advantages of the newly proposed model. [ABSTRACT FROM AUTHOR]

In numerous scientific disciplines and practical applications, addressing optimization challenges is a common imperative. Nature-inspired optimization algorithms represent a highly valuable and pragmatic approach to tackling these complexities. This paper introduces Dendritic Growth Optimization (DGO), a novel algorithm inspired by natural branching patterns. DGO offers a novel solution for intricate optimization problems and demonstrates its efficiency in exploring diverse solution spaces. The algorithm has been extensively tested with a suite of machine learning algorithms, deep learning algorithms, and metaheuristic algorithms, and the results, both before and after optimization, unequivocally support the proposed algorithm's feasibility, effectiveness, and generalizability. Through empirical validation using established datasets like diabetes and breast cancer, the algorithm consistently enhances model performance across various domains. Beyond its working and experimental analysis, DGO's wide-ranging applications in machine learning, logistics, and engineering for solving real-world problems have been highlighted. The study also considers the challenges and practical implications of implementing DGO in multiple scenarios. As optimization remains crucial in research and industry, DGO emerges as a promising avenue for innovation and problem solving. [ABSTRACT FROM AUTHOR]

As an important part of the bridge structure system, the underwater pile-pier structure usually occurs various defects on its surfaces due to its complex hydrological environment. The existing conventional defect detection approaches exist two aspects of problems: (1) insufficient definition and color distortion of the underwater images, and (2) low efficiency and error-prone. To solve these problems, this paper proposed the target defect detection model by integrating the image-fusion enhancement algorithm and the deep learning algorithm. Firstly, by analyzing the reasons for the degradation of the underwater images, the ACE (automatic color equalization) and CLAHE (contrast limited adaptive histogram equalization) algorithms are selected to enhance the image, respectively. Secondly, the two enhanced images are fused based on the point sharpness weight, and then the fusion results are further sharpened by the USM (unsharp mask) algorithm, thus obtaining the final fused images. Thirdly, 3,200 fused images are taken as the training set, by adopting the YOLOv3 algorithm to train the detection model, and then the training model is validated and tested by the other each 400 fused images, thus building up the target automatic detection model of underwater pile-pier surface defects. Finally, a series of comparison and discussion were conducted to validate the effectiveness of image-fusion and the robustness and effectiveness of the target detection model. The results found that the target detection model has excellent robustness against noise and effectiveness in the surface defect detection. This indicates that the image-fusion approach proposed in this paper can effectively enhance the image features, and the target detection model is feasible, robust, and effective in the automatic detection of surface defects on underwater pile-pier structures. [ABSTRACT FROM AUTHOR]

This paper investigates the output feedback (OPFB) tracking control problem for discrete-time linear (DTL) systems with unknown dynamics. To solve this problem, we use an augmented system approach, which first transforms the tracking control problem into a regulation problem with a discounted performance function. The solution to this problem is derived using a Bellman equation, based on the Q-function. In order to overcome the challenges of unmeasurable system state variables, we employ a multistep Q-learning algorithm that surpasses the advantages of the policy iteration (PI) and value iteration (VI) techniques and state reconstruction methods for output feedback control. As such, the requirement for an initial stabilizing control policy for the PI method is removed and the convergence speed of the learning algorithm is improved. Finally, we demonstrate the effectiveness of the proposed scheme using a simulation example. [ABSTRACT FROM AUTHOR]

Machine learning (ML) is evolving rapidly and has made many theoretical breakthroughs while widely applied in various fields. ML allows systems the ability to access data and use it to enable computers to execute cognitive processes such as learning and improving from previous experiences and solving complicated issues. Many first-order stochastic optimization methods have been used to solve the optimization model of ML. These algorithms adopt Barzilai-Borwein (BB) step size instead of fixed or diminishing step size to improve performance. However, the BB step size format involves fractional calculation, which inevitably leads to a zero denominator, especially when the objective function is non-convex. The BB technique will be violated if the denominator is near 0 or even negative. To improve the computation of the step size, a Positive Defined Stabilized Barzilai-Borwein (PDSBB) approach is introduced in this paper. Integrating PDSBB with the stochastic variance reduced gradient (SVRG) approach, a new method SVRG-PDSBB is proposed. Numerical experiments have shown that the new algorithm has stabilized the performance of the new step size, which successfully avoiding zero denominators and effectively solving the common problems in machine learning. The convergence of SVRG-PDSBB is theoretically and numerically proven, and the effectiveness of the new algorithm is shown by comparison with other algorithms. [ABSTRACT FROM AUTHOR]

The widespread use of the internet and the exponential growth in small hardware diversity enable the development of Internet of things (IoT)-based localization systems. We review machine-learning-based approaches for IoT localization systems in this paper. Because of their high prediction accuracy, machine learning methods are now being used to solve localization problems. The paper's main goal is to provide a review of how learning algorithms are used to solve IoT localization problems, as well as to address current challenges. We examine the existing literature for published papers released between 2020 and 2022. These studies are classified according to several criteria, including their learning algorithm, chosen environment, specific covered IoT protocol, and measurement technique. We also discuss the potential applications of learning algorithms in IoT localization, as well as future trends. [ABSTRACT FROM AUTHOR]

Online portfolio optimization with transaction costs is a big challenge in large-scale intelligent computing community, since its undersample from rapidly-changing market and complexity from varying transaction costs. In this paper, we focus on this problem and solve it by machine learning system. Specifically, we reformulate the optimization problem with the minimization over simplex containing three items, which are negative expected return, the elastic net regularization of transaction costs controlled term and portfolio variable, respectively. We propose to apply linearized augmented Lagrangian method (LALM) and the alternating direction method of multipliers (ADMM) to solve the optimization model in a higher efficiency, meanwhile theoretically guarantee their convergence and deduce closed-form solutions of their subproblems in each iteration. Furthermore, we conduct extensive experiments on five benchmark datasets from real market to demonstrate that the proposed algorithms outperform compared state-of-the-art strategies in most cases in six dimensions. [ABSTRACT FROM AUTHOR]

In the context of collaborative robotics, handing over hand-held objects to a robot is a safety-critical task. Therefore, a robust distinction between human hands and presented objects in image data is essential to avoid contact with robotic grippers. To be able to develop machine learning methods for solving this problem, we created the OHO (Object Hand-Over) dataset of tools and other everyday objects being held by human hands. Our dataset consists of color, depth, and thermal images with the addition of pose and shape information about the objects in a real-world scenario. Although the focus of this paper is on instance segmentation, our dataset also enables training for different tasks such as 3D pose estimation or shape estimation of objects. For the instance segmentation task, we present a pipeline for automated label generation in point clouds, as well as image data. Through baseline experiments, we show that these labels are suitable for training an instance segmentation to distinguish hands from objects on a per-pixel basis. Moreover, we present qualitative results for applying our trained model in a real-world application. [ABSTRACT FROM AUTHOR]

Originally, tangles were invented as an abstract tool in mathematical graph theory to prove the famous graph minor theorem. In this paper, we showcase the practical potential of tangles in machine learning applications. Given a collection of cuts of any dataset, tangles aggregate these cuts to point in the direction of a dense structure. As a result, a cluster is softly characterized by a set of consistent pointers. This highly flexible approach can solve clustering problems in various setups, ranging from questionnaires over community detection in graphs to clustering points in metric spaces. The output of our proposed framework is hierarchical and induces the notion of a soft dendrogram, which can help explore the cluster structure of a dataset. The computational complexity of aggregating the cuts is linear in the number of data points. Thus the bottleneck of the tangle approach is to generate the cuts, for which simple and fast algorithms form a sufficient basis. In our paper we construct the algorithmic framework for clustering with tangles, prove theoretical guarantees in various settings, and provide extensive simulations and use cases. Python code is available on github. [ABSTRACT FROM AUTHOR]

Engineers strive to realize the goal of control in the best possible way based on a given quality criterion when developing control system. The well-known optimal control problem requires finding a solution as a function of time. Such a function cannot be directly used to control a real object because its application corresponds to an open-loop control system and engineers usually complete it with a feedback stabilization system. Hence, there is an obvious need to reformulate the optimal control problem so that its solution can be directly applied to real objects. The paper presents a new extended statement of the optimal control problem. An additional requirement for the control function is introduced to give the system describing the control object properties that will ensure the stability of solutions. The desired control function must provide for the optimal trajectory given the properties of the attractor in the neighbourhood. The solution to the extended optimal control problem can be directly used to control a real object. The paper presents a computational machine learning approach to solving the extended problem of optimal control based on the application of a synthesized optimal control technique. Examples of the practical solution to the stated problem are given to illustrate the efficiency of the approach, where the solution to the conventional optimal control problem is compared with the proposed extended one in the presence of perturbations in models and initial conditions. [ABSTRACT FROM AUTHOR]

With the abundance of data, machine learning applications engaged increased attention in the last decade. An attractive approach to robustify the statistical analysis is to preprocess the data through clustering. This paper develops a low-complexity Riemannian optimization framework for solving optimization problems on the set of positive semidefinite stochastic matrices. The low-complexity feature of the proposed algorithms stems from the factorization of the optimization variable $\mathbf {X}=\mathbf {Y}\mathbf {Y}^{\mathrm{T}}$ X = Y Y T and deriving conditions on the number of columns of $\mathbf {Y}$ Y under which the factorization yields a satisfactory solution. The paper further investigates the embedded and quotient geometries of the resulting Riemannian manifolds. In particular, the paper explicitly derives the tangent space, Riemannian gradients and Hessians, and a retraction operator allowing the design of efficient first and second-order optimization methods for the graph-based clustering applications of interest. The numerical results reveal that the resulting algorithms present a clear complexity advantage as compared with state-of-the-art euclidean and Riemannian approaches for graph clustering applications. [ABSTRACT FROM AUTHOR]

Routing is one of the most important and time-consuming stages of physical design. As the process node of semiconductors keeps scaling down, the routing process faces increasing challenges, and the traditional solutions are not sufficiently efficient. In recent years, machine learning has aroused much interest in this context, and an increasing number of algorithms have introduced advanced machine learning techniques to help solve the routing problem. In this paper, we survey the recent development of machine learning-based routing algorithms. • This paper provides an overview of recent published machine learning-based routing algorithms. • Some trends on the development of machine learning-based routing algorithms are revealed. • The possible advantages and disadvantages on machine learning-based routing methods are discussed. [ABSTRACT FROM AUTHOR]

This paper considers the offline assignment of fixed priorities in partitioned preemptive real-time systems where tasks have precedence constraints. This problem is crucial in this type of systems, as having a good fixed priority assignment allows for an efficient use of the processing resources while meeting all the deadlines. In the literature, we can find several proposals to solve this problem, which offer varying trade-offs between the quality of their results and their computational complexities. In this paper , we propose a new approach, leveraging existing algorithms that are widely exploited in the field of Machine Learning: Gradient Descent, the Adam Optimizer, and Gradient Noise. We show how to adapt these algorithms to the problem of fixed priority assignment in conjunction with existing worst-case response time analyses. We demonstrate the performance of our proposal on synthetic task-sets with different sizes. This evaluation shows that our proposal is able to find more schedulable solutions than previous heuristics, approximating optimal but intractable algorithms such as MILP or brute-force, while requiring reasonable execution times. [ABSTRACT FROM AUTHOR]

Background: In many applications of bioinformatics, data stem from distinct heterogeneous sources. One of the well-known examples is the identification of drug–target interactions (DTIs), which is of significant importance in drug discovery. In this paper, we propose a novel framework, manifold optimization based kernel preserving embedding (MOKPE), to efficiently solve the problem of modeling heterogeneous data. Our model projects heterogeneous drug and target data into a unified embedding space by preserving drug–target interactions and drug–drug, target–target similarities simultaneously. Results: We performed ten replications of ten-fold cross validation on four different drug–target interaction network data sets for predicting DTIs for previously unseen drugs. The classification evaluation metrics showed better or comparable performance compared to previous similarity-based state-of-the-art methods. We also evaluated MOKPE on predicting unknown DTIs of a given network. Our implementation of the proposed algorithm in R together with the scripts that replicate the reported experiments is publicly available at https://github.com/ocbinatli/mokpe. [ABSTRACT FROM AUTHOR]

Getting stuck is an inevitable part of learning programming. Long-term stuck decreases the learner's motivation and learning efficiency. The current approach to supporting learning in lectures involves teachers finding students who are getting stuck, reviewing their source code, and solving the problems. However, it is difficult for teachers to grasp every learner's stuck situation and to distinguish stuck or deep thinking only by their source code. Teachers should advise learners only when there is no progress and they are psychologically stuck. This paper proposes a method for detecting when learners get stuck during programming by using multi-modal data, considering both their source code and psychological state measured by a heart rate sensor. The evaluation results of the proposed method show that it can detect more stuck situations than the method that uses only a single indicator. Furthermore, we implemented a system that aggregates the stuck situation detected by the proposed method and presents them to a teacher. In evaluations during the actual programming lecture, participants rated the notification timing of application as suitable and commented that the application was useful. The questionnaire survey showed that the application can detect situations where learners cannot find solutions to exercise problems or express them in programming. [ABSTRACT FROM AUTHOR]

Automated scoring of free drawings or images as responses has yet to be used in large-scale assessments of student achievement. In this study, we propose artificial neural networks to classify these types of graphical responses from a TIMSS 2019 item. We are comparing classification accuracy of convolutional and feed-forward approaches. Our results show that convolutional neural networks (CNNs) outperform feed-forward neural networks in both loss and accuracy. The CNN models classified up to 97.53% of the image responses into the appropriate scoring category, which is comparable to, if not more accurate, than typical human raters. These findings were further strengthened by the observation that the most accurate CNN models correctly classified some image responses that had been incorrectly scored by the human raters. As an additional innovation, we outline a method to select human-rated responses for the training sample based on an application of the expected response function derived from item response theory. This paper argues that CNN-based automated scoring of image responses is a highly accurate procedure that could potentially replace the workload and cost of second human raters for international large-scale assessments (ILSAs), while improving the validity and comparability of scoring complex constructed-response items. [ABSTRACT FROM AUTHOR]

Network virtualization is a technology proven to be a key enabling a family of strategies in different targets, such as energy efficiency, economic revenue, network usage, adaptability or failure protection. Network virtualization allows us to adapt the needs of a network to new circumstances, resulting in greater flexibility. The allocation decisions of the demands onto the physical network resources impact the costs and the benefits. Therefore it is one of the major current problems, called virtual network embedding (VNE). Many algorithms have been proposed recently in the literature to solve the VNE problem for different targets. Due to the current successful rise of artificial intelligence, it has been widely used recently to solve technological problems. In this context, this paper investigates the requirements and analyses the use of the Q-learning algorithm for energy-efficient VNE. The results achieved validate the strategy and show clear improvements in terms of cost/revenue and energy savings, compared to traditional algorithms. [ABSTRACT FROM AUTHOR]

Facial expression recognition based on residual networks is important for technologies related to space human-robot interaction and collaboration but suffers from low accuracy and slow computation in complex network structures. To solve these problems, this paper proposes a multiscale feature fusion attention lightweight wide residual network. The network first uses an improved random erasing method to preprocess facial expression images, which improves the generalizability of the model. The use of a modified depthwise separable convolution in the feature extraction network reduces the computational effort associated with the network parameters and enhances the characterization of the extracted features through a channel shuffle operation. Then, an improved bottleneck block is used to reduce the dimensionality of the upper layer network feature map to further reduce the number of network parameters while enhancing the network feature extraction capability. Finally, an optimized multiscale feature lightweight attention mechanism module is embedded to further improve the feature extractability of the network for human facial expressions. The experimental results show that the accuracy of the model is 73.21%, 98.72%, and 95.21% on FER2013, CK+ and JAFFE, respectively, with a covariance of 10.14 M. Compared with other networks, the model proposed in this paper has faster computing speed and better accuracy at the same time. [ABSTRACT FROM AUTHOR]

With the development of machine learning, data-driven mechanical fault diagnosis methods have been widely used in the field of PHM. Due to the limitation of the amount of fault data, it is a difficult problem for fault diagnosis to solve the problem of unbalanced data sets. Under unbalanced data sets, faults with little historical data are always difficult to diagnose and lead to economic losses. In order to improve the prediction accuracy under unbalanced data sets, this paper proposes MeanRadius-SMOTE based on the traditional SMOTE oversampling algorithm, which effectively avoids the generation of useless samples and noise samples. This paper validates the effectiveness of the algorithm on three linear unbalanced data sets and four step unbalanced data sets. Experimental results show that MeanRadius-SMOTE outperforms SMOTE and LR-SMOTE in various evaluation indicators, as well as has better robustness against different imbalance rates. In addition, MeanRadius-SMOTE can take into account the prediction accuracy of the overall and minority class, which is of great significance for engineering applications. [ABSTRACT FROM AUTHOR]

Based on ML algorithm, this paper puts forward a method that can search instructional resources through keyword indexing technology, and then cluster and recombine the related results and present them centrally. In this paper, the semantic processing of user query based on the subject index of educational resources is adopted to make up for the deficiency of query semantics, solve the problem of mismatch between query words and document words, and improve the recall and precision of resource retrieval. It is proposed to select the category feature items manually and establish the category feature model. In the environment of small sample set, the weight of category feature items is trained by ML method. The research shows that the user rating of this system is ideal, reaching 93.21% at the highest. In addition, the stability of the system can still reach 89.31% under the condition of relatively large usage, and its performance is excellent. This system can effectively solve the problem of scattered distribution of English instructional resources and make the presentation of knowledge more in line with the needs of users, thereby further improving the utilization rate of English instructional resources and users' satisfaction. [ABSTRACT FROM AUTHOR]

The research on artistic psychological intervention to judge emotional fluctuations by extracting emotional features from interactive vocal signals has become a research topic with great potential for development. Based on the interactive vocal music instruction theory of emotion recognition, this paper studies the design of artistic psychological intervention system. This paper uses the vocal music emotion recognition algorithm to first train the interactive recognition network, in which the input is a row vector composed of different vocal music characteristics, and finally recognizes the vocal music of different emotional categories, which solves the problem of low data coupling in the artistic psychological intervention system. Among them, the vocal music emotion recognition experiment based on the interactive recognition network is mainly carried out from six aspects: the number of iterative training, the vocal music instruction rate, the number of emotion recognition signal nodes in the artistic psychological intervention layer, the number of sample sets, different feature combinations, and the number of emotion types. The input data of the system is a training class learning video, and actions and expressions need to be recognized before scoring. In the simulation process, before the completion of the sample indicators is unbalanced, the R language statistical analysis tool is used to balance the existing unbalanced data based on the artificial data synthesis method, and 279 uniformly classified samples are obtained. The 279 ∗ 7 dataset was used for statistical identification of the participants. The experimental results show that under the guidance of four different interactive vocal music, the vocal emotion recognition rate is between 65.85%-91.00%, which promotes the intervention of music therapy on artistic psychological intervention. [ABSTRACT FROM AUTHOR]

State space models (SSMs) are important for multi-variable performance analysis and controller design of aero-engines. In order to solve the problems of the traditional state space modeling methods that rely on component-level models (CLMs) and cannot be carried out in real time, an aero-engine state space modeling method based on adaptive forgetting factor online sequential extreme learning machine (AFOS-ELM) is proposed in this paper. The structure of the extreme learning machine (ELM) is determined according to the form of the state space model, and the inverse-free ELM algorithm is used to automatically select the appropriate number of hidden nodes to improve the efficiency of offline initialization. The focus of the ELM on current operation performance is enhanced by the adaptive renewed forgetting factor, which reduces the impact of aero-engine history and deviated data on the current output and improves the accuracy of the model. Then, according to the analytical equation of the ELM model, the state space model of an aero-engine at each sampling time is obtained by using the partial derivative method. The simulation results based on engine test data show that the real-time performance and accuracy of the state space model established online in this paper can meet the needs of aero-engine control system requirement. [ABSTRACT FROM AUTHOR]

Fast and accurate smoke detection is very important for reducing fire damage. Due to the complexity and changeable nature of smoke scenes, existing smoke detection technology has the problems of a low detection rate and a high false negative rate, and the robustness and generalization ability of the algorithms are not high. Therefore, this paper proposes a smoke detection model based on the improved YOLOv5. First, a large number of real smoke and synthetic smoke images were collected to form a dataset. Different loss functions (GIoU, DIoU, CIoU) were used on three different models of YOLOv5 (YOLOv5s, YOLOv5m, YOLOv5l), and YOLOv5m was used as the baseline model. Then, because of the problem of small numbers of smoke training samples, the mosaic enhancement method was used to randomly crop, scale and arrange nine images to form new images. To solve the problem of inaccurate anchor box prior information in YOLOv5, a dynamic anchor box mechanism is proposed. An anchor box was generated for the training dataset through the k-means++ clustering algorithm. The dynamic anchor box module was added to the model, and the size and position of the anchor box were dynamically updated in the network training process. Aiming at the problem of unbalanced feature maps in different scales of YOLOv5, an attention mechanism is proposed to improve the network detection performance by adding channel attention and spatial attention to the original network structure. Compared with the traditional deep learning algorithm, the detection performance of the improved algorithm in this paper was is 4.4% higher than the mAP of the baseline model, and the detection speed reached 85 FPS, which is obviously better and can meet engineering application requirements. [ABSTRACT FROM AUTHOR]

Aiming to obtain more discriminative features in scene images and overcome the impacts of intra-class differences and inter-class similarities, the paper proposes a scene recognition method that combines attention and context information. First, we introduce the attention mechanism and build a multi-scale attention model. Discriminative information considers salient objects and regions by means of channel attention and spatial attention. Besides, the central loss function joint supervision strategy is introduced to further reduce the misjudgment of intra-class differences. Second, a model based on multi-level context information is proposed to describe the positional relationship between objects, which can effectively alleviate the influence of the similarity of objects between classes. Finally, the two models are merged to give full play to the compatibility of features, so that the final feature representation not only focuses on the effective discriminant information, but also manifests the relative position relationship between significant objects. Extensive experiments have proved that the method in this paper effectively solves the problem of insufficient feature representation in scene recognition tasks, and improves the accuracy of scene recognition. [ABSTRACT FROM AUTHOR]

The learning with errors (LWE) problem is a problem derived from machine learning that is believed to be intractable for quantum computers. This paper proposes a method that can reduce an LWE problem to a set of maximum independent set (MIS) problems, which are graph problems that are suitable for a quantum annealing machine to solve. The reduction algorithm can reduce an n-dimensional LWE problem to several small MIS problems with at most 2 O (n) nodes when the lattice-reduction algorithm used in the LWE-reduction method successfully finds short vectors. The algorithm is useful for solving LWE problems in a quantum-classical hybrid manner by using an existing quantum algorithm to solve the MIS problems. For example, the smallest LWE challenge problem is reduced to MIS problems with about 40,000 vertices. This result means that the smallest LWE challenge problem will be within the scope of a real quantum computer in the near future. [ABSTRACT FROM AUTHOR]

In this paper, a machine-learning model is utilized to estimate the temperature-dependent moduli of neat, thermally reduced graphene and covalently functionalized graphene/epoxy nanocomposites. In addition, the governed mathematical expressions have been used to solve the buckling problem of beams fabricated from such nanocomposites in the presence of a thermal gradient. In order to do so, an energy-based method including the shear deformable beam hypothesis is used. The beam structure is rested on the Winkler–Pasternak substrate. The reported verifications demonstrate the impressive precision of the presented ML model, as well as the buckling response of the under-study structures. Finally, in the framework of some numerical case studies, the impact of several parameters on the buckling of nanocomposite beams is depicted. The results of this study delineate that temperature has a vital role in the determination of the critical buckling load that the nanocomposite structures can endure. [ABSTRACT FROM AUTHOR]

The power quality and efficiency of the hydro-power station depend on the stable operation of the hydro-generator unit, which needs to continue to operate and it is prone to axis failure. Therefore, to adopt effective axis adjustment technology to eliminate faults. This paper proposes a new method for axis adjustment of hydro-generator unit based on an improved grey prediction model and swarms intelligence optimization neural network. First of all, it proposes a sequence acceleration translation and mean value transformation method, which is used to pre-process the axis net total swing sequence that exhibits oscillating fluctuations. It uses e1 and e2 factor transformation to establish an improved axis net total swing gray prediction model. Then, the advanced flamingo search algorithm is used to search the maximum value of the sine function of the net total pendulum of the axis, and the axis adjustment orientation is obtained. This method solves the problem that GM(1, 1) can only be predicted by monotone sequence in the past and the problem that the search algorithm is easy to fall into local optimum, effectively improves the calculation efficiency of axis and shorts the search time. Simulation examples show that the proposed method can significantly improve accuracy of axis adjustment. This method greatly improves the efficiency of azimuth search for axis adjustment. [ABSTRACT FROM AUTHOR]

At present, there are some problems in underwater low light image, such as low contrast, blurred details, color distortion. In the process of low illumination image enhancement, there are often problems such as artifacts, loss of edge details and noise amplification in the enhanced image. In this paper, we propose an underwater low-light enhancement algorithm based on U-shaped generative adversarial network, combined with bright channel prior and attention mechanism, to address the problems. For the problems of uneven edges and loss of details that occurred in traditional enhanced images, we propose a two-channel fusion technique for the input channel. Aiming at the problems of brightness, texture and color distortion in enhanced images, we propose a feature extraction technique based on the attention mechanism. For the problems of noise in enhanced output images, we propose a multi-loss function to constrain the network. The method has a wide range of applications in underwater scenes with large depth. This method can be used for target detection or biological species identification in underwater low light environment. Through the enhancement experiment of underwater low light image, the proposed method effectively solves the problems of low contrast, blurred details, color distortion, etc. of underwater low light image. Finally, we performed extensive comparison experiments and completed ablation experiments on the proposed method. The experimental results show that the proposed method is optimal in human visual experience and underwater image quality evaluation index. [ABSTRACT FROM AUTHOR]

The summary of case–public opinion refers to the generation of case-related sentences from public opinion information related to judicial cases. Case–public opinion news refers to the judicial cases (intentional homicide, rape, etc.) that cause large public opinion. The public opinion news in these cases usually contains case element information such as the suspect, victim, time, place, process, and sentencing of the case. In the multi-document summary of case–public opinion, due to the problem of information cross and information redundancy between different documents under the same case, in order to generate a concise and smooth summary, this paper proposes an abstractive summary model of case–public opinion based on the attention of a case element diagram. Firstly, multiple public opinion documents in the same case are split into paragraphs, and then the paragraphs and case elements are coded based on the transformer method to construct a heterogeneous graph containing paragraph nodes and case element nodes. Finally, in the decoding process, the two-layer attention mechanism is applied to the case element node and paragraph node, so that the model can effectively solve the redundancy problem in summary generation. [ABSTRACT FROM AUTHOR]

The opportunity for large amounts of open-for-public and available data is one of the main drivers of the development of an information society at the beginning of the 21st century. In this sense, acquiring knowledge from these data using different methods of machine learning is a prerequisite for solving complex problems in many spheres of human activity, starting from medicine to education and the economy, including traffic as today's important economic branch. Having this in mind, this paper deals with the prediction of the risk of traffic incidents using both historical and real-time data for different atmospheric factors. The main goal is to construct an ensemble model based on the use of several machine learning algorithms which has better characteristics of prediction than any of those installed when individually applied. In global, a case-proposed model could be a multi-agent system, but in a considered case study, a two-agent system is used so that one agent solves the prediction task by learning from the historical data, and the other agent uses the real time data. The authors evaluated the obtained model based on a case study and data for the city of Niš from the Republic of Serbia and also described its implementation as a practical web citizen application. [ABSTRACT FROM AUTHOR]

In this paper, we study the problem of privacy-preserving data synthesis (PPDS) for tabular data in a distributed multi-party environment. In a decentralized setting, for PPDS, federated generative models with differential privacy are used by the existing methods. Unfortunately, the existing models apply only to images or text data and not to tabular data. Unlike images, tabular data usually consist of mixed data types (discrete and continuous attributes) and real-world datasets with highly imbalanced data distributions. Existing methods hardly model such scenarios due to the multimodal distributions in the decentralized continuous columns and highly imbalanced categorical attributes of the clients. To solve these problems, we propose a federated generative model for decentralized tabular data synthesis (HT-Fed-GAN). There are three important parts of HT-Fed-GAN: the federated variational Bayesian Gaussian mixture model (Fed-VB-GMM), which is designed to solve the problem of multimodal distributions; federated conditional one-hot encoding with conditional sampling for global categorical attribute representation and rebalancing; and a privacy consumption-based federated conditional GAN for privacy-preserving decentralized data modeling. The experimental results on five real-world datasets show that HT-Fed-GAN obtains the best trade-off between the data utility and privacy level. For the data utility, the tables generated by HT-Fed-GAN are the most statistically similar to the original tables and the evaluation scores show that HT-Fed-GAN outperforms the state-of-the-art model in terms of machine learning tasks. [ABSTRACT FROM AUTHOR]

Stochastic gradient descent is the method of choice for solving large-scale optimization problems in machine learning. However, the question of how to effectively select the step-sizes in stochastic gradient descent methods is challenging, and can greatly influence the performance of stochastic gradient descent algorithms. In this paper, we propose a class of faster adaptive gradient descent methods, named AdaSGD, for solving both the convex and non-convex optimization problems. The novelty of this method is that it uses a new adaptive step size that depends on the expectation of the past stochastic gradient and its second moment, which makes it efficient and scalable for big data and high parameter dimensions. We show theoretically that the proposed AdaSGD algorithm has a convergence rate of O (1 / T) in both convex and non-convex settings, where T is the maximum number of iterations. In addition, we extend the proposed AdaSGD to the case of momentum and obtain the same convergence rate for AdaSGD with momentum. To illustrate our theoretical results, several numerical experiments for solving problems arising in machine learning are made to verify the promise of the proposed method. [ABSTRACT FROM AUTHOR]

Semantic segmentation of remote sensing images plays a crucial role in urban planning and development. How to perform automatic, fast, and effective semantic segmentation of considerable size and high-resolution remote sensing images has become the key to research. However, the existing segmentation methods based on deep learning are complex and often difficult to apply practically due to the high computational cost of the excessive parameters. In this paper, we propose an end-to-end lightweight progressive attention semantic segmentation network (LPASS-Net), which aims to solve the problem of reducing computational costs without losing accuracy. Firstly, its backbone features are based on a lightweight network, MobileNetv3, and a feature fusion network composed of a reverse progressive attentional feature fusion network work. Additionally, a lightweight non-local convolutional attention network (LNCA-Net) is proposed to effectively integrate global information of attention mechanisms in the spatial dimension. Secondly, an edge padding cut prediction (EPCP) method is proposed to solve the problem of splicing traces in the prediction results. Finally, evaluated on the public datasets BDCI 2017 and ISPRS Potsdam, the mIoU reaches 83.17% and 88.86%, respectively, with an inference time of 0.0271 s. [ABSTRACT FROM AUTHOR]

Personalised recommendation is a difficult problem that has received a lot of attention to academia and industry. Because of the sparse user–item interaction, cold-start recommendation has been a particularly difficult problem. Some efforts have been made to solve the cold-start problem by using model-agnostic meta-learning on the level of the model and heterogeneous information networks on the level of data. Moreover, using the memory-augmented meta-optimisation method effectively prevents the meta-learning model from entering the local optimum. As a result, this paper proposed memory-augmented meta-learning on meta-path, a new meta-learning method that addresses the cold-start recommendation on the meta-path furthered. The meta-path builds at the data level to enrich the relevant semantic information of the data. To achieve fast adaptation, semantic-specific memory is utilised to conduct the model with semantic parameter initialisation, and the method is optimised by a meta-optimisation method. We put this method to the test using two widely used recommended data set and three cold-start scenarios. The experimental results demonstrate the efficiency of our proposed method. [ABSTRACT FROM AUTHOR]

As more and more smart devices are deployed in homes, the communication between these smart home devices and elastic computing services may face some risks of privacy disclosure. Different device events (such as the camera on, video on, etc.) will generate different data traffic during communication. However, the current smart home system lacks monitoring of these device events, which may cause the disclosure of private data collected by these devices. In this paper, we present our device event monitor system, HomeMonitor. HomeMonitor runs in the OpenWRT system and supports complete event monitoring for smart home devices. HomeMoitor solves the problem that machine learning models for detecting device events do not scale flexibly. It uses the network packet size and the direction of the device event for unique identification during training. When detecting, it only needs to get the packet size and timestamp and then query the policy table for signature matching to control the device events. We evaluated the effectiveness of HomeMonitor, and the experiments show that the match rate of our method is 98.8%, the false positive rate is 1.8%, and the detection time is only 16.67% for PINBALL. The results mean that our method achieves the balance of applicable protocol scope, detection performance, and detection accuracy. [ABSTRACT FROM AUTHOR]

One of the useful knowledge representation tools, which can describe the joint probability distribution between some random variables with a graphical model and can be trained by a dataset, is the Bayesian network (BN). A BN is composed of a network structure and a conditional probability distribution table for each node. Discovering an optimal BN structure is an NP‐hard optimization problem that various meta‐heuristic algorithms are applied to solve this problem by researchers. The genetic algorithms, ant colony optimization, evolutionary programming, artificial bee colony, and bacterial foraging optimization are some of the meta‐heuristic methods to solve this problem using a dataset. Most of these methods are applying a scoring metric to generate the best network structure from a set of candidates. A Fixed Structure Learning Automata‐Based (FSLA‐B) algorithm is presented in this paper to solve the structure learning problem of BNs. There is a fixed structure learning automaton for each pair of vertices in the BN's graph structure in the proposed algorithm. The action of this automaton determines the presence and direction of an edge between the vertices. The proposed algorithm performs a guided search procedure using the FSLA and escapes from local optimums. Several datasets are utilised in this paper to evaluate the performance of the proposed algorithm. By performing various experiments, multiple meta‐heuristic algorithms are compared with the introduced new one. The obtained results represented that the proposed algorithm could produce competitive results and find the near‐optimal solution for the BN structure learning problem. [ABSTRACT FROM AUTHOR]

Cultural and creative design is a new design mode based on the Internet platform, which gathers the design wisdom of the public and provides online solutions for design tasks. Evaluation indicators are used to score cultural and creative design schemes that require manual construction of an evaluation matrix, which makes the evaluation efficiency very low and increases the product design cycle. Therefore, based on the deep belief network of cultural creation, the method of constructing the evaluation model of the scheme is designed in this paper. According to the characteristics of the deep belief network, the k-medoids clustering method is adopted, and the clustering key value k is set. Meanwhile, the clustering result is calculated, and k central point elements are obtained. Moreover, project the central point elements back to the cultural and creative design plan to obtain the characteristic plan, and use manual evaluation to calculate the comprehensive score. Besides, the feature scheme data and the data simulation formula are used to construct the training set of the deep belief network, which are used to train the evaluation model of the deep belief network to solve the problem of the length of time that requires a large amount of manual evaluation to construct the output of the training set. In addition, according to the number of evaluation indicators, the number of nodes in the input layer of the deep belief network is determined to define the initialization method of network weights and biases, formulating experimental procedures to determine the optimal network structure. In the simulation experiment, through the test analysis of the comparison model, the function synchronization rate in this paper is 4.2% better than that of the comparison method. After multiple iterations, the test effect is 7 times higher than that of the comparison method, realizing the optimal evaluation of the design schemes that have not been manually evaluated in the same design scheme set. [ABSTRACT FROM AUTHOR]

The traditional English teaching system has certain problems in the acquisition of teaching resources and the innovation of teaching models. In order to improve the effect of subsequent English online teaching, this paper improves the machine learning algorithm to make it a core algorithm that can be used by artificial intelligence systems. Moreover, this paper combines the WBIETS system to expand the system function, analyzes the needs of the English network teaching system, and constructs the system function modules and logical structure. The data layer, logic layer, and presentation layer in the system constructed in this paper are independent of each other and can be effectively expanded when subsequent requirements change. In addition, this paper solves the problem of acquiring English teaching resources through the WBIETS system. To evaluate the performance of the English network teaching system, this paper performs comprehensive mathematical and experimental analysis. The experimental results show that the system constructed in this paper basically meets the actual teaching requirements. [ABSTRACT FROM AUTHOR]

This paper is devoted to the problem of class imbalance in machine learning, focusing on the intrusion detection of rare classes in computer networks. The problem of class imbalance occurs when one class heavily outnumbers examples from the other classes. In this paper, we are particularly interested in classifiers, as pattern recognition and anomaly detection could be solved as a classification problem. As still a major part of data network traffic of any organization network is benign, and malignant traffic is rare, researchers therefore have to deal with a class imbalance problem. Substantial research has been undertaken in order to identify these methods or data features that allow to accurately identify these attacks. But the usual tactic to deal with the imbalance class problem is to label all malignant traffic as one class and then solve the binary classification problem. In this paper, however, we choose not to group or to drop rare classes but instead investigate what could be done in order to achieve good multi-class classification efficiency. Rare class records were up-sampled using SMOTE method (Chawla et al., 2002) to a preset ratio targets. Experiments with the 3 network traffic datasets, namely CIC-IDS2017, CSE-CIC-IDS2018 (Sharafaldin et al., 2018) and LITNET-2020 (Damasevicius et al., 2020) were performed aiming to achieve reliable recognition of rare malignant classes available in these datasets. Popular machine learning algorithms were chosen for comparison of their readiness to support rare class detection. Related algorithm hyper parameters were tuned within a wide range of values, different data feature selection methods were used and tests were executed with and without over-sampling to test the multiple class problem classification performance of rare classes. Machine learning algorithms ranking based on Precision, Balanced Accuracy Score, G ¯ , and prediction error Bias and Variance decomposition, show that decision tree ensembles (Adaboost, Random Forest Trees and Gradient Boosting Classifier) performed best on the network intrusion datasets used in this research. [ABSTRACT FROM AUTHOR]

Background: Automatic and accurate recognition of various biomedical named entities from literature is an important task of biomedical text mining, which is the foundation of extracting biomedical knowledge from unstructured texts into structured formats. Using the sequence labeling framework and deep neural networks to implement biomedical named entity recognition (BioNER) is a common method at present. However, the above method often underutilizes syntactic features such as dependencies and topology of sentences. Therefore, it is an urgent problem to be solved to integrate semantic and syntactic features into the BioNER model. Results: In this paper, we propose a novel biomedical named entity recognition model, named BioByGANS (BioBERT/SpaCy-Graph Attention Network-Softmax), which uses a graph to model the dependencies and topology of a sentence and formulate the BioNER task as a node classification problem. This formulation can introduce more topological features of language and no longer be only concerned about the distance between words in the sequence. First, we use periods to segment sentences and spaces and symbols to segment words. Second, contextual features are encoded by BioBERT, and syntactic features such as part of speeches, dependencies and topology are preprocessed by SpaCy respectively. A graph attention network is then used to generate a fusing representation considering both the contextual features and syntactic features. Last, a softmax function is used to calculate the probabilities and get the results. We conduct experiments on 8 benchmark datasets, and our proposed model outperforms existing BioNER state-of-the-art methods on the BC2GM, JNLPBA, BC4CHEMD, BC5CDR-chem, BC5CDR-disease, NCBI-disease, Species-800, and LINNAEUS datasets, and achieves F1-scores of 85.15%, 78.16%, 92.97%, 94.74%, 87.74%, 91.57%, 75.01%, 90.99%, respectively. Conclusion: The experimental results on 8 biomedical benchmark datasets demonstrate the effectiveness of our model, and indicate that formulating the BioNER task into a node classification problem and combining syntactic features into the graph attention networks can significantly improve model performance. [ABSTRACT FROM AUTHOR]

In order to categorize feature classes by capturing subtle differences, hyperspectral images (HSIs) have been extensively used due to the rich spectral-spatial information. The 3D convolution-based neural networks (3DCNNs) have been widely used in HSI classification because of their powerful feature extraction capability. However, the 3DCNN-based HSI classification approach could only extract local features, and the feature maps it produces include a lot of spatial information redundancy, which lowers the classification accuracy. To solve the above problems, we proposed a spatial attention network (SATNet) by combining 3D OctConv and ViT. Firstly, 3D OctConv divided the feature maps into high-frequency maps and low-frequency maps to reduce spatial information redundancy. Secondly, the ViT model was used to obtain global features and effectively combine local-global features for classification. To verify the effectiveness of the method in the paper, a comparison with various mainstream methods on three publicly available datasets was performed, and the results showed the superiority of the proposed method in terms of classification evaluation performance. [ABSTRACT FROM AUTHOR]

In the development of anti-breast cancer drugs, the quantitative structure-activity relationship model of compounds is usually used to select potential active compounds. However, the existing methods often have problems such as low model prediction performance, lack of overall consideration of the biological activity and related properties of compounds, and difficulty in directly selection candidate drugs. Therefore, this paper constructs a complete set of compound selection framework from three aspects: feature selection, relationship mapping and multi-objective optimization problem solving. In feature selection part, a feature selection method based on unsupervised spectral clustering is proposed. The selected features have more comprehensive information expression ability. In the relationship mapping part, a variety of machine learning algorithms are used for comparative experiments. Finally, the CatBoost algorithm is selected to perform the relationship mapping between each other, and better prediction performance is achieved. In the multi-objective optimization part, based on the analysis of the conflict relationship between the objectives, the AGE-MOEA algorithm is improved and used to solve this problem. Compared with various algorithms, the improved algorithm has better search performance. [ABSTRACT FROM AUTHOR]

The paper deals with the construction of interpretable machine learning models. The approximation problem is solved for a set of shapes on a contour image. Assumptions that the shapes are second-order curves are introduced. When approximating the shapes, information about the type, location, and shape of curves as well as about the set of their possible transformations is used. Such information is called expert information, and the machine learning method based on expert information is called expert-augmented learning. It is assumed that the set of shapes is approximated by the set of local models. Each local model based on expert information approximates one shape on the contour image. To construct the models, it is proposed to map second-order curves into a feature space in which each local model is linear. Thus, second-order curves are approximated by a set of linear models. In a computational experiment, the problem of approximating an iris on a contour image is considered. [ABSTRACT FROM AUTHOR]

In recent years, deep learning methods have been widely used for road extraction in remote sensing images. However, the existing deep learning semantic segmentation networks generally show poor continuity in road segmentation due to the high-class similarity between roads and buildings surrounding roads in remote sensing images, and the existence of shadows and occlusion. To deal with this problem, this paper proposes strip attention networks (SANet) for extracting roads in remote sensing images. Firstly, a strip attention module (SAM) is designed to extract the contextual information and spatial position information of the roads. Secondly, a channel attention fusion module (CAF) is designed to fuse low-level features and high-level features. The network is trained and tested using the CITY-OSM dataset, DeepGlobe road extraction dataset, and CHN6-CUG dataset. The test results indicate that SANet exhibits excellent road segmentation performance and can better solve the problem of poor road segmentation continuity compared with other networks. [ABSTRACT FROM AUTHOR]