Search

Machine learning is a popular tool for solving problems, however, incorporating it into a use case with additional business logic poses many challenges. Training, managing and storing many different models is not an easy task, requiring the use of multiple frameworks and languages. To take full advantage of existing frameworks it is necessary to facilitate communication between different programming languages. This paper presents an approach to integrating machine learning in a real-world use case which involves predicting demand for a diverse set of products and combining it with business rules and other components to establish a system that improves and automates the ordering process. Machine learning models are trained on real-world data from a retailer in Austria and the predictions are incorporated into a heuristic that controls and manages stock levels. This work focuses on the challenges that emerge from the integration of machine learning and presents a message bus based architecture to address them. [ABSTRACT FROM AUTHOR]

Routing is one of the most important and time-consuming stages of physical design. As the process node of semiconductors keeps scaling down, the routing process faces increasing challenges, and the traditional solutions are not sufficiently efficient. In recent years, machine learning has aroused much interest in this context, and an increasing number of algorithms have introduced advanced machine learning techniques to help solve the routing problem. In this paper, we survey the recent development of machine learning-based routing algorithms. • This paper provides an overview of recent published machine learning-based routing algorithms. • Some trends on the development of machine learning-based routing algorithms are revealed. • The possible advantages and disadvantages on machine learning-based routing methods are discussed. [ABSTRACT FROM AUTHOR]

This paper considers the offline assignment of fixed priorities in partitioned preemptive real-time systems where tasks have precedence constraints. This problem is crucial in this type of systems, as having a good fixed priority assignment allows for an efficient use of the processing resources while meeting all the deadlines. In the literature, we can find several proposals to solve this problem, which offer varying trade-offs between the quality of their results and their computational complexities. In this paper , we propose a new approach, leveraging existing algorithms that are widely exploited in the field of Machine Learning: Gradient Descent, the Adam Optimizer, and Gradient Noise. We show how to adapt these algorithms to the problem of fixed priority assignment in conjunction with existing worst-case response time analyses. We demonstrate the performance of our proposal on synthetic task-sets with different sizes. This evaluation shows that our proposal is able to find more schedulable solutions than previous heuristics, approximating optimal but intractable algorithms such as MILP or brute-force, while requiring reasonable execution times. [ABSTRACT FROM AUTHOR]

Domain adaptation is a practicable tool in real world application where there exists data scarcity. Multi-source domain adaptation attracts increasing attention due to its ability to enrich transfer knowledge by combining information from multiple domains. However, knowledge transfer can trigger privacy concerns by accessing source data. In addition, multiple domains can have label heterogeneity problem. In this paper, to solve the mentioned problems, we conduct an incomplete multi-source domain adaptation (IMSDA) method which can address transfer learning with and without the access to source data. As far as we are aware, this is the first work handling source-free incomplete domain adaptation. To take the benefits of multiple sources, multi-task learning is adopted to learn a general source model which can perform on multiple domains. A data matching strategy with and without source data forcing target sample to source latent feature space is developed to combine with self-supervision to adapt source model to the target domain. Experiments on real-world datasets indicate the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Students face problems that might hinder their academic pursuit toward success, problems ranging from trivial matters such as class condition, feeling of the student to severe matters such as family breakdown, economic reasons, and many more. This is a major problem because students shape the future of a nation – which will affect many things in the future. Teachers are looking for an effective way to find what might generally be the best solution for solving certain problems, as each student may face different problems, solving one at a time is not possible with the number of students each year. In this paper, we will try to find factors that might hinder or improve student performance using Pearson correlation between each feature toward the student G3 result. Based on the result, past failures will negatively impact student grades with -0.360415 correlation, and then Mother's Education will positively impact student grades with 0.217147. After finding out which factor affects student grade, we try to predict student grade using ML models to prove whether that factor actually affects student grade. Our MLP 12-Neuron model performs the best with RMSE value of 4.32, followed by Random Forest with RMSE value of 4.52, and finally Decision Tree with RMSE value of 5.69. [ABSTRACT FROM AUTHOR]

In real-world scenarios, new audio classes with insufficient samples usually emerge continually, which motivates the study of few-shot class-incremental audio classification (FCAC) in this paper. FCAC aims to enable the model to recognize new audio classes while remembering the base ones continually. To solve the FCAC problem, the discriminability of the prototypes is vital to the model's classification performance. Thus, we proposed a method to learn the discriminative prototypes from two aspects. First, since the generalization ability of the embedding module (EM) significantly affects the discriminability of the prototypes, the proposed method employs a scheme of pseudo-episodic incremental training to train the EM by simulating the test scenario. Second, to enable the model to achieve a balanced classification performance on both base and new audio classes, the proposed method employs a selective-attention module to adjust different prototypes to enhance their discriminability. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in solving the FCAC problem. Notably, the proposed method achieves a comprehensive performance score (CPS) of 87.82% and 59.25% on the Neural Synthesis musical notes of 100 classes (NSynth-100) and Free sound clips of 89 classes (FSC-89) datasets, respectively, which outperforms the comparison methods. Our code is available at https://github.com/chester-w-xie/DPL_FCAC. • The few-shot class-incremental audio classification problem is studied in this paper. • The proposed method expands the model with discriminative prototypes. • A scheme is proposed to enhance the generalization ability of the embedding module. • A selective-attention-based module is proposed for prototype adjustment. • The proposed method achieves state-of-the-art performance in solving the problem. [ABSTRACT FROM AUTHOR]

"Open world" environments are those in which novel objects, agents, events, and more can appear and contradict previous understandings of the environment. This runs counter to the "closed world" assumption used in most AI research, where the environment is assumed to be fully understood and unchanging. The types of environments AI agents can be deployed in are limited by the inability to handle the novelties that occur in open world environments. This paper presents a novel cognitive architecture framework to handle open-world novelties. This framework combines symbolic planning, counterfactual reasoning, reinforcement learning, and deep computer vision to detect and accommodate novelties. We introduce general algorithms for exploring open worlds using inference and machine learning methodologies to facilitate novelty accommodation. The ability to detect and accommodate novelties allows agents built on this framework to successfully complete tasks despite a variety of novel changes to the world. Both the framework components and the entire system are evaluated in Minecraft-like simulated environments. Our results indicate that agents are able to efficiently complete tasks while accommodating "concealed novelties" not shared with the architecture development team. [ABSTRACT FROM AUTHOR]

Medical artificial intelligence (AI) systems have been remarkably successful, even outperforming human performance at certain tasks. There is no doubt that AI is important to improve human health in many ways and will disrupt various medical workflows in the future. Using AI to solve problems in medicine beyond the lab, in routine environments, we need to do more than to just improve the performance of existing AI methods. Robust AI solutions must be able to cope with imprecision, missing and incorrect information, and explain both the result and the process of how it was obtained to a medical expert. Using conceptual knowledge as a guiding model of reality can help to develop more robust, explainable, and less biased machine learning models that can ideally learn from less data. Achieving these goals will require an orchestrated effort that combines three complementary Frontier Research Areas: (1) Complex Networks and their Inference, (2) Graph causal models and counterfactuals, and (3) Verification and Explainability methods. The goal of this paper is to describe these three areas from a unified view and to motivate how information fusion in a comprehensive and integrative manner can not only help bring these three areas together, but also have a transformative role by bridging the gap between research and practical applications in the context of future trustworthy medical AI. This makes it imperative to include ethical and legal aspects as a cross-cutting discipline, because all future solutions must not only be ethically responsible, but also legally compliant. • 3 frontier research areas facilitating ethical responsible and legally compliant AI. • Networks & their inference, graph causal models, verification/explainability methods. • Unified view of information fusion enabling integrative multi-modal machine learning. • Counterfactual paths to support explainability, interpretability and causability. • Fostering cross domain research to solve challenges within multi-science problems. [Display omitted] [ABSTRACT FROM AUTHOR]

We present a novel methodology based on Physics-Informed Neural Networks (PINNs) for solving systems of partial differential equations admitting discontinuous solutions. Our method, called Gradient-Annihilated PINNs (GA-PINNs), introduces a modified loss function that forces the model to partially ignore high-gradients in the physical variables, achieved by introducing a suitable weighting function. The method relies on a set of hyperparameters that control how gradients are treated in the physical loss. The performance of our methodology is demonstrated by solving Riemann problems in special relativistic hydrodynamics, extending earlier studies with PINNs in the context of the classical Euler equations. The solutions obtained with the GA-PINN model correctly describe the propagation speeds of discontinuities and sharply capture the associated jumps. We use the relative l 2 error to compare our results with the exact solution of special relativistic Riemann problems, used as the reference "ground truth", and with the corresponding error obtained with a second-order, central, shock-capturing scheme. In all problems investigated, the accuracy reached by the GA-PINN model is comparable to that obtained with a shock-capturing scheme, achieving a performance superior to that of the baseline PINN algorithm in general. An additional benefit worth stressing is that our PINN-based approach sidesteps the costly recovery of the primitive variables from the state vector of conserved variables, a well-known drawback of grid-based solutions of the relativistic hydrodynamics equations. Due to its inherent generality and its ability to handle steep gradients, the GA-PINN methodology discussed in this paper could be a valuable tool to model relativistic flows in astrophysics and particle physics, characterized by the prevalence of discontinuous solutions. [ABSTRACT FROM AUTHOR]

The representation of consistent mixed reality (XR) environments requires adequate real and virtual illumination composition in real-time. Estimating the lighting of a real scenario is still a challenge. Due to the ill-posed nature of the problem, classical inverse-rendering techniques tackle the problem for simple lighting setups. However, those assumptions do not satisfy the current state-of-art in computer graphics and XR applications. While many recent works solve the problem using machine learning techniques to estimate the environment light and scene's materials, most of them are limited to geometry or previous knowledge. This paper presents a CNN-based model to estimate complex lighting for mixed reality environments with no previous information about the scene. We model the environment illumination using a set of spherical harmonics (SH) environment lighting, capable of efficiently represent area lighting. We propose a new CNN architecture that inputs an RGB image and recognizes, in real-time, the environment lighting. Unlike previous CNN-based lighting estimation methods, we propose using a highly optimized deep neural network architecture, with a reduced number of parameters, that can learn high complex lighting scenarios from real-world high-dynamic-range (HDR) environment images. We show in the experiments that the CNN architecture can predict the environment lighting with an average mean squared error (MSE) of 7.85 × 10−4 when comparing SH lighting coefficients. We validate our model in a variety of mixed reality scenarios. Furthermore, we present qualitative results comparing relights of real-world scenes. [Display omitted] • Automatic end-to-end method to estimate the environment lighting in XR applications. • A CNN architecture that learns a latent-space of the environment lighting. • A methodology to generate egocentric mixed-reality-views from HDR panoramas. • Real-time lighting estimation that does not make assumptions about the XR scene. [ABSTRACT FROM AUTHOR]

Convex optimization is explored in this paper to maximize the security against power analysis attacks while minimizing the power and performance overheads for a cryptographic circuit employs a random dynamic voltage and frequency scaling (RDVFS) technique. To realize this optimization goal, firstly, non-convex functions are used for modeling all the critical parameters related to the RDVFS technique to create a non-convex objective function. Subsequently, the non-convex objective function is appropriately converted into a convex objective function in order to grab the global optimum value. By transforming the acquired convex objective function into the Lagrange dual function and using Karush–Kuhn–Tucker (KKT) conditions to solve this optimization problem, the best RDVFS strategy can be uncovered. As shown in the result, after employing convex optimization for a cryptographic circuit with a RDVFS technique, the corresponding figure-of-merit (FOM) is enhanced by 43.4%. Moreover, in comparison with the conventional machine learning technique, the proposed convex optimization technique improves the overall FOM related with RDVFS technique by 17.6% while reducing 85.3% computational time. • Random dynamic voltage and frequency scaling (RDVFS) is beneficial for improving the robustness of cryptographic circuits. • RDVFS technique undermines the performance and power saving of cryptographic circuits. • Convex optimization can be utilized for improving the security of RDVFS technique without increasing the overall overhead significantly. • Convex optimization methodology has a better efficiency for optimizing RDVFS technique as compared to the training methodology related with MLP neural networks. [ABSTRACT FROM AUTHOR]

Contemporary learning models for computer vision are typically trained on very large (benchmark) datasets with millions of samples. These may, however, contain biases, artifacts, or errors that have gone unnoticed and are exploitable by the model. In the worst case, the trained model does not learn a valid and generalizable strategy to solve the problem it was trained for, and becomes a "Clever Hans" predictor that bases its decisions on spurious correlations in the training data, potentially yielding an unrepresentative or unfair, and possibly even hazardous predictor. In this paper, we contribute by providing a comprehensive analysis framework based on a scalable statistical analysis of attributions from explanation methods for large data corpora. Based on a recent technique — Spectral Relevance Analysis — we propose the following technical contributions and resulting findings: (a) a scalable quantification of artifactual and poisoned classes where the machine learning models under study exhibit Clever Hans behavior, (b) several approaches we collectively denote as Class Artifact Compensation, which are able to effectively and significantly reduce a model's Clever Hans behavior, i.e., we are able to un-Hans models trained on (poisoned) datasets, such as the popular ImageNet data corpus. We demonstrate that Class Artifact Compensation, defined in a simple theoretical framework, may be implemented as part of a neural network's training or fine-tuning process, or in a post-hoc manner by injecting additional layers, preventing any further propagation of undesired Clever Hans features, into the network architecture. Using our proposed methods, we provide qualitative and quantitative analyses of the biases and artifacts in, e.g., the ImageNet dataset, the Adience benchmark dataset of unfiltered faces, and the ISIC 2019 skin lesion analysis dataset. We demonstrate that these insights can give rise to improved, more representative, and fairer models operating on implicitly cleaned data corpora. [Display omitted] • We extend Spectral Relevance Analysis for large scale analyses of Clever Hans traits. • We introduce Class Artifact Compensation to unlearn or suppress confounders. • We show that Deep Neural Networks tend to overfit to Clever Hans artifacts in data. • Using both methods, we discover and unlearn Clever Hans artifacts on modern datasets. • We demonstrate that Class Artifact Compensation leads to more trustworthy models. [ABSTRACT FROM AUTHOR]

The data of the power Internet of Things (IOT) system is transferred from the IaaS layer to the SaaS layer. The general data preprocessing method mainly solves the problem of big data anomalies and missing at the PaaS layer, but it still lacks the ability to judge the high error data that meets the timing characteristics, making it difficult to deal with heterogeneous power inconsistent issues. This paper shows this phenomenon and its physical mechanism, showing the difficulty of building a quantitative model forward. A data-driven method is needed to form a hybrid model to correct the data. The research object is the electricity meter data on both sides of a commercial building transformer, which comes from different power IOT systems. The low-voltage side was revised based on the high-voltage side. Compared with the correction method based on purely using neural networks, the combined method, Linear Regression (LS) + Differential Evolution (DE) + Extreme Learning Machine (ELM), further reduces the deviation from approximately 4% to 1%. • The heterogeneous electrical data from complex IOT system is inconsistent. • The positive mechanism of inconsistency has been revealed, but not easy to correct. • Reverse data correction used a combination of simple methods and the effect is better. [ABSTRACT FROM AUTHOR]

In this paper, a graph-based optimization problem related to epidemics control is studied. This multiobjective optimization problem requires determining which graph nodes to protect in order to limit the number of nodes affected by a threat spreading in the graph. In the paper, a classifier-based evolutionary algorithm is proposed in which population initialization, crossover, mutation and local search can use a previously trained classifier to decide which graph nodes to protect. The classifier is trained on a dataset collected by running an optimization algorithm on small problem instances and it is subsequently reused for solving multiple, larger instances of the same optimization problem. Selecting graph nodes to protect using a previously trained classifier allows the algorithm to find better solutions to the optimization problem compared to an algorithm not using a classifier. In the experiments, several machine learning models were evaluated and a multilayer perceptron (MLP) neural network was selected as the best performing classifier. The proposed optimization method was tested and found to work effectively when the MLP classifier was trained on data collected when solving problem instances with 1000 graph nodes. In tests on problem instances with up to 20000 nodes the proposed method outperformed an evolutionary optimizer not using any classifier-based components thereby showing that knowledge can be transferred from smaller problem instances to larger ones. [Display omitted] • A graph-based optimization problem related to epidemics control is studied. • A classifier-based evolutionary multiobjective optimization algorithm is proposed. • A multilayer perceptron (MLP) neural network is the best performing classifier. • The proposed method outperforms an evolutionary optimizer not using any classifier-based components. • Results show that knowledge can be transferred from smaller problem instances to larger ones. [ABSTRACT FROM AUTHOR]

Truecasing is the process of finding the proper capitalization of words in a text where the correct case information of the word is not given. The case of letters is often lost while performing language translation from non-capitalized languages (eg: Japanese, Arabic, Hindi) to capitalized languages (eg: Greek, Latin, English). In Optical Character Recognition (OCR) where the different cases for letters like 'p' and 'P' are similar, the case as well as the context and proper meaning of the text is lost. A similar situation is observed for text generated from Automatic Speech Recognition. This paper proposes to solve the problem of truecasing while focusing on three different architectures, all based on transformers. Transformers have generally proved to outperform recurrent and convolutional models. A proposal for a novel light-weight transformer architecture has been made which performs competitively with existing models that are computationally expensive. To provide a basis for comparison, two existing transformer architectures were fine-tuned for truecasing - BERT and GPT-2. These architectures are analyzed and compared on various metrics. The proposed architecture shows an accuracy of 98.25%, which is also an improvement of 0.41 against the current state-of-art. [ABSTRACT FROM AUTHOR]

The paper presents an approach to machine learning involving the use of a hybrid method in Neural ODE. The authors consider a way of the Neural ODE application with an embedded hybrid method for intelligent manipulator control in a non-deterministic environment. To reduce computational resources, the number of layers of the neural network using ODE is reduced, the solution of which is carried out by a hybrid transformation method. The hybrid method includes normalizing the system, linear transformation, and polynomial transformation. Using the hybrid method in Neural ODE can increase the learning and computation speed. The transformation method has high accuracy with low computational complexity of the algorithm. As a result of applying the hybrid method, the original nonlinear system is transformed to a simpler form. To solve the simplified transformed system, the scheme of the numerical Runge-Kutta method of the third order with an adaptive step is used. The results of the computational experiment show an increase in the speed and accuracy of the calculation. The presented approach to implementing the hybrid method in Neural ODE allows faster training of the neural network of the world model on limited data obtained from the real environment. After training on the data generated by the model of the world, a robotic arm always successfully reaches the object during subsequent interaction with the real environment. This solves the problem of the limited ability of the agent to interact when learning with the real environment in robotic systems. [ABSTRACT FROM AUTHOR]

Nearest neighbor-based methods are classic techniques that, due to their efficiency, still are widely used today. However, they have not been broadly applied to solve the multi-instance multi-label (MIML) problem, a supervised learning paradigm that combines multi-instance (MI) and multi-label (ML) learning. This work presents new neighbor-based approaches for solving MIML problems. On the one hand, MIML data are transformed into ML data and ML nearest neighbor algorithms are used. On the other hand, algorithms that directly address MIML data and use a bag-based distance are proposed. A comprehensive study and an overall comparison have been conducted to study the performance of these methods using different configurations. Experiments included 16 datasets and 8 performance metrics. The results and statistical tests showed that the problem transformation applied and the distance function used impacted the performance and that the approaches that do not transform the problem obtained the best predictive results. Furthermore, most of the proposed algorithms outperformed the MIMLkNN algorithm, the state-of-art algorithm for MIML learning based on nearest-neighbor. Therefore, the relevance and capabilities of neighbor-based approaches to obtain competitive results in MIML learning are shown. Finally, all the algorithms developed in this paper have been included in the MIML library to facilitate the comparison with other future proposals. • New kNN methods for MIML based on logistic regression, gravitational model, and MAP. • Exhaustive study with 16 datasets and 8 evaluation measures to study the performance. • The transformation and the distance function influenced the predictive performance. • The gravitational model gets the best results with reasonable time complexity. • All the proposals outperform the state-of-art kNN-based algorithm in MIML. [ABSTRACT FROM AUTHOR]

This paper investigates how institutional pressures affect the development of Circular Economy (CE) in firms. Using Institutional Entrepreneurship as a theoretical framework, this paper considers three different levels of institutional pressures (coercive, normative, and mimetic) to examine the effect of each pressure and their interactions on the development of CE. Seeking to clarify the debate on the effect of institutional pressures, this paper considers that the main limitation arises from the fact that previous research has analysed the relationship between institutional pressures without considering the interaction between them and the non-linearity of the processes. Deviating from previous papers, our analysis combines regression methods with Machine learning (i.e. Artificial Neural Networks), and employs data from the EU survey on Public Consultation on the Circular Economy. This research finds that while coercive pressures have a compulsory effect on the development of CE, mimetic and normative pressures do not have an effect by themselves, but only in interaction with coercive pressures. Moreover, this paper shows that the application of machine learning tools has an important contribution in solving interaction problems. From the perspective of environmental policy, this means that a comprehensive policy is required, which implies the coexistence or interaction of the three types of pressures. [Display omitted] • Discrepancies on the effect of institutional pressures are due to methodology. • Coercive pressure has a compulsory effect on the development of Circular Economy. • Mimetic & Normative pressures only have effect in interaction with coercive pressure. • Artificial Neural Networks contribute to solving interaction problems. • Comprehensive environmental policy that considers these interactions is required. [ABSTRACT FROM AUTHOR]

Graph-based semi-supervised dimension reduction can use the inherent graph structure of samples to propagate label information, and has become a hot research field in machine learning. However, most current methods have strict linear constraints and cannot handle data sampled from nonlinear manifolds; and rely on predefined graphs, which cannot capture the local structure information of data and cannot handle complex non-Gaussian data. To address these issues, this paper proposes a new locality-preserved flexible dimension reduction framework, called Semi-supervised Flexible Adaptive Graph Embedding (SFAG), which learns the embedding space that can preserve the local neighborhood structure by constructing a k 1 -nearest neighbor graph over labeled samples. Then, another k 2 -nearest neighbor graph is constructed on all samples to adaptively construct the optimal graph, clustering labeled and unlabeled embedding sample points with neighborhood relations into the same sub-manifold sharing the same label information. Last but not least, the hard linear projection constraint is relaxed by adding residual terms to obtain not only the nonlinear embedding of the training samples, but also the linear projection matrix applied directly to the out-of-sample. In addition, two different semi-supervised dimension reduction methods for adaptive construction of optimal graphs are proposed based on the SFAG framework. Several evaluation experiments validate the effectiveness of our method in exploring manifold structures and classification tasks. • A novel semi-supervised adaptive flexible dimension reduction method is proposed. • The local and global structure of the data is explored by constructing kNN graphs. • The hard linear constraints on the projection are relaxed by adding a residual term. • An algorithm is designed to solve the norm-constrained problem of feature extraction. [ABSTRACT FROM AUTHOR]

Multi-agent reinforcement learning (MARL) has been successfully applied in many fields. In MARL, the policy evaluation problem is one of crucial problems. In order to solve this problem, distributed Temporal-Difference (TD) learning algorithm is one of the most popular methods in a cooperative manner. Despite its empirical success, however, the theory of the adaptive variant of distributed TD learning still remain limited. To fill this gap, we propose an adaptive distributed temporal-difference algorithm (referred to as MS - ADTD) under Markovian sampling over time-varying networks. Furthermore, we rigorously analyze the convergence of MS - ADTD , the theoretical results show that the local estimation can converge linearly to the optimal neighborhood. Meanwhile, the theoretical results are verified by simulation experiments. • This paper proposes a distributed adaptive TD learning algorithm under Markovian sampling. • It theoretically analyzes the non-asymptotic convergence performance of the proposed algorithm. • It verifies the performance of the proposed algorithm by experiments. [ABSTRACT FROM AUTHOR]

IEEE 802.11ah, also known as Wi-Fi HaLow, improves the scalability of the Internet of Things (IoT) by connecting a large number of low-power devices. IEEE 802.11ah has developed a novel feature called Restricted Access Window (RAW) that aims to address one of the main problems of the IoT: excessive channel contention in large sensor networks. All stations can access the channel from the same group when using RAW, which allows the Access Point (AP) to split stations into different groups. Existing station grouping schemes support only the homogeneous traffic scenarios, having the same data transmission interval, packet size, and Modulation and Coding Scheme (MCS). In this paper, we highlight two contributions to solving this problem. The first is a Machine Learning (ML) based model that predicts RAW configuration parameters and performance under specific network conditions. It predicts accurately and is very quick to train. Second, a technique for a channel access mechanism that uses an ML model to determine the optimal RAW station grouping configuration for dynamic traffic conditions and heterogeneous stations. We compare our ML-based model with the existing state-of-the-art solutions. We calculated the performance of our proposed model and compared it to traditional 802.11ah and the Traffic Adaptive RAW Optimization Algorithm (TAROA). The results show that our proposed model outperforms a significant improvement in terms of throughput, packet received ratio, delay, and execution time. [ABSTRACT FROM AUTHOR]

• Applying MSE to the field of lithology discrimination. • Designing a lithology recognition model based on SA optimized SVM algorithm. • Forming a set of lithology recognition methods based on machine learning. • Establishing a lithology intelligent recognition model. Lithology identification is an important part of petroleum drilling engineering. Accurate identification of lithology is the foundation to ensure the smooth operation of drilling engineering. Conventional lithology recognition mainly relies on human experience. The recognition accuracy depends on the level of technical personnel and the recognition response time is lagging. It is difficult to meet the demand. How to achieve rapid and intelligent recognition of lithology is one of the core technical problems faced by oil drilling. In order to solve this problem, this paper uses the mechanical specific energy and other ground parameters to establish a rapid and intelligent recognition method of lithology based on the simulated annealing optimization support vector machine model. In order to improve the accuracy of the model recognition, a large number of methods have been developed from two classes and multiple classes, simulation analysis results show that the lithology recognition model based on the principle of mechanical specific energy and the simulated annealing optimization support vector machine algorithm can predict a priori unknown data with a prediction accuracy of over 90%. Compared with the support vector machine model and the K-means model, simulated annealing optimization support vector machine is used for comparative analysis, the algorithm to establish a lithology recognition model has better performance and higher accuracy. The intelligent identification model of lithology based on the principle of mechanical specific energy and simulated annealing optimization support vector machine algorithm established in this paper can quickly and accurately identify lithology, and provide new technical support for oil drilling formation analysis. [ABSTRACT FROM AUTHOR]

• This paper proposes an unsupervised deep learning model. • This paper proposes an optimized sparse representation method. • This paper proposes a new image fusion algorithm based on unsupervised deep learning-optimized sparse representation. The image fusion method based on deep learning has problems such as the supervised learning of the model, the edge and noise of the fused image, and the setting of the image fusion weight map. To solve these problems, this paper proposes an end-to-end unsupervised deep learning model that performs one-to-many focus image fusion. It solves the training problem encountered by supervised deep learning models while avoiding the unreasonable image fusion weight maps. In addition, this paper proposes an optimized sparse representation method that divides an image into a target area and a background area. Then, it uses super complete dictionary learning to obtain a sparse representation of the image background area. This approach makes the proposed unsupervised deep learning image fusion method robust to noise. Finally, using this method to carry out image fusion experiments, the results show that the quality evaluation indicators of the fused image obtained by this method substantially outperform those of both mainstream machine learning and deep learning image fusion methods. [ABSTRACT FROM AUTHOR]

This paper presents an online rescue method based on offline learning of dynamics knowledge to solve the problem of the optimal rescue orbit and flight trajectory optimization (OROTO) of launch vehicles experiencing thrust-drop faults. Due to the unknown of the rescue orbit, solving the OROTO problem by the conventional aerospace orbit and trajectory optimization method is time-consuming. In this paper, benefiting from the decision-making of the optimal rescue orbit by the machine learning technology, the OROTO problem is decoupled into a decision-making of the optimal rescue orbit and a trajectory optimization problem with a known orbit. In the decision-making of the optimal rescue orbit, instead of the conventional iteration optimization process based on dynamics, the optimal rescue orbit is determined by the "fault-rescue" knowledge integration (FRKI) model which consists of probabilistic neural network (PNN) and radial basis function neural network (RBFNN) trained by "fault-rescue" knowledge. In the trajectory optimization part, the output of the FRKI model provides terminal constraints for the trajectory optimization problem to decrease the search scope for the optimal solution. Numerical simulation results show that the proposed method can solve the OROTO problem rapidly and accurately, and can potentially be implemented for online applications. • Proposing an accurate and rapid rescue method for launch vehicles experiencing thrust drop faults. • The "fault-rescue" knowledge integration model is used to replace the conventional iteration optimization process. • Decoupling the original rescue problem into a decision-making of the rescue orbit and an online trajectory optimization. • Overcoming the positive error of the decision-making by an online two-step trajectory optimization method. [ABSTRACT FROM AUTHOR]

Resource constrained project scheduling problem is an NP-hard problem that attracts many researchers because of its complexity and daily use. In literature there are a lot of various solving methods for this problem. The priority rules are one of the prominent methods used in practice. Because of their simplicity, speed, and possibility to react to changes in the system, they can be used in a dynamic environment. In this paper, ensembles of priority rules were created to improve the performance of priority rules created with genetic programming. For ensemble creation, four different methods will be considered: simple ensemble combination, BagGP, BoostGP, and cooperative coevolution. The priority rules that are part of the ensemble will be combined with the sum and vote methods in reaching the final decision. Additionally, the ensemble subset search method will be applied to the created ensembles to find the optimal subset of priority rules. The results achieved in this paper show that ensembles of priority rules can achieve significantly better results than those achieved when using only a single priority rule. • Ensembles of priority rules were created. • Four different learning methods for creating ensembles were used. • Two different methods for combining individual rules in ensembles were analyzed. • The ensemble subset search method was applied to created ensembles. • Ensembles of priority rules achieved better results than single priority rule. [ABSTRACT FROM AUTHOR]

Artificial intelligence (AI), often known as machine learning, is a powerful tool for solving engineering problems. The evaluation of the network reliability of a flow network is a NP-hard problem, with computational effort growing exponentially with the number of nodes and arcs in the network. Also, the components assignment issue is NP-hard, and the computational effort increases with the number of available components. Many candidate solutions are typically examined during optimal components or optimal capacity assignment, each requiring reliability calculation. Consequently, this paper proposes an artificial neural network (ANN) predictive model to evaluate the flow network reliability. The neural network is one of the artificial intelligence tools constructed, trained, and validated using the maximum capacity of each component input and the network reliability as the target. The proposed ANN model provides empirical proof that neural networks can accurately estimate reliability by modeling the connection between the maximum capacities of network components and the reliability value. [ABSTRACT FROM AUTHOR]

The depth of field (DOF) of camera equipment is generally limited, so it is very difficult to get a fully focused image with all the objects clear after taking only one shot. A way to obtain a fully focused image is to use a multi-focus image fusion method, which fuses multiple images with different focusing depths into one image. However, most of the existing methods focus too much on the fusion accuracy of a single pixel, ignoring the integrity of the target and the importance of shallow features, resulting in internal errors and boundary artifacts, which need to be repaired after a long time of post-processing. In order to solve these problems, we propose a cascade network based on Transformer and attention mechanism, which can directly obtain the decision map and fusion result of focusing/defocusing region through end-to-end processing of source image, avoiding complex post-processing. For improving the fusion accuracy, this paper introduces the joint loss function, which can optimize the network parameters from three aspects. Furthermore, In order to enrich the shallow features of the network, a global attention module with shallow features is designed. Extensive experiments were conducted, including a large number of ablation experiments, 6 objective measures and a variety of subjective visual comparisons. Compared with 9 state-of-the-art methods, the results show that the proposed network structure can improve the quality of multi-focus fusion images and the performance is optimal. • Intermediate attention mechanism and decision module to generate the final decision graph. • Reinforcing the learning ability of local semantic information. • Achieving a balance between PATCH self-attention and global self-attention. • The shallow feature attention module is used to extract the initial global self-attentive feature map to compensate for Transformer's poor primary semantic acquisition capability. [ABSTRACT FROM AUTHOR]

Recently, low-rank matrix completion (LRMC) has shown a promising performance in human motion capture (mocap) data recovery, which assumes that the input data is low-rank. However, for most mocap data comprising multiple different activities, only using LRMC to restore the data may result in erroneous results, since the low-rank assumption is not necessarily satisfied. The current method solves this problem in a two-stage manner by combining the LRMC and subspace clustering. But it ignores the fact that the processes of recovering and clustering depend on each other, and so is sub-optimal. In this paper, we propose a novel discrete subspace structure (DSS) constrained approach to recover human mocap data, which jointly optimizing the tasks of subspace clustering and the LRMC. The proposed DSS algorithm learns different subspaces' indicators such that the mocap recovery problem is divided into several LRMC subproblems, where each matrix of the points drawn from a single subspace is low-rank. As a byproduct, we can obtain the temporal clustering results of mocap data. To better approximate the low-rank property, we utilize the t th power of Schatten p-norm to approximate the rank instead of the nuclear norm. Moreover, we add two regularization terms to take care of the noise effect and temporal stability of mocap data. The obtained model comes down to a non-convex optimization problem, which we solve tactically and efficiently by employing the alternating direction method of multipliers (ADMM). Experiments on the CMU dataset and the HDM05 dataset validate the effectiveness of the proposed algorithm in both mocap data recovery and temporal subspace clustering. • A DSS model is proposed which formulates the motion recovery and subspace clustering into a unified non-convex framework. • The t-th power of Schatten p-norm is utilized to approximate the rank rather than the nuclear norm. • An iterative algorithm is tactically designed to efficiently solve the resulting optimization problem. • The temporal subspace clustering results of motion sequences as a byproduct can be obtained by means of the learned subspaces' indicators. [ABSTRACT FROM AUTHOR]

Long-term and long-sequence prediction for multivariate time series (MTS) data is an important and challenging problem. The key to solving this problem is learning the long-term dependency period in a time series. Different attention-based models like RNN variants and transformers are actively researched to provide short-term and long-term predictions for a time series. These attention-based models consider time series as sequential data. They are not designed to learn the different characteristics (trends, seasonality, and irregularity) of a time series. The complex temporal patterns of data prohibit these models from finding a reliable period of dependency. This paper proposes a dual-stage advanced deep learning framework (DST2V-TRANSFORMER), which exploits moving average smoothing, the percentage change in data, and time embedding to improve transformers for time series forecasting. In the first stage, data is smoothened using the moving average method, and to learn the periodic and non-periodic temporal patterns in time series, a vector representation for time is added as a feature to the data. This preprocessed data is passed through a transformer to learn temporal correlation and find the reliable period of dependency. The DST2V-TRANSFORMER achieves state-of-the-art efficiency and accuracy, with at least a 49% relative improvement in error on seven MTS datasets from the financial domain. [ABSTRACT FROM AUTHOR]

For years, meta-heuristic algorithms have been widely studied and many improved versions have been developed: from the evolution of the swarm topologies of the Particle Swarm Optimization algorithm, to the using of machine learning to Differential Evolutionary algorithms. However, the tuning of the fundamental meta-heuristic parameters has been less studied, but may lead to significant improvements on the convergence accuracy of these algorithms. This paper aims at developing an automated methodology to calibrate the parameters of population-based meta-heuristic algorithms for optimization problems. Based on the kriging estimation of the best combination of parameters, the Automated parameter tuning of Meta-heuristics (AptM) methodology gives the optimal algorithm setup for each considered problem in order to lead to a better convergence accuracy. The proposed AptM methodology is used to tune three different meta-heuristic algorithms, each applied to twelve mathematical unimodal or multimodal objective functions. AptM methodology performance is assessed by comparison of classical setups usually used in the literature. The numerical results show that the AptM methodology allows a significant improvement of the convergence accuracy of meta-heuristics with an average improvement of 62.02%, 69.12% and 64.94% on optimization problems defined in dimensions 10, 30 and 50 respectively. An experimental criterion is defined based on the convergence accuracy of the AptM methodology over the classical setups, assessing the AptM performances. The previous experimental criterion allows to compare the AptM methodology over the base-set. The AptM methodology shows a significant improvement of the algorithms performance on 97.2% of the tested problems. [ABSTRACT FROM AUTHOR]

Imbalanced learning is important and challenging since the problem of the classification of imbalanced datasets is prevalent in machine learning and data mining fields. Sampling approaches are proposed to address this issue, and cluster-based oversampling methods have shown great potential as they aim to simultaneously tackle between-class and within-class imbalance issues. However, all existing clustering methods are based on a one-time approach. Due to the lack of a priori knowledge, improper setting of the number of clusters often exists, which leads to poor clustering performance. Besides, the existing methods are likely to generate noisy instances. To solve these problems, this paper proposes a deep instance envelope network-based imbalanced learning algorithm with the multilayer fuzzy c-means (MlFCM) and a minimum interlayer discrepancy mechanism based on the maximum mean discrepancy (MIDMD). This algorithm can guarantee high quality balanced instances using a deep instance envelope network in the absence of prior knowledge. First, the MlFCM is designed for the original minority class instances to obtain deep instances and increase the diversity of instances. Then, the MIDMD is proposed to avoid the generation of noisy instances and maintain the consistency of the interlayers of instances. Next, the multilayer FCM and minimum interlayer discrepancy mechanism are combined to construct a deep instance envelope network – the MlFC&IDMD. Finally, an imbalance learning algorithm is proposed based on the MlFC&IDMD. In the experimental section, thirty-three popular public datasets are used for verification, and over ten representative algorithms are used for comparison. The experimental results show that the proposed approach significantly outperforms other popular methods. • For the first time, a multilayer FCM (MlFCM) algorithm for classification is proposed to mine the instances for more information. • A minimum interlayer discrepancy mechanism (MIDMD) is proposed to make the distribution of instances before and after clustering consistent. • A deep instance envelope network (MlFC&IDMD) is constructed by combining the MlFCM and MIDMD. • An efficient strategy is proposed to determine the number of layers L of the deep instance envelope network. • A new imbalance learning algorithm is proposed based on the MlFC&IDMD. [ABSTRACT FROM AUTHOR]

The alternating direction method of multipliers (ADMM) is frequently used to solve optimization problems derived from machine learning and statistics. Hence it is important to investigate the asymptotic behavior of ADMM. Most works in this direction deal with smooth objective functions excluding various important applications in the nonsmooth case. In this paper, we study the asymptotic behavior of the solutions of differential inclusions arising from ADMM applied to nonsmooth objective functions. More precisely, we associate a multiflow to this differential inclusion and analyze the structure of both ω -limit points and global attractors for this multiflow. [ABSTRACT FROM AUTHOR]

Recognizing the corresponding neural activities of independent components(ICs) obtained by independent component analysis(ICA) is of prime importance to take use of ICA in EEG analysis. There are many methods trying to solve this problem. But most of them combining ICA, a unsupervised method, and recognition of ICs in a separate way. In this paper, we propose a supervised method to extract the independent components corresponding to different motion imagery(MI) activities in the brain. By designing a new optimization objective and solving it, we combine the idea of ICA with principle of MI in an individual algorithm. From the perspective of event-related desynchronization and synchronization (ERD/ERS), specific frequency band power of the motion-related component should be enhanced or suppressed when executing or imaging movement of body. Therefore, the new optimization function extract the components that satisfy both independence and band power maximization for specific motions. Then, we solve this optimization problem based on the fixed-point iteration scheme. In the experimental stages, we show that our methods can extract motion-related independent components without losing independence. Experimental results show that, although basing on the principle of ERD/ERS, our methods' effectiveness can be verified in the perspective of movement-related potential (MRP). Additionally, by identifying features in the extracted motion-related independent components, we can achieve better motion recognition accuracy. When using the proposed algorithms with different schema, the results yielded significant accuracy imporvements of 6.9%(p < 0.001) and 7.9%(p < 0.01). [ABSTRACT FROM AUTHOR]

Feature selection (FS) is recognized as one of the majority public and challenging problems in the Machine Learning domain. FS can be examined as an optimization problem that needs an effective optimizer to determine its optimal subset of more informative features. This paper proposes a wrapper FS method that combines chaotic maps (CMs) and binary Group Search Optimizer (GSO) called CGSO, which is used to solve the FS problem. In this method, five chaotic maps are incorporated with the GSO algorithm's main procedures, namely, Logistic, Piecewise, Singer, Sinusoidal, and Tent. The GSO algorithm is used as a search strategy, while k-NN is employed as an induction algorithm. The objective function is to integrate three main objectives: maximizing the classification accuracy value, minimizing the number of selected features, and minimizing the complexity of generated k-NN models. To evaluate the proposed methods' performance, twenty well-known UCI datasets are used and compared with other well-known published methods in the literature. The obtained results reveal the superiority of the proposed methods in outperforming other well-known methods, especially when using binary GSO with Tent CM. Finally, it is a beneficial method to be utilized in systems that require FS pre-processing. • A novel feature selection method is proposed for classification task. • Five Chaotic Group Search Optimizers are investigated. • Twenty UCI datasets were used in the experiments. • Our methods got a higher accuracy results using a fewer number of features. [ABSTRACT FROM AUTHOR]

Corporate financial distress prediction research has been ongoing for more than half a century, during which many models have emerged, among which ensemble learning algorithms are the most accurate. Most of the state-of-the-art methods of recent years are based on gradient boosted decision trees. However, most of them do not consider using feature importance for feature selection, and a few of them use the feature importance method with bias, which may not reflect the true importance of features. To solve this problem, a heuristic algorithm based on permutation importance (PIMP) is proposed to modify the biased feature importance measure in this paper. This method ranks and filters the features used by machine learning models, which not only improves accuracy but also makes the results more interpretable. Based on financial data from 4,167 listed companies in China between 2001 and 2019, the experiment shows that compared with using the random forest (RF) wrapper method alone, the bias in feature importance is indeed corrected by combining the PIMP method. After the redundant features are removed, the performance of most machine learning models is improved. The PIMP method is a promising addition to the existing financial distress prediction methods. Moreover, compared with traditional statistical learning models and other machine learning models, the proposed PIMP-XGBoost offers higher prediction accuracy and clearer interpretation, making it suitable for commercial use. • The model Combines a corrected feature selection measure and XGBoost. • Permutation importance can correct the bias of feature importance. • The model is validated on Chinese listed companies datasets over five metrics. • The model is proved to outperform several benchmark techniques. • The feature importance and partial dependence plot enhance model interpretation. [ABSTRACT FROM AUTHOR]

• A federated learning scheme with homomorphic re-encryption is proposed, which collects data from the IoT devices, completes model training through the fog nodes, and finally aggregates the data on the servers. • The proposed scheme can protect the data frome the IoT devices, and solve the problem of high computational cost and storage cost. • It give the security analysis and the experimental results, which show that our scheme are secure and efficient in the real world applications. Unlike traditional centralized machine learning, distributed machine learning provides more efficient and useful application scenarios. However, distributed learning may not meet some security requirements. For example, in medical treatment and diagnosis, an increasing number of people are using IoT devices to record their personal data, when training medical data, the users are not willing to reveal their private data to the training party. How to collect and train the data securely has become the main problem to be resolved. Federated learning can combine a large amount of scattered data for training, and protect user data. Compared with general distributed learning, federated learning is more suitable for training on scattered data. In this paper, we propose a privacy-preserving federated learning scheme that is based on the cryptographic primitive of homomorphic re-encryption, which can protect user data through homomorphic re-encryption and trains user data through batch gradient descent (BGD). In our scheme, we use the IoT device to encrypt and upload user data, the fog node to collect user data, and the server to complete data aggregation and re-encrypting. Besides, the security analysis and experimental results show that our scheme can complete model training while preserving user data and local models. [ABSTRACT FROM AUTHOR]

• We target MultiParametric optimization of chemical process operation under uncertainty. • Situations where process model is complex, highly nonlinear and/or black-box are considered. • A ML-based method for the solution of this problem is presented in this work. • ML models are built to capture the mapping between the optimal solution and the uncertain parameters. • The method shows high accuracy, robustness and applicability to many case-studies. Chemical process operation optimization aims at obtaining the optimal operating set-points by real-time solution of an optimization problem that embeds a steady-state model of the process. This task is challenged by unavoidable Uncertain Parameters (UPs) variations. MultiParametric Programming (MPP) is an approach for solving this challenge, where the optimal set-points must be updated online, reacting to sudden changes in the UPs. MPP provides algebraic functions describing the optimal solution as a function of the UPs, which allows alleviating large computational cost required for solving the optimization problem each time the UPs values vary. However, MPP applicability requires a well-constructed mathematical model of the process, which is not suited for process operation optimization, where complex, highly nonlinear and/or black-box models are usually used. To tackle this issue, this paper proposes a machine learning-based methodology for multiparametric solution of continuous optimization problems. The methodology relies on the offline development of data-driven models that accurately approximate the multiparametric behavior of the optimal solution over the UPs space. The models are developed using data generated by running the optimization using the original complex process model under different UPs values. The models are, then, used online to, quickly, predict the optimal solutions in response to UPs variation. The methodology is applied to benchmark examples and two case studies of process operation optimization. The results demonstrate the methodology effectiveness in terms of high prediction accuracy (less than 1% of NRMSE, in most cases), robustness to deal with problems of different natures (linear, bilinear, quadratic, nonlinear and/or black boxes) and significant reduction in the complexity of the solution procedure compared to traditional approaches (a minimum of 67% reduction in the optimization time). [ABSTRACT FROM AUTHOR]

• A framework of building energy modeling for data predictive control is developed. • The framework integrates active learning strategy with feature selection technique. • The framework is demonstrated in a virtual testbed (EnergyPlus and BCVTB). • The case study shows improvement in energy modeling accuracy and DPC performance. Three gaps impede the development of cost-effective and accurate data-driven building energy modeling/models (DBEM) for energy forecasting and predictive control strategies. Gap 1: data bias is common in building operation data, but this topic is hardly studied in DBEM; Gap 2: high data dimensionality is common in DBEM, but a systematic and scalable methodology is lacking to solve the problem; Gap 3: the interactions between data bias and high dimensionality have not been systematically studied for DBEM and predictive control in buildings. In this paper, to address the three gaps mentioned above, we develop a framework that integrates active learning and feature selection for DBEM used for whole building data predictive control (or DPC, which is a branch of model predictive control). The framework provides a systematic methodology and automatic workflow that starts with raw data from building automation systems to the establishment of data-driven energy models for DPC controllers. The developed strategies and framework are evaluated in a virtual testbed based on EnergyPlus and BCVTB. Improved performance and reduced computational complexity are observed from the DBEM built with the developed framework, as well as the DPC controller based on that DBEM, indicating the effectiveness of the developed framework. [ABSTRACT FROM AUTHOR]

Due to the actual operation of the wind turbine, the collected fault data sets are limited, and it is difficult to realize fault diagnosis through the correlation between variables. To solve this problem, a fault diagnosis method based on virtual expansion and spherical mapping model is proposed in this paper. Firstly, Hermite interpolation is applied to discrete wind power data samples to obtain an interpolation curve about the characteristics of the sample, and a synchronous sampling method is adopted for the interpolation curve to construct a virtual sample. Then, the features of the virtual sample are mapped to a three-dimensional space. Define the spherical data model and perform spherical fitting in a three-dimensional coordinate system. Finally, feature extraction is performed on the fitted spherical surface for training and testing extreme learning machine (ELM). The distribution law of fault data in the spherical model is summarized. Using the data generated based on the Bootstrap method as a control group, comparative experiments were carried out in back-propagation neural network (BP), Probabilistic Neural Network (PNN), General Regression Neural Network (GRNN), and Support Vector Machine (SVM), which verified the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning-based target detection techniques have already made a wide-range impact on our daily life. Currently, a feature pyramid is a widely utilized technique for multiscale target detection, the effectiveness of the technique has already been proved. Nevertheless, in the pyramid structure, problems, such as multiscale feature alignment, model turmoil after fusion, feature redundancy, and no-local feature fusion, exist. In this paper, we propose a novel progressive structure network to solve the aforementioned problems. The proposed structure contains three modules: multiscale feature alignment fusion, different scale channels & spatial location adaptive weighted fusion, and multiscale global and local feature fusion. The proposed structure is capable of fusing information from different feature layers more effectively. Subsequently, the semantic gaps among different scales can be reduced. Furthermore, the proposed structure can maintain the stability of the detection network and its performance has been proved by comparing with other state-of-art feature fusion method. The proposed progressive network structure has also been applied to actual target detection tasks and the practical application effectiveness of our method has been verified. [ABSTRACT FROM AUTHOR]