Search

Label distribution learning represents the relevance of labels to samples using description degree, which can provide richer semantic information, thus finding wider applications. Exploiting label correlations is an effective approach to narrow down the hypothesis space of label distribution learning models. In existing works that utilize low-rank assumptions or label linear dependence to mine correlations, it is assumed that a label can be linearly expressed by other labels. However, this assumption can only be satisfied when there are linear dependency relationships between labels, thus the label correlation obtained by such methods is subject to certain distortion. To address this issue, this paper assumes that labels can be linearly represented by the same set of bases. The correlation between labels is represented by sharing common bases. Specifically, the paper employs matrix factorization to extract bases that can be used to represent all labels. And then designs a label distribution learning algorithm based on the property of sharing the same set of bases of the ground truth label distribution and predict label distribution. The effectiveness of the algorithm is verified through experimental validation. Generally speaking, the algorithm presented in this paper achieves optimal performance at 73.15% of the cases, with the best average ranking. In the two-tailed t -test, the algorithm in this paper exhibits statistical superiority compared to all comparison algorithms. [ABSTRACT FROM AUTHOR]

Object detection is a basic vision task that accompanies people's daily lives all the time. The development of object detection technology has experienced an evolution from traditional-based algorithms to deep learning-based algorithms, which has made a qualitative leap in both detection accuracy and detection speed. With the advancement of deep learning, object detection techniques are increasingly becoming a part of everyday life, with the YOLO series of algorithms being extensively applied in various industries. In this paper, we initially present the frequently utilized datasets and evaluation criteria for object detection. Subsequently, we delve into the evolution of traditional object detection algorithms, highlighting two-stage and one-stage approaches through illustrative examples of classical methods. We also conduct a comprehensive summary and analysis of the detection results obtained by these methods. In addition, we introduce object detection applications in daily life, as well as the importance and some difficulties of these applications. Finally, we analyze and summarize the difficulties and challenges facing the task of object detection, and we look forward to the future development direction of object detection. • This survey is an extended version of our paper in ICIC2023. • We review the development and challenges of object detection. • We present the application of object detection and make prospects for the future. [ABSTRACT FROM AUTHOR]

This paper investigates fully distributed scaled non-negative edge-consensus problems of networked systems with actuator saturation and unmeasurable internal states. By using the adaptive control method, output-feedback-based low-gain technique, and graph theory, a new adaptive algorithm is designed to obtain scaled edge-consensus conditions, under which the difficulties caused by the constraints on edge states and controllers are overcome. There are three interesting characteristics that any global information of networks is not used in this paper's algorithm and result, including the number of vertexes and edges; the feasible solutions of the scaled edge-consensus conditions exist and can be easily obtained; the convergence rate of the controller is adjustable. Furthermore, the designed algorithm is expanded in the cases without actuator saturation. Finally, three examples are given to verify the theoretical results. [ABSTRACT FROM AUTHOR]

Deep image clustering networks have the capability to categorize unlabeled images, thereby effectively utilizing them. This paper synthesizes recent researches about deep image clustering network and summarizes them within a general framework. Image Preprocessing part transforms collected images into a dataset accepted by the network, then Image Embedding part embeds images from the dataset into vectors which represents image features. After Feature Processing part further reduces the dimensionality or enhances these features, the following Feature Clustering part divides the images into several categories. Cluster Result Processing part treats the clustering outcomes to implement weighted, multi-view, subspace, multimodal, or self-supervised methods. Downstream Applications part employ the clustering results to address various real-world problems. The performance of common baseline clustering methods and several feature extraction architectures are also compared and analyzed. The results indicate that within deep image clustering networks, the choice of clustering algorithm has a relatively minor impact on clustering performance, whereas the selection of the feature extraction network architecture is decisive for clustering metrics. The choice of architecture should be customized based on the characteristics of the dataset. Finally, we provide suggestions for potential research directions in deep image clustering networks. • Analyzes deep image clustering theory, its performance benefits, and limitations. • Summarizes recent deep image clustering improvements with experimental data analysis. • Introduces deep image clustering applications and future improvement prospects. [ABSTRACT FROM AUTHOR]

In recent years, in the field of Multi-Agent Systems (MAS), significant progress has been made in the research of algorithms that combine Reinforcement Learning (RL) with Attention Mechanism (AM). However, there is a lack of comprehensive reviews in this field. Based on this, this paper does the following work. Firstly, it reviews the classical algorithms of RL and AM; Secondly, it systematically introduces the combination of RL and AM; Thirdly, it sorts out their application in the field of single-agent and multi-agent, and pays attention to and looks forward to the challenges and future research prospects in this field; The last part offers a comprehensive analysis of the challenges encountered by research in this field and anticipates future research paths. The research offers a conceptual understanding and theoretical foundation for future applications of RL using AM and facilitates further in-depth study in this field for researchers. [ABSTRACT FROM AUTHOR]

Ordinal regression (OR) methods are designed for a type of classification problems where data labels have natural orders. In practice, data may be corrupted by label noise, which affects the training process thus degrading the generalization performance of OR methods. In OR, data are usually assumed to have latent variables underlying the ordinal labels, and label noise exhibits a special characteristic that it usually causes large latent variable value deviations. However, there are few existing works on OR considering label noise, and the existing works do not utilize the above-mentioned characteristic. Besides, most of the existing OR methods are centralized, which are inapplicable in some realistic distributed applications. In this paper, we utilize the characteristic of label noise in OR to develop a distributed robust support vector ordinal regression method (dRSVOR) under label noise. Specifically, after analyzing the characteristic of label noise in OR, we take the form of SVOR with explicit constraints to achieve robustness to one type of mislabeled samples. Then, we adopt correntropy, an information-theoretic measure, to achieve robustness to the other type of mislabeled samples. Theoretically, we analyze the consensus and convergence of dRSVOR. Experimentally, we conduct experiments on both synthetic data and real OR datasets to illustrate the effectiveness of the proposed method. The results show that the centralized version of dRSVOR outperforms several state-of-the-art OR methods considering label noise in centralized circumstances with label noise, and dRSVOR could approach the performance of the centralized version despite additional constraints in distributed scenarios. [ABSTRACT FROM AUTHOR]

This paper concentrates on the bipartite finite-time consensus (BFTC) problem of nonlinear multi-agent systems (MASs) under intermittent communication via event-triggered impulsive control. First, a finite-time consensus (FTC) protocol with an event-triggered mechanism is developed for the MASs with nonlinear dynamics to save limited resources. Then, an impulsive control scheme is introduced to the designed algorithm to further improve the control performance. Based on the structurally balanced graph theory, both antagonistic and cooperative interactions are discussed in this article. Regarding whether the MASs have encountered intermittent communication, sufficient conditions are provided for achieving consensus of the MASs by using the Lyapunov theory and inductive methods. Moreover, the Zeno behavior is excluded. Finally, numerical simulations are given to validate the effectiveness of the algorithm. The result shows that the designed event-triggered BFTC control protocol with an impulsive strategy has a faster convergence rate than that without an impulsive strategy. [ABSTRACT FROM AUTHOR]

Dynamic area coverage algorithm is often used in multiple unmanned aerial vehicle (multi-UAV) systems to realize searching or monitoring an area of interest (AOI). Improving coverage efficiency is one of the important research issues of dynamic area coverage. In this paper, we propose a distributed dynamic area coverage algorithm based on a reinforcement learning (RL) algorithm to enhance the coverage efficiency in complex application environments. A coverage information fusion based discrete soft actor–critic algorithm (herein FDSAC) is proposed to solve the non-stationary problem of RL algorithm in area coverage task under the communication constraint environment. Considering the equivalence of UAVs in the area coverage tasks, we introduce a multi-UAV cooperative learning method for FDSAC by sharing the strategy and experience of the UAVs to accelerate the learning speed of FDSAC in multi-UAV system. With the FDSAC algorithm, the UAVs can intelligently select the best coverage points and achieve almost optimal coverage point planning for the entire coverage process. In addition, we also provide a coverage point adjustment method based on the distribution of obstacles to enable the proposed algorithm adapt to obstacle environments. The experimental results demonstrate that the proposed algorithm can efficiently complete the area coverage task in complex communication and stochastic control noise environment. The good scalability, environment adaptability, and practicality of our algorithm have been also confirmed in different complex environments of different experimental platforms. [ABSTRACT FROM AUTHOR]

The multi-robot task allocation problem has garnered significant attention in research and development. This paper addresses the multi-robot task allocation problem by introducing a unified model that seamlessly transforms into four popular mathematical models through simple parameter adjustments. We propose an efficient indicator-based multi-objective evolutionary algorithm with a hybrid encoding scheme for task execution order and robot starting point information. The algorithm employs the hypervolume indicator for environmental selection to enhance convergence and the modified crowding distance for archive updates to promote diversity. Additionally, an adaptive archive update mechanism is designed for time efficiency. In experiments comparing our proposed algorithm with 8 state-of-the-art algorithms on 18 randomly generated instances of various sizes, along with 4 real-world problems from the TSPLIB benchmark, our algorithm consistently outperformed the comparison algorithms in five key indicators. These results underscore the effectiveness of our approach in addressing real-world multi-robot task allocation problems. [ABSTRACT FROM AUTHOR]

In the last decade it became increasingly apparent the inability of technical metrics such as accuracy, sustainability, and non-regressiveness to well characterize the behavior of intelligent systems. In fact, they are nowadays requested to meet also ethical requirements such as explainability, fairness, robustness, and privacy increasing our trust in their use in the wild. Of course often technical and ethical metrics are in tension between each other but the final goal is to be able to develop a new generation of more responsible and trustworthy machine learning. In this paper, we focus our attention on machine learning algorithms and associated predictive models, questioning for the first time, from a theoretical perspective, if it is possible to simultaneously guarantee their performance in terms of both technical and ethical metrics towards machine learning algorithms that we can trust. In particular, we will investigate for the first time both theory and practice of deterministic and randomized algorithms and associated predictive models showing the advantages and disadvantages of the different approaches. For this purpose we will leverage the most recent advances coming from the statistical learning theory: Complexity-Based Methods, Distribution Stability, PAC-Bayes, and Differential Privacy. Results will show that it is possible to develop consistent algorithms which generate predictive models with guarantees on multiple trustworthiness metrics. • AI is nowadays requested to optimize both technical and ethical metrics. • Focus on ML algorithms and associated models (both deterministic and randomized). • We prove that it is possible to develop consistent algorithms and models. • We bound generalization in terms of both technical and ethical metrics. • We leverage the most recent advances coming from the statistical learning theory. [ABSTRACT FROM AUTHOR]

The target net method has been the foundation of deep reinforcement learning since Deepmind first proposed it in 2015. Almost all the current popular reinforcement learning algorithms include target net. However, while the slowly updated target network improves the stability of the algorithm, it also reduces the performance of the algorithm. In this paper, the authors design a novel triple-network algorithm(TPN). TPN combines the temporal-difference(TD) algorithm and policy gradient(PG) theorem. Using three networks to estimate the state value(v), action value (q) , and policy(π). These networks have no primary or secondary distinction but are trained synchronously and influence each other. The author found that through this TPN architecture, the convergence and stability of the algorithm can be greatly improved without increasing the amount of calculation. Although it is only a basic framework at present. The calculation process of TPN is simple and easy to implement. Experiments prove that the convergence speed and stability of TPN in discrete cases are better than PPO. • Proposed a new deep reinforcement learning algorithm that does not rely on target networks. • Presented the complete derivation process of the TPN algorithm. • Verified the effectiveness of proposed algorithm through experiments. [ABSTRACT FROM AUTHOR]

Most traditional multi-view spectral clustering algorithms involve two separate steps: solving the spectral embedding matrix and clustering, which may introduce errors during the clustering process. Moreover, in practical applications, the number of available views may increase over time, and the approach of re-fusing all views in each computation would result in elevated computational costs. In this paper, we propose a novel model: One-step Incremental Multi-view Spectral Clustering based on Graph Linkage learning (OIMvSC). OIMvSC only needs to store the consensus spectral embedding matrix and the consensus label matrix of all previous views and combines them with the spectral embedding matrix of the newly available view to solve the fused consensus spectral embedding matrix and consensus label matrix. To further enhance clustering performance, OIMvSC introduces graph linkage learning, which reduces erroneous connections between clusters while preserving correct connections within clusters. A convergent iterative algorithm for solving OIMvSC is proposed. Experimental results demonstrate that OIMvSC exhibits excellent clustering performance. [ABSTRACT FROM AUTHOR]

In the research of complex networks, how to find a set of nodes in the network with the most extensive range in the propagation process, i.e., the Influence Maximization (IM) problem, is one of the focal topics. Existing studies mainly consider the information dissemination process on networks and how to select diffusive nodes efficiently, but little attention has been paid to changes related to the network structure. In reality, networked systems are exposed to uncertain interferences and even destructive sabotages, and cascading failures are one common destruction that can cause networks to collapse even if only a small number of nodes fail. In the case of various complex environmental factors, how to select robust and influential nodes, i.e., the robust influence maximization (RIM) problem, is of great importance in promoting the realistic application of the influence maximization problem. This paper investigates the RIM problem under cascading failures to address the shortcomings in previous studies. Based on existing research, a new performance evaluation metric, R S-cf , is designed to assess the level of robust influence in a numerical form. For solving the seed determination problem, a Memetic algorithm towards the RIM problem under cascading failures, MA-RIM CF , is designed to find nodes with stable information propagation capability guided by R S-cf. Experiments have been conducted on both synthetic and realistic networks to validate the performance of the algorithm. Results indicate that MA-RIM CF can obtain competitive candidates over existing approaches, and seeds with robust and influential abilities are generated to solve diffusion dilemmas. [ABSTRACT FROM AUTHOR]

In numerous domains, there is ample evidence indicating that a significant portion of real-world data exhibits temporal variations, such as medical research and meteorological studies. Particularly in the era of big data, not only does the size of data change dynamically but also its dimensions at an unprecedented pace. Consequently, employing traditional methods to handle such dynamic data becomes highly impractical. To address this limitation, this paper proposes an incremental feature selection algorithm tailored for dynamic feature variations. For scenarios involving an increase in features, the novel algorithm efficiently identifies a target subset with effective features. In order to showcase the efficacy of the proposed algorithm, this paper conduct experiments using four commonly used classifiers and five UCI data sets. The experimental results further validate both the feasibility and efficiency of the new approach. [ABSTRACT FROM AUTHOR]

With the enhancement of people's security awareness, transmitting secret information securely has gradually become a demand for the public. Steganography is a technology of representing secret information within another carrier, aiming at transmitting secret information without causing suspicion. Most of the traditional steganographic algorithms hide secret information by modifying the statistical characteristics, which will leave traces to the carriers. Although these modifications are too tiny to be distinguished by human eyes, they can be detected by steganalysis algorithms. Differently, coverless steganography, also written as steganography without embedding, accomplish the process of information hiding by constructing the relationship between secret information and carriers. Due to no modification to the carriers, all of the steganalysis algorithms are expired. In this paper, more than 90 papers are included to provide a review in coverless steganographic algorithms, covering the major development process of coverless image and video steganographic algorithms. The main contribution of the existing methods is summarized. Besides, the current general issues of capacity, robustness, and security are discussed adequately for both image and video algorithms. Especially, the security of coverless steganography is discussed for the first time from theoretical analysis to actual investigation in this review. [ABSTRACT FROM AUTHOR]

Combinatorial optimization problems (COPs) play an important role in both industrial production and theoretical research. As a classical class of optimization algorithms, evolutionary algorithms have shown good performance in COPs. However, as the size of the problem increases, it becomes difficult for evolutionary algorithms to produce satisfactory solutions within the specified time. In recent years, neural network algorithms have been applied to solve COPs and have shown superior effects. Inspired by this, this paper proposes a new algorithm based on graph structure for solving COPs. Through relaxation, COPs are transformed into their corresponding continuous optimization problems. The output state of our algorithm is converted to a discrete solution by approximation method. Compared with previous algorithms, our algorithm can not only solve COPs with general form such as binary quadratic programming (BQP), but also solve COPs with special form, which greatly expands the application scope of neural network algorithms. In addition, our algorithm can be embedded into other problem frameworks such as solving multi-objective combinatorial optimization problem (MOCOP) through scalarization method. A new penalty function is added to the objective function, which enables our algorithm to overcome the inconsistency between the optimal solution of the original problem and the corresponding continuous problem, so that our algorithm can perform well on various problems. Several numerical simulations are used to illustrate the effectiveness of our algorithm in solving COPs. Experimental results show that our algorithm can effectively solve COPs with general form and has a shorter solving time compared to traditional evolutionary algorithms. [ABSTRACT FROM AUTHOR]

The Wasserstein distance in the optimal transport (OT) model is being widely used in machine learning applications due to its mathematical and geometric properties. Since the entropy regularized OT is a convex problem, and for its large-scale problems, many algorithms have been proposed. Existing methods mainly adopt iterative random projection techniques to approximate the Wasserstein distance in the OT model. In this paper, we employ the entropic domain decomposition algorithm with mini-batch learning to calculate the entropy regularized OT problem and investigate the stability of this algorithm. The algorithm performs on different cells of the domain instead of using the projection technique and it is based on a mini-batch loss function. The corresponding solution in each iteration is derived from the dual problem of entropy regularized OT on each decomposed cell. Once the solutions to the dual problem on all decomposed cells are obtained, the corresponding transport plan in each iteration can be obtained. We show that the transport plan computed by its dual variables is admissible, and we also give the sub-gradient approximation result. In addition, we exploit the statistical concentration result of the subsampled mini-batch Wasserstein distance on decomposed cells. Finally, the numerical experiments validate that the proposed algorithm can save computational time further. [ABSTRACT FROM AUTHOR]

In this paper, a novel population robustness-based switching response framework (PR-SRF) is proposed to develop effective dynamic multi-objective optimization algorithm (DMOA), which integrates different response strategies to comprehensively cope with the dynamic behaviors. In particular, the population robustness is described by the quantification of how severely the environmental changes affect current population, which is timely graded as three levels of weak, strong, and normal to enable the adaptive switch of three different responses of diversity-enhancement, diversity-maintenance, and the knowledge-transfer, respectively. In this way, associations between the adopted responses and the changing environments are successfully established, thereby facilitating more intelligent decision when handling the dynamic behaviors. According to the benchmark evaluation results, the proposed PR-SRF-DMOA yields better comprehensive performance than several other DMOAs with popular response strategies, and it also outperforms another three DMOAs with hybrid responses, which demonstrates the great competitiveness of our algorithm. In addition, ablation study proves that the proposed PR-SRF can sufficiently exploit the merits of different responses, which effectively alleviates the negative knowledge transfer in extremely fluctuating environments, thereby providing valuable references for the development of evolutionary transfer optimization (ETO) algorithms. [ABSTRACT FROM AUTHOR]

An important objective in computer vision is to analyze multiple images and subsequently reconstruct the shape and structure in 3D. Traditional multi-view 3D reconstruction techniques extract and match key features from images with known camera parameters. However, this approach is inefficient and fails to fully exploit the advantages of multi-view information. Advancements in deep learning have revolutionized multi-view 3D reconstruction by enabling end-to-end 3D shape inferencing without the need for sequential feature matching typically found in conventional algorithms. Recent rapid progress in this field necessitates a thorough review of current algorithms and provide insight into method of improving 3D reconstruction performance. This review classifies reconstruction algorithms according to their resultant model, including depth map, voxel, point cloud, mesh, and implicit surface. Additionally, this review encompasses the inclusion of frequently employed network training loss functions for network training, assessment metrics, and the incorporation of 3D datasets. Experimental results are also presented to assess the performance of different algorithms. Finally, the paper concludes with a summary, discussion of challenges, and potential future directions. [ABSTRACT FROM AUTHOR]

Anomaly detection is an important field in data science that has been widely researched and applied, generating many methods. Among these methods, the isolation forest algorithm is outstanding because of its efficiency and effectiveness, especially in regard to large-scale data. Unfortunately, this algorithm has some drawbacks, such as being unable to effectively handle local outliers, possibly leading to normal data masking the detection of outliers due to space partitioning along the coordinate axes, and low utilization of training data. To address these issues, in this paper we propose an improved isolation forest algorithm based on multilayer subspace dividing, named layered isolation forest, which adapts to the different distributions of the dataset by dividing the sample space into subspaces and evaluating the anomaly degree of data in different spatial ranges. This algorithm obtains a more accurate and reasonable anomaly score, avoids the problems of the original algorithm, and improves the performance metrics. According to the experimental results, the proposed method maintains the efficiency of the original algorithm and exhibits the best comprehensive performance compared with similar algorithms on artificial synthetic datasets and real-world datasets from multiple domains. • Isolation forest algorithm demonstrates limitations in isolating local anomalies. • Local anomalies are more easily isolated in subspace. • Combining global and local isolation forests enhances anomaly identification. • Layered Isolation Forest (LIF) overcomes traditional isolation forest limitations. [ABSTRACT FROM AUTHOR]

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the Q -values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon (n), and our empirical experiments show that a poorly chosen static value for n can in many cases lead to worse performance than single-step DQN. Inspired by the success of n -step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call 'Elastic Step DQN' (ES-DQN) to alleviate overestimation bias in DQNs. ES-DQN dynamically varies the step size horizon in multi-step updates based on the similarity between states visited. Our empirical evaluation shows that ES-DQN out-performs n -step with fixed n updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias. [ABSTRACT FROM AUTHOR]

In this paper, a novel learnable population filter is proposed to solve the dynamic multi-objective problems (DMOPs), whose main idea is to pick out potential valuable individuals in the changing situation via the trained filter to facilitate the population convergence in a new environment. In particular, by selecting training samples from three different optional pools, rich information in the previous searching experiences can be sufficiently used to cope with the dynamic behaviors in DMOPs, thereby handling the imperfect history data availability caused by the environmental uncertainty to some extents. Then, the screened population in the new environment, which is regarded to have reliable quality that benefits accelerating the evolution, is used to initialize the static optimizer. In addition, partial random solutions are employed as supplement to the initial population to further enhance the diversity. It is shown from the benchmark evaluations that the proposed method outperforms some other popular algorithms on both convergence and diversity by seamlessly combining the mainstream strategies in dealing with the dynamic behaviors, which is a competitive method that conforms to the novel evolutionary transfer optimization (ETO) trend. Moreover, effectiveness of the core components is validated through extensive ablation studies, which brings a new sight in developing the learnable DMOP solvers. [ABSTRACT FROM AUTHOR]

In this paper, a distributed economic dispatch algorithm based on event-triggered and fixed-time consensus is proposed for optimal dispatching among generators in smart grid. Aiming to quickly achieve economic dispatch in distributed smart grid and significantly reduce the frequency of communication between generators, the algorithm can address the supply–demand balance between the generation side and the consumption side, as well as the capacity constraints of the generators. Different from the relevant work, the proposed incremental cost can converge in fixed-time and the settling time is independent of the initial state of the generators. It will provide more flexibility for the system. Meanwhile, a fully distributed event-triggered strategy is adopted, which can reduce the interactive information between generators and avoid the waste of communication resources. Furthermore, the proposed algorithm is robust to the time-varying communication network topology. Finally, several examples are presented that validate the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]