Search

Adversarial training has been considered to be one of the most effective strategies to defend against adversarial attacks. Most existing adversarial training methods have shown a trade-off between training cost and robustness. This paper explores a new optimization direction from training data to reduce the computational cost of adversarial training without scarifying robustness. First, we show that some adversarial examples are less important, meaning that removing them does not hurt the robustness. Second, we propose a method to identify insignificant adversarial examples at a minimal cost. Third, we demonstrate that our approach can be integrated with other adversarial training frameworks with few modifications. The experimental results show that combined with previous works, our approach not only reduces about 20% of computational cost on the CIFAR10 and CIFAR100 datasets but also improves about 1.5% natural accuracy. With less computational cost, it achieves 58.22%, 30.68%, and 41.92% robust accuracy on CIFAR10, CIFAR100, and ImageNet datasets respectively, which are higher than those of the original methods. • Analyze the effectiveness of adversarial examples for adversarial training. • Design a method to remove inefficient adversarial examples but maintain the accuracy. • Design an algorithm to detect inefficient adversarial examples without extra cost. • Conduct experiments to demonstrate the compatibility with other frameworks. [ABSTRACT FROM AUTHOR]

Cluster analysis is one of the most important research issues in data mining and machine learning. To date, numerous clustering algorithms have been proposed to tackle the fixed-length vector data. In many real applications, we need to detect clusters from a set of discrete sequences in which each sequence is an ordered list of items. Due to the sequential and discrete nature, the discrete sequence clustering problem is more challenging and most of existing vector data clustering algorithms cannot be directly employed. In this paper, we present a stochastic algorithm for clustering discrete sequences. Our method first quickly generates a set of random partitions over the sequential data set and then merges these random clustering results via weighted graph construction and partition. We perform extensive empirical comparisons on real data sets to show that our method is comparable to those state-of-the-art clustering algorithms with respect to both accuracy and efficiency. • A randomized algorithm is proposed to solve the discrete sequence clustering issue. • Our algorithm provides high success rate for sequence linkage in clusters. • Experimental results demonstrate the feasibility and effectiveness of our method. [ABSTRACT FROM AUTHOR]

This paper investigates algorithms for matrix factorization when some or many components are missing, a problem that arises frequently in computer vision and pattern recognition. We demonstrate that the Jacobian used in the damped Wiberg (DW) method is exactly the same as that of Kaufman's simplified variable projection (VP) algorithm. Our analysis provides a novel perspective on the efficiency of VP algorithms by improving the strong convexity of the approximate function. To enhance numerical stability, we set a lower bound on the damping parameter instead of adding a null space like the DW algorithm. Another challenge of low-rank matrix decomposition with missing data is the existence of many sharp local minima, which are often distributed in narrow valleys of the landscape of objection functions. Falling into such minima tends to result in poor reconstruction results. To address this issue, we design a non-monotonic VP algorithm, which can facilitate the algorithm to escape from sharp minima and converge to flatter minima. Numerical experiments confirm the effectiveness and efficiency of the proposed nonmonotone VP algorithm. • We provide a new perspective to examine the superiority of the VP algorithm from a convexity point of view. • We propose a nonmonotone VP algorithm, which can facilitate the algorithm to escape from sharp minima. • We conduct extensive experiments to validate the effectiveness of the proposed nonmonotone algorithm. [ABSTRACT FROM AUTHOR]

Wide-angle video frames typically contain more information, but they also exhibit distortion that degrades the visual quality, especially at the edges. To eliminate this distortion from videos, we propose a self-supervised iterative optimization method in this paper. Specifically, we construct a motion parameter estimation model utilizing two consecutive distorted frames, where motion parameters comprise affine transform and distortion parameters. We apply the Gauss–Newton algorithm to minimize the sum-of-squares error between frames and update parameters. Treating inter-frame motion as undistort-affine-distort transformations, frame alignment is achieved by continuously adjusting transform parameters. Ultimately, frames are corrected using the converged parameters. We generated a synthetic dataset with various distortion parameters for evaluation. Experiments demonstrate superior performance versus state-of-the-art methods on synthetic and real wide-angle videos. Our algorithm also achieves higher parameter estimation accuracy without sacrificing efficiency. • proposes a algorithm for video distortion correction based on iterative optimization. • the distort transform and undistort transform are derived separately and the range of parameters and the experimental error are given. • Based on this, we transform the distortion correction into a parameter estimation problem in image alignment. • optimize the parameter estimation problem using Gauss-Newton algorithm. • Experimental results show that the proposed algorithms outperform some state-of-art algorithms in accuracy, without reducing robustness and efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we investigate L 0 -regularized logistic regression models, and design two fast and efficient algorithms for high-dimensional correlated data and massive data, respectively. Our first algorithm, the Variable Sorted Active Set (VSAS) algorithm, is based on the local quadratic approximation of the KKT conditions for L 0 -penalized maximum log-likelihood function in high-dimensional correlated data. We establish an L ∞ error upper bound for the estimator obtained by the VSAS algorithm and prove its optimal convergence rate. Moreover, when the target signal exceeds the detectable level, the estimator obtained by the VSAS algorithm can achieve the oracle estimator with high probability. Our second algorithm, Communication Effective Variable Sorted Active Set (CEVSAS), aims to solve high-dimensional and large-sample L 0 -regularized logistic regression models by reduce computational and communication costs, while maintaining estimation efficiency. Finally, simulations and real data demonstrate the effectiveness of our proposed VSAS and CEVSAS algorithms. • We develop a fast VSAS algorithm for L 0 regularized logistic regression involving high-dimensional correlated data. • An L ∞ upper bound is established on the estimation error of the solution sequence generated by VSAS algorithm. • The CEVSAS altorithm is proposed to reduce computational costs and ensure higher estimation efficiency. • We propose two adaptive versions of the VSAS and CEVSAS algorithms for selecting the optimal tuning parameters. [ABSTRACT FROM AUTHOR]

• We propose an improved differential evolution (DE) algorithm by composite binary crossover and Pareto refining operations to the DE. • DE solves the quantifying fraudulent transactions problem by minimizing the selected feature cost while maximizing the quantification ability. • We show that the proposed DE outperforms four other variants of DE and five state-of-the-art EAs on various credit datasets. • The results are also statically validated through the Friedman test and Wilcoxon post-hoc analysis. [Display omitted] Identification of fraudulent credit card transactions is a complex problem mainly due to the following factors: 1) The relative behavior of customers and fraudsters may alter over time. 2) The ratio of legitimate to fraudulent transactions is highly imbalanced, and 3) Investigators examine a small segment of transactions in a reasonable time frame. Researchers have proposed various algorithms to identify potential fraud in a new incoming transaction. However, these approaches require significant human investigator effort and are sometimes misleading. To address this issue, this paper proposes an improved multiobjective differential evolution (DE) algorithm to estimate the distribution of fraudulent transactions in a set of new incoming transactions, referred to as quantifying fraudulent transactions. Our paper has three major novelties. First, we present the problem formulation of cost-based feature selection with maximum quantification ability. Second, we improve the DE by applying effective trial vector generation algorithms to the random control parameter settings to exploit the advantage of individual DE variants. Third, we develop the maximum-relevancy-minimum-redundancy-based Pareto refining operator to enhance the self-learning ability of individuals in Pareto solutions. We compare our approach against four other modifications of DE and five state-of-the-art evolutionary algorithms on real-time credit datasets in streaming and non-streaming frameworks using hyper-volume, two-set coverage, and spread performance metrics. [ABSTRACT FROM AUTHOR]

• The first attempt to design a GEP-based codematrix generation algorithm. The individual structure of GEP allows the codematrix to be optimized in a much faster and more effective way. • The feature selection scheme is combined with the design of codematrix, to accomplish the feature selection and codematrix optimization simultaneously. • A greedy strategy-based adaptive optimization method to further improve the individual's quality to deal with hard classes by appending proper columns to the codematrix. The multi-class classification task is one of the most common tasks in machine learning. As a typical solution based on a partitioning strategy, Error-Correcting Output Codes (ECOC) can transform a multi-class classification problem into multiple binary classification problems. The key of ECOC is to construct an effective codematrix to represent a set of class decomposition schemes, which transforms a multiclass problem into a group of binary class problems. Consequently, the design of a fast and effective ECOC codematrix generation method is of great research significance and value for solving multi-class classification problems. In ECOC algorithms, the design of codematrix is treated as a combination problem between different code columns, in which the evolutionary algorithm shows a great advantage. Based on this consideration, the Gene Expression Programming (GEP) is applied to search for the codematrix with high performance because its expressive tree structure makes it well represent codematrcies for subsequent optimization operations. This paper proposes an adaptive ECOC algorithm based on Gene Expression Programming (GEP) and similarity measurement matrix, named GEP-ECOC. In our GEP, each individual represents a set of columns to form a random ECOC codematrix, which is optimized in the evolutionary process. Meanwhile, the crossover and mutation operations are modified to include a legality checking process to ensure that the generated codematrix satisfies the ECOC constraints. The GEP-based ECOC codematrix generation algorithm can quickly produce a codematrix with better performance, which ensures the efficiency of the algorithm to a certain extent. In addition, an adaptive algorithm based on a similarity measurement matrix is proposed to add new columns to the current codematrix, aiming to better handle hard classes. Our algorithm is compared with other algorithms on various data sets, and the experimental results confirm that our GEP-ECOC can balance the efficiency and performance of the algorithm and achieve higher performance. [ABSTRACT FROM AUTHOR]

Class imbalance problem commonly exists in multi-label classification (MLC) tasks. It has non-negligible impacts on the classifier performance and has drawn extensive attention in recent years. Borderline oversampling has been widely used in single-label learning as a competitive technique in dealing with class imbalance. Nevertheless, the borderline samples in multi-label data sets (MLDs) have not been studied. Hence, this paper deeply discussed the borderline samples in MLDs and found they have different neighboring relationships with class borders, which makes their roles different in the classifier training. For that, they are divided into two types named the self-borderline samples and the cross-borderline samples. Further, a novel MLDs resampling approach called Multi-Label Borderline Oversampling Technique (MLBOTE) is proposed for multi-label imbalanced learning. MLBOTE identifies three types of seed samples, including interior, self-borderline, and cross-borderline samples, and different oversampling mechanisms are designed for them, respectively. Meanwhile, it regards not only the minority classes but also the classes suffering from one-vs-rest imbalance as those in need of oversampling. Experiments on eight data sets with nine MLC algorithms and three base classifiers are carried out to compare MLBOTE with some state-of-art MLDs resampling techniques. The results show MLBOTE outperforms other methods in various scenarios. • A new borderline oversampling technique for multi-label imbalanced learning. • Defining self-borderline and cross-borderline samples in multi-label data sets. • Handling one-vs-rest imbalance in multi-label imbalanced learning. • Performing competitively in the experiments with C4.5, RBF kernel SVM and linear SVM. [ABSTRACT FROM AUTHOR]

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Encouraging results are also observed when applying PEEL to compress the semantic segmentation model. Our code is available at https://github.com/cardwing/Codes-for-PEEL. • Previous methods typically perform iterative pruning and are tedious and expensive. • A simple yet effective method is proposed to quickly produce a slim model with negligible cost. • Our method constructs a predefined backbone and conducts resource reallocation on it. • Resource reallocation shifts parameters from less informative layers to more important ones in one round. • Extensive experiments on ImageNet with various models validate the effectiveness. [ABSTRACT FROM AUTHOR]

• The establishment of a new mathematical model to realize dense light field reconstruction. • The innovative application of optical flow method. • Simple algorithm structure and relatively easy to implement. • Algorithm strong adaptability, good dense reconstruction was completed. Dense light field sampling is an important basis for refocusing, depth estimation and 3-D imaging. It is difficult to obtain high resolution dense light field with a large-scale camera array and expensive equipment. At the same time, the current storage devices and transmission bandwidth also limit this technology's post-processing and application. In order to effectively reconstruct the angle domain of the light field based on the sparse light field data, this paper analyzes the correlation and constraint relationship between the optical flow field and the motion field of multi-view images in the same scene, extends the traditional optical flow constraint equation of two-dimensional imaging to the optical flow constraint equation of four-dimensional light field, and establishes an effective mathematical model. The coordinate position of the original pixel in the new angle image is determined by coordinate search, and its intensity is reconstructed. The experimental results of multi-scene dense reconstruction show that the proposed method can reconstruct the texture, shadow and color information in the light field of a long-baseline scene with high quality. The quantitative evaluation results show that the algorithm can be applied to dense light field reconstruction of complex scenes. The algorithm in this paper is only suitable for the case of optical flow constraint in a linear light field, and the follow-up research will focus on the case of nonlinear optical flow constraint. [ABSTRACT FROM AUTHOR]

• For multi-view input, we propose the global-aware fusion to adapt the global state when fusing features from different views. • Our algorithm includes a powerful but relatively lightweight network, novel loss function, and specified training strategy. • The reconstructor exceeds existing SOTA methods while the amount of parameters is far less than the same type of algorithm. • An extra view-reduction method based on maximizing diversity for our model can be used to tradeoff cost and performance. Deep learning technology has made great progress in multi-view 3D reconstruction tasks. At present, the mainstream solutions adopt different ways to fusion the features from several views. Among them, attention-based aggregation function performs relatively well and stably, however, it still has an obvious shortcoming the strong independence of each view during predicting the weights for merging leads to a lack of adaption of the global state. In this paper, we propose a global-aware attention-based fusion approach that builds the correlation between each branch and the global feature to provide a comprehensive foundation for weights inference. On the basis of this, we design a complete reconstruction algorithm. Experiments on ShapeNet verify that our method outperforms existing SOTA methods. Furthermore, we propose a view-reduction method based on maximizing diversity and discuss the cost-performance tradeoff of our model to achieve a better performance when facing a heavy input amount and limited computational cost. [ABSTRACT FROM AUTHOR]

• A novel rank-based approach is proposed for regression tasks; • The method allows to combine predictions from a base regressor and neighborhood information; • Re-ranking strategies and weighted functions are used for better exploiting neighborhood information; • Several experiments considering various datasets, base regressors and re-ranking methods; • Significant decrease of prediction errors considering traditional and state-of-the-art regressors. Several approaches based on regression have been developed in the past few years with the goal of improving prediction results, including the use of ranking strategies. Re-ranking has been exploited and successfully employed in several applications, improving rankings by encoding the manifold structure and redefining distances among elements from a dataset. Despite the promising results observed, re-ranking has not been evaluated in regressions tasks. This paper proposes a novel, generic, and customizable framework entitled Regression by Re-ranking (RbR) , which explores the ability of re-ranking algorithms in determining relevant rankings of objects in prediction tasks. The framework relies on the integration of a base regressor, unsupervised re-ranking learning techniques, and predictions associated with nearest neighbours weighted according to their ranking positions. The RbR framework was evaluated under a rigorous experimental protocol and presented significant results in improving the prediction when compared to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• Providing the condition under which an AHC does not effectively extract clusters. • Introducing the entanglement to measure the dendrogram quality. • Identifying the root cause of a density bias of traditional AHC algorithms. • Improving AHC performance with a data-dependent kernel. Agglomerative hierarchical clustering (AHC) is one of the popular clustering approaches. AHC generates a dendrogram that provides richer information and insights from a dataset than partitioning clustering. However, a major problem with existing distance-based AHC methods is: it fails to effectively identify adjacent clusters with varied densities, regardless of the cluster extraction methods applied to the resultant dendrogram. This paper aims to reveal the root cause of this issue and provides a solution by using a data-dependent kernel. We analyse the condition under which existing AHC methods fail to effectively extract clusters, and give the reason why the data-dependent kernel is an effective remedy. This leads to a new approach to kernerlise existing hierarchical clustering algorithms including the traditional AHC algorithms, HDBSCAN, GDL, PHA and HC-OT. Our extensive empirical evaluation shows that the recently introduced Isolation Kernel produces a higher quality or purer dendrogram than distance, Gaussian Kernel and adaptive Gaussian Kernel in all the above mentioned AHC algorithms. [ABSTRACT FROM AUTHOR]

• Possibilistic c-means (PCM) clustering by incorporating the sparsity idea with feature weights is further studied. • We propose two approaches that make the PCM clustering with the least absolute shrinkage and selection operator (Lasso), called S-PCM1 and S-PCM2. • Synthetic and real data sets are used to compare the proposed S-PCM1 and S-PCM2 with some existing algorithms. • Experimental results and comparisons demonstrate the good effectiveness and usefulness of the proposed S-PCM1 and S-PCM2 algorithms. Krishnapuram and Keller first proposed possibilistic c-means (PCM) clustering in 1993. Afterward, PCM was widely studied with various extensions. The PCM algorithm and its extensions always treat feature components under equal importance, but, in real applications, different features may better have different weights. Recently, Yang and Benjamin in 2021 proposed a feature-weighted PCM clustering with feature reduction. Although Yang and Benjamin (2021) can reduce feature dimensions, it still encounters the curse of dimensionality for high dimensional data. One possible way to address this problem is to conduct a sparse clustering technique. In this paper, we further study the PCM clustering by incorporating the idea of sparsity with different feature weights. We propose two approaches that use the PCM clustering with the least absolute shrinkage and selection operator (Lasso). The first one is the sparse PCM subject to a Lasso constraint of feature weights, called S-PCM1. The second is the sparse PCM by adding a Lasso penalty term of feature weights in the objective function, called S-PCM2. We show that S-PCM1 and S-PCM2 are theoretically the same, and both can induce sparsity in features, but they use different procedures in algorithms. Synthetic and real data sets are used to compare S-PCM1 and S-PCM2 with some existing sparsity clustering algorithms. Experimental results and comparisons demonstrate the effectiveness and usefulness of the proposed S-PCM1 and S-PCM2 clustering algorithms. [ABSTRACT FROM AUTHOR]

• A novel end-to-end network architecture is proposed for pose transfer. • The proposed model utilizes attention links at different resolution levels to retain coarse and fine appearance attributes in the generated images. • The proposed method outperforms the existing algorithms in qualitative and quantitative benchmarks. • The generality of the proposed method is shown by extending the model to different application scenarios, such as virtual try-on and style manipulation. Pose transfer refers to the probabilistic image generation of a person with a previously unseen novel pose from another image of that person having a different pose. Due to potential academic and commercial applications, pose transfer has been extensively studied in recent years. Among the various approaches to the problem, attention guided progressive generation is shown to produce state-of-the-art results in most cases. This paper presents an improved network architecture for pose transfer by introducing attention links at every resolution level of the encoder and decoder. By utilizing such dense multi-scale attention guided approach, we are able to achieve significant improvement over the existing methods both visually and analytically. We conclude our findings with extensive qualitative and quantitative comparisons against several existing methods on the DeepFashion dataset. We also show the generality of the proposed network architecture by extending it to multiple application domains, such as semantic reconstruction, virtual try-on and style manipulation. 1 1 For reproducibility, the code implementation and the pre-trained models are available at https://github.com/prasunroy/pose-transfer. [ABSTRACT FROM AUTHOR]

• We propose a strategy to adaptively estimate minimal structure sizes by using previously obtained minimal structure sizes. • We propose another strategy to generate effective initial model hypotheses by jointly performing feature matching and proximity sampling. • We present an efficient sampling algorithm based on the above two proposed strategies. • Extensive experimental results demonstrate the effectiveness of the proposed sampling algorithm. Greedy search-based guided sampling is a promising research field in model fitting to data with multiple structures in the presence of a large number of outliers. However, these greedy search-based guided sampling algorithms are sensitive to the fixed minimal (acceptable) structure size and the initial model hypothesis: when the fixed minimal structure size is too small, data subsets sampled by these algorithms are not representative. In contrast, when it is too large, data subsets might be contaminated by outliers. Furthermore, these algorithms may fail to obtain an accurate model hypothesis, if the initial model hypothesis is far from the true model. In this paper, we address the above-mentioned two issues by proposing two greedy search-based strategies: one aims to adaptively estimate minimal structure sizes and the other aims to generate effective initial model hypotheses. Specifically, on one hand, to avoid using the fixed minimal structure size, a strategy is proposed to adaptively estimate minimal structure sizes by using previously obtained ones. On the other hand, to reduce the impact of outliers, a strategy is proposed to filter out outliers to obtain a reduced data subset by using a feature matching algorithm. Then, this strategy generates promising initial model hypotheses by using a proximity sampling on the reduced data subset. Finally, an efficient sampling algorithm based on the two proposed greedy search-based strategies is applied to three vision tasks, i.e., fundamental matrix estimation, homography plane detection and 3D motion segmentation. Extensive experimental results demonstrate the effectiveness of the proposed sampling algorithm. [ABSTRACT FROM AUTHOR]

• Human activities recognition (HAR) and human intent recognition (HIR) are important for medical diagnosis and human-robot interaction. HAR and HIR usually rely on the signals of some wearable sensors, such as inertial measurement unit (IMU), but these signals may be user-dependent, which degrades the performance of the recognition algorithm on new subjects. Traditional supervised learning methods require labeling signals and training specific classifiers for each new subject, which is burdensome. To deal with this problem, this paper proposes a novel non-adversarial cross-subject adaptation method called Gaussian-guided feature alignment (GFA). The proposed GFA metric quantifies the discrepancy between the labeled features of source subjects and the unlabeled features of target subjects so that minimizing the GFA metric leads to the alignment of the source and target features. The GFA metric is estimated by calculating the divergence between the feature distribution and Gaussian distribution, as well as the mean squared error of the mean and variance between source and target features. This paper analytically proves the effect of the GFA metric and validates its performance using three public human activity datasets. Experimental results show that the proposed GFA achieves 1% higher target classification accuracy and 0.5% lower variance than state-of-the-art methods in case of cross-subject validation. These results indicate that the proposed GFA is feasible for improving the generalization of the HAR and HIR. • The key contributions of the present paper include: The cross-subject validation problems are resolved by proposing a novel Gaussian-guided Feature Alignment (GFA) metric to extract the features of the two domains in a Gaussian distribution space without projecting the features to another vector space, e.g., Hilbert space. A better alignment is achieved by adjusting the mean and variance of the Gaussian distribution based on the mean and variance of the corresponding feature distribution and sorting the data sampled from the feature and Gaussian distributions. The effect of the GFA metric on aligning the target distribution with the source distribution and decreasing the upper bound of the target error is analytically proved. Moreover, the feature extractor is optimized to decrease the variance of the target classification accuracy on the training set to further improve the performance of the cross-subject adaptation. The experimental results show that the proposed method outperforms state-of-the-art methods in the case of the cross-subject validation on one motion-sensor-based HIR dataset, one motionsensor- based HAR dataset, and one skeleton-based HAR dataset. Human activities recognition (HAR) and human intent recognition (HIR) are important for medical diagnosis and human-robot interaction. HAR and HIR usually rely on the signals of some wearable sensors, such as inertial measurement unit (IMU), but these signals may be user-dependent, which degrades the performance of the recognition algorithm on new subjects. Traditional supervised learning methods require labeling signals and training specific classifiers for each new subject, which is burdensome. To deal with this problem, this paper proposes a novel non-adversarial cross-subject adaptation method called Gaussian-guided feature alignment (GFA). The proposed GFA metric quantifies the discrepancy between the labeled features of source subjects and the unlabeled features of target subjects so that minimizing the GFA metric leads to the alignment of the source and target features. The GFA metric is estimated by calculating the divergence between the feature distribution and Gaussian distribution, as well as the mean squared error of the mean and variance between source and target features. This paper analytically proves the effect of the GFA metric and validates its performance using three public human activity datasets. Experimental results show that the proposed GFA achieves 1% higher target classification accuracy and 0.5% lower variance than state-of-the-art methods in case of cross-subject validation. These results indicate that the proposed GFA is feasible for improving the generalization of the HAR and HIR. [ABSTRACT FROM AUTHOR]

• A directed convolutional neural network model is proposed • Partial dilation-Global erosion generates resistance against information loss • Several recent approaches for the segmentation of graphemes/characters are discussed • The over-traces issue is addressed for the first time in the literature • MMOs can provide real solutions if a convenient strategy is adopted Due to the nature of Arabic handwriting, segmenting words into characters/graphemes is the most difficult and critical task of the recognition system. The present paper proposes an approach to segment handwritten Arabic words into graphemes based on a directed Convolutional Neural Network (CNN) and Mathematical Morphology Operations (MMO). Arabic script is cursive, which means that almost all graphemes are connected via horizontal links; therefore, a technique to remove links will facilitate the segmentation of graphemes. In general, an MMO such as erosion seems suitable for getting the job done, but since Arabic handwriting is difficult, MMOs cause information loss and suffer from many issues such as diacritics and over-traces, which lead to over/under/bad segmentations. To overcome limitations, the present paper addresses these issues in the following order: the over-traces issue is addressed for the first time in the literature; a robust algorithm for diacritics extraction is provided; and finally, the main segmentation algorithm adopts a strategy based on a Partial Dilation (PD)-Global Erosion (GE) technique to combat the information loss issue. The PD phase amplifies important regions, while GE eliminates links between graphemes. The complementarity between PD and GE facilitates the extraction of graphemes and creates resistance against information loss. To properly tackle these difficult problems, this article exploits the robustness of CNNs, so a new directed CNN model is suggested. The idea is to draw the model's attention to certain targeted features, which are selected according to the nature of the problem addressed. The proposed directed CNN is used in all phases of the segmentation process. The experimental results are very encouraging and show that the proposed directed CNN model outperformed basic CNN in many experiments. The results also reveal that the followed strategy improved the ability of MMOs to perform segmentation and to compete with other approaches in this research area. [ABSTRACT FROM AUTHOR]

• Designing an ensemble hierarchical clustering framework based on cluster consensus selection approach. • Aggregation of primary clusters using the clusters clustering technique as a consensus function. • Development of NMI robustness measure for calculating merit at cluster and partition levels. • Defining an innovative criterion for calculating similarity based on merit scores and clusters size. • Perform extensive experiments to demonstrate the efficacy of the proposed clustering algorithm and give credence to our idea. Ensemble clustering has emerged as a combination of several basic clustering algorithms to achieve high quality final clustering. However, this technique is challenging due to the complexities in primary clusters such as overlapping, vagueness, instability and uncertainty. Typically, ensemble clustering uses all the primary clusters into partitions for consensus, where the merits of a cluster or a partition can be considered to improve the quality of the consensus. In general, the robustness of a partition may be poorly measured, while having some high-quality clusters. Inspired by the evaluation of cluster and partition, this paper proposes an ensemble hierarchical clustering algorithm based on the cluster consensus selection approach. Here, the selection of a subset of primary clusters from partitions based on their merit level is emphasized. Merit level is defined using the development of Normalized Mutual Information measure. Clusters of basic clustering algorithms that satisfy the predefined threshold of this measure are selected to participate in the final consensus. In addition, the consensus of the selected primary clusters to create the final clusters is performed based on the clusters clustering technique. In this technique, the selected primary clusters are re-clustered to create hyper-clusters. Finally, the final clusters are formed by assigning instances to hyper-clusters with the highest similarity. Here, an innovative criterion based on merit and cluster size for defining similarity is presented. The performance of the proposed algorithm has been proven by extensive experiments on real-world datasets from the UCI repository compared to state-of-the-art algorithms such as CPDM, ENMI, IDEA, CFTLC and SSCEN. [ABSTRACT FROM AUTHOR]

• A novel framework of joint optimization of scoring and thresholding models is proposed for online multi-label classification. • Based on this framework, two types of algorithms are proposed, one of which adaptively learns an instance-based thresholding model and the other of which uses a fixed thresholding model. • For each type of algorithms, a first-order method and a second-order one are provided for updating the online multi-label classifier. • All the proposed algorithms can be proved to achieve a sub-linear regret. Using Mercer kernels, two first-order algorithms can be easily extended to handle nonlinear multi-label prediction tasks. • Experiments show the advantage of the proposed algorithms, in terms of various multi-label performance metrics. Existing online multi-label classification works cannot well handle the online label thresholding problem and lack regret analysis for their online algorithms. This paper proposes a novel framework of joint optimization of scoring and thresholding models for online multi-label classification, with the aim to overcome the above drawbacks. The key feature of our framework is that both scoring and thresholding models are included as important components of the online multi-label classifier and are incorporated into one online optimization problem. Based on this framework, we present two adaptive label thresholding algorithms and two fixed thresholding algorithms. For each type of algorithms, a first-order method and a second-order one are provided for updating the online multi-label classifier. Both methods enjoy a closed-form update. Our proposed algorithms are proved to achieve a sub-linear regret. Using Mercer kernels, two first-order algorithms can be extended to handle nonlinear multi-label prediction tasks. Experiments show the advantage of the adaptive and the fixed thresholding algorithms, in terms of various multi-label performance metrics. [ABSTRACT FROM AUTHOR]

1 A new ECOC encoding algorithm is proposed, which separates classes into two groups by maximizing the ratio of the inter-group distance to intra-group distance. 2 A novel Soft-Coded (SC) ECOC algorithm is proposed by softening the codewords of the ECOC codematrix by considering the subordination degree of each class to both groups. 3 A new measure coverage is designed to evaluate the subordination degrees of different classes to the positive and the negative groups, and the values are set as the elements of the codematrix. 4 A self-adaptive strategy is designed to adjust codewords, making them better fit the associated learners. Error-Correcting Output Codes (ECOC) algorithms enable multiclass classification by reassigning multiple classes to the positive/negative group with the class reassignment schemes being recorded as binary/ternary hard-coded (HC) codematrices. Different classes tend to get diverse subordination degrees to the positive/negative group, providing clues to correct potential errors. However, the HC codematrices are unable to provide the information in the subordination degrees. In this paper, a Soft-Coded ECOC (SC-ECOC) scheme, namely, the Sequential Forward Floating Selection algorithm, is proposed by filling codematrices with real values instead of hard codes to improve classification performance. This algorithm divides multiple classes into two groups by maximizing the ratio of inter-group distance to intra-group distance. Then a new measure coverage is designed to evaluate the subordination degrees of different classes to both groups, which are set as the elements to form a codematrix. Furthermore, a self-adaptive strategy adjusts the value of each element to fit learners better. Experiments are carried out to verify the performance of our algorithm on various data sets, and results confirm that our algorithm can achieve more balanced results compared with the traditional HC ECOC algorithms. Besides, the values of soft codes correlate with the difficulty level of various classes to improve the multiclass classification ability. [ABSTRACT FROM AUTHOR]

• A novel density peaks clustering algorithm called BC-DPC is proposed based on mutual nearest neighbor and connectivity, which can quickly find the correct cluster centers and obtain satisfactory clustering results. • A new definition of density is proposed, which can elimilate the density difference among different clusters and accurately estimate the density of the data points. • This algorithm fully considers the density change and connectivity between data points when calculating the relative distance. This makes the domino effect can be avoided in the clustering process. • The algorithm proposes a novel fast search strategy, which is used to calculate relative distance of data points. This strategy makes the efficiency of the proposed algorithm greatly improved. • The performance of the proposed algorithm is compared to DPC and five improved DPC in synthetic, UCI, and image datasets. • The experimental results show the effectiveness and the efficiency of the proposed algorithm. Density peaks clustering (DPC) algorithm regards the density peaks as the potential cluster centers, and assigns the non-center point into the cluster of its nearest higher-density neighbor. Although DPC can discover clusters with arbitrary shapes, it has some limitations. On the one hand, the density measure of DPC fails to eliminate the density difference among different clusters, which affects the accuracy of recognizing cluster center. On the other hand, the nearest higher-density point is determined without considering connectivity, which leads to continuously clustering errors. Therefore, DPC fails to obtain satisfactory clustering results on datasets with great density difference among clusters. In order to eliminate these limitations, a novel DPC algorithm based on balance density and connectivity (BC-DPC) is proposed. First, the balance density is proposed to eliminate the density difference among different clusters to accurately recognize cluster centers. Second, the connectivity between a data point and its nearest higher-density point is guaranteed by mutual nearest neighbor relationship to avoid continuously clustering errors. Finally, a fast search strategy is proposed to find the nearest higher-density point. The experimental results on synthetic, UCI, and image datasets demonstrate the efficiency and effectiveness of the proposed algorithm in this paper. [ABSTRACT FROM AUTHOR]

• In this paper, an object-based clustering validity index with densities referred to as OCVD is proposed. This index uses densities of clusters to capture the exclusive contribution of each data object in both separation and compactness of clusters. • OCVD is superior to many existing clustering validity indices that capture the properties of clusters by using representative statistics such as mean, variance, diameter, etc. The reason is that those indices might not perform well in capturing the whole characteristics of clusters with arbitrary shapes but OCVD is well capable of doing so. • Although there are some existing density-based validity indices, studies show that they have problems such as poor performance on clusters with arbitrary shapes which are not necessarily perfectly separated and poor performance due to relying only on some representative data objects in clusters. Clustering validity indices are typically used as tools to find the correct number of clusters in a data set and/or to evaluate the quality of the clusters formed by clustering algorithms. Clustering validity indices measure separation and compactness of clusters. Typically, when applying a clustering algorithm, the input includes the number of clusters. After applying the algorithm with several different numbers of clusters, we determine the number of clusters to be the one with the best validity index. There are two types of clustering validity indices: external indices that are supervised, and internal indices that are unsupervised. The focus of this paper is on internal validity indices. Some existing internal validity indices capture the properties of the clusters by using representative statistics such as mean, variance, diameter, etc., however, these do not perform well when clusters have arbitrary shapes. One approach to overcome this issue is to use the density of the data objects in each cluster. That provides the advantage of capturing the full characteristics of the cluster which is most beneficial when there are clusters with arbitrary shapes. In the literature, a few density-based clustering validity indices have been proposed. However, some of them show poor performance when the clusters are not perfectly separated. Some others perform poorly because they use only representative objects from each cluster instead of all objects. The contribution of this paper is an internal validity index named the object-based clustering validity index with densities (OCVD). OCVD is a single number that averages the density-based contribution of individual data objects to both separation and compactness of clusters. The methodology behind calculating the density-based contributions of the objects is kernel density estimation. We show through several experiments that OCVD performs well in detecting the correct number of clusters in data sets with different cluster shapes including arbitrary shapes. [ABSTRACT FROM AUTHOR]

• Convolutional Neural Networks can be improved in terms of the classification performance and robustness by using variable weight structures. • Analysis of different data processing methods, models' robustness and statistical properties. • Comparative analysis of variable weight convolutional neural networks and other widely used machine learning techniques. • Medical applications to the classification of seizure phases and types. Deep learning techniques have recently achieved impressive results and raised expectations in the domains of medical diagnosis and physiological signal processing. The widely adopted methods include convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, the existing models possess static connection weights between layers, which might limit the generalization capability and the classification performance of the models as the weights of different layers are fixed after training. Furthermore, to deal with a large amount of data, a neural network with a sufficiently large size is required. This paper proposes the variable weight convolutional neural networks (VWCNNs), which are a type of network structure employing dynamic weights instead of static weights in their convolutional layers and fully-connected layers. VWCNNs are able to adapt to different characteristics of input data and can be viewed as an infinite number of traditional, fixed-weight CNNs. We will show that the proposed VWCNN structure outperforms the conventional CNN in terms of the classification accuracy, generalization capability, and robustness when the inputs are contaminated by noise. In this paper, VWCNNs are applied to the classification of three seizure phases (seizure-free, pre-seizure and seizure) based on measured electroencephalography (EEG) data. VWCNNs achieve 100% test accuracy and show strong robustness in the classification of the three seizure phases, and thus show the potential to be a useful classification tool for medical diagnosis. Furthermore, the classification of seven types of seizures is investigated in this paper using the world's largest open source database of seizure recordings, TUH EEG seizure corpus. Comparisons with conventional CNNs, RNN, MobileNet, ResNet, DenseNet and traditional machine learning methods including random forest, decision tree, support vector machine, K-nearest neighbours, standard neural networks, and Naïve Bayes are being conducted using realistic test data sets. The results demonstrate that VWCNNs have advantages over other classifiers in terms of classification accuracy and robustness. [ABSTRACT FROM AUTHOR]

The paper presents numerical experiments and some theoretical developments in prediction with expert advice (PEA). One experiment deals with predicting electricity consumption depending on temperature and uses real data. As the pattern of dependence can change with season and time of the day, the domain naturally admits PEA formulation with experts having different "areas of expertise". We consider the case where several competing methods produce online predictions in the form of probability distribution functions. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). A popular example of scoring rule for continuous outcomes is Continuous Ranked Probability Score (CRPS). In this paper the problem of combining probabilistic forecasts is considered in the PEA framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk aggregating algorithm using CRPS as a loss function can be obtained. Also, we incorporate a "smooth" version of the method of specialized experts in this scheme which allows us to combine the probabilistic predictions of the specialized experts with overlapping domains of their competence. [ABSTRACT FROM AUTHOR]

• In the paper we propose a novel approach for solving puzzles whose pieces are of general shape, which need not necessarily match precisely due to erosion, and which have degraded unevenly over time. • We define the notion of valid transformations and propose a method of sampling them. This method may find uses in other domains. • We define a novel dissimilarity score that takes into account the special characteristics of the domain. • We introduce the notion of confidence in the dissimilarity scores, which depends not only on the value of the dissimilarity, but also on those of the competing matches. Confidence may be used not only in re-assembly, but also in other applications that are based on dissimilarity. This paper focuses on the re-assembly of an archaeological artifact, given images of its fragments. This problem can be considered as a special challenging case of puzzle solving. The restricted case of re-assembly of a natural image from square pieces has been investigated extensively and was shown to be a difficult problem in its own right. Likewise, the case of matching "clean" 2D polygons/splines based solely on their geometric properties has been studied. But what if these ideal conditions do not hold? This is the problem addressed in the paper. Three unique characteristics of archaeological fragments make puzzle solving extremely difficult: (1) The fragments are of general shape; (2) They are abraded, especially at the boundaries (where the strongest cues for matching should exist); and (3) The domain of valid transformations between the pieces is continuous. The key contribution of this paper is a fully-automatic and general algorithm that addresses puzzle solving in this intriguing domain. We show that our approach manages to correctly reassemble dozens of broken artifacts and frescoes. [ABSTRACT FROM AUTHOR]

The paper is devoted to two problems: (1) reinforcement of SVM algorithms, and (2) justification of memorization mechanisms for generalization. (1) Current SVM algorithm was designed for the case when the risk for the set of nonnegative slack variables is defined by l 1 norm. In this paper, along with that classical l 1 norm, we consider risks defined by l 2 norm and l ∞ norm. Using these norms, we formulate several modifications of the existing SVM algorithm and show that the resulting modified SVM algorithms can improve (sometimes significantly) the classification performance. (2) Generalization ability of existing learning algorithms is usually explained by arguments involving uniform convergence of empirical losses to the corresponding expected losses over a given set of functions. However, along with bounds for uniform convergence of empirical losses to the expected losses, the VC theory also provides bounds for relative uniform convergence. These bounds lead to a more accurate estimate of the expected loss. Advanced methods of estimating of expected risk of error have to leverage these bounds, which also support mechanisms of training data memorization, which, as the paper demonstrates, can improve classification performance. [ABSTRACT FROM AUTHOR]

Streaming feature selection (SFS), is the task of selecting the most informative features in dealing with high-dimensional or incrementally growing problems. Several SFS algorithms have been proposed in the literature. However, they do not consider all feature subsets at the redundancy analysis step due to computational concerns. Moreover, they do not reconsider previously removed features which leads to losing most of the useful information. In this paper, the redundancy analysis step is defined as a binary optimization problem. Then, a binary bat algorithm (BBA) is adopted to find the minimal informative subsets. In this way, a large number of feature subsets can be considered effectively at the redundancy analysis step. In addition, an effective priority list is used to maintain previously removed redundant features. Such a list allows the re-examination of informative features. As a result, it is possible to consider the mutual information between features that are not streamed in an small time interval. Experimental studies on fifteen different types of datasets show that our approach is superior to state-of-the-art online and offline streaming feature selection methods in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

• A unified model for sparse optimal scoring is proposed by employing L u -norm (0 ≤ q ≤ 1) regular term where L o − norm and(L u) -norm (0 ≤ q ≤ 1) will be selected adaptively to find more sparser solutions. • We derive an efficient iterative algorithm based on alternating direction method of multipliers (ADMM) for the new formulation. The new proposed method can solve (L o −) norm regularized problem directly rather than using convex or nonconvex approximations of (L o −) norm. • The convergence of the algorithm is analyzed theoretically. • Extensive numerical experiments show that our algorithm is efficient not only in classification accuracy but also in sparsity. Optimal scoring (OS), an equivalent form of linear discriminant analysis (LDA), is an important supervised learning method and dimensionality reduction tool. However, it is still a challenge for the classical OS on small sample size (SSS) datasets. In this paper, to find sparse discriminant vectors, we propose a unified model for sparse optimal scoring (SOS) by virtue of the generalized ℓ q -norm (0 ≤ q ≤ 1). To overcome the difficulty in treating the generalized ℓ q -norm, we propose an efficient alternative direction method of multipliers (ADMM), where proximity operator of ℓ q -norm is employed for different q values. Meanwhile, the convergence results of our method are also established. Numerical experiments on artificial and benchmark datasets demonstrate the effectiveness and feasibility of our proposed method. [ABSTRACT FROM AUTHOR]

• A hybrid Machine Learning (ML) approach for an efficient wrapper feature selection is proposed for enhancing the performance and reducing the computation time of IDS while increasing the accuracy to detect the intruder in IoT networks. • The hybrid ML method consists of two techniques: Differential Evolution (DE) and Extreme Learning Machine (ELM). The DE is responsible for choosing the beneficial features while the ELM is employed to assess the selected features. • Extensive experiments are implemented to evaluate the proposed approach using different performance metrics. In addition, to verify the efficiency of proposed model, the comparison is achieved with some related existing methods such as CFS + SVM [20] , CFS + ANN [19] , GA + Bagging [18] , CFS + RF [30] , and Chi-squared + RF [30]. The intrusion detection system (IDS) has gained a rapid increase of interest due to its widely recognized potential in various security fields, however, it suffers from several challenges. Different network datasets have several redundant and irrelevant features that affect the decision of the IDS classifier. Therefore, it is essential to decrease these features to improve the system performance. In this paper, an efficient wrapper feature selection method is proposed for improving the performance and decreasing the processing time of the IDS. The proposed approach employs a differential evaluation algorithm to select the useful features whilst the extreme learning machine classifier is applied after feature selection to evaluate the selected features. Many experiments are performed using the full NSL-KDD dataset to evaluate the performance of the proposed method. The results prove that the proposed approach can efficiently reduce the features, increase the accuracy, reduce the false alarm rates, and improve the processing time of the IDS in comparison to other recent related works. [ABSTRACT FROM AUTHOR]

• We proposed a novel AdaptiveWeighted Guided Image Filtering algorithm which outperforms the existing guided filters to preserve accurate edges. • We designed four special cases of the adaptive adjustment algorithm to remove the noise while enhancing depth details. • We proposed a new strategy based on AWGIF algorithm for depth enhancement in Shape-From-Focus. • Our proposed method are applied to the emerging AR/VR on hand-held devices in real-time. Existing shape from focus (SFF) techniques cannot preserve depth edges and fine structural details from a sequence of multi-focus images. Moreover, noise in the sequence affects the accuracy of the depth map. In this paper, a novel depth enhancement algorithm for the SFF based on an adaptive weighted guided image filtering (AWGIF) is proposed to address the above issues. The AWGIF is applied to decompose an initial depth map estimated by the traditional SFF into base and detail layers. In order to preserve the edges accurately in the refined depth map, the guidance image is constructed from the sequence, and the coefficient of the AWGIF is utilized to suppress the noise while enhancing the fine depth details. Experiments on real and synthetic objects demonstrate the superiority of our algorithm in terms of anti-noise, and the ability to preserve depth edges and fine structural details w.r.t. existing methods. [ABSTRACT FROM AUTHOR]

• The paper evaluates algorithms that approximate unorganised point clouds by skeletons. • A Homologically Persistent Skeleton (HoPeS) has guarantees proved for the first time. • The optimality guarantee means that subgraphs of HoPeS show cycles hidden in a cloud. • Other guarantees give conditions when a graph can be reconstructed from a noisy sample. • Evaluations on large data show highest levels of noise for correct reconstructions. Data Science aims to extract meaningful knowledge from unorganised data. Real datasets usually come in the form of a cloud of points. It is a requirement of numerous applications to visualise an overall shape of a noisy cloud of points sampled from a non-linear object that is more complicated than a union of disjoint clusters. The skeletonisation problem in its hardest form is to find a 1-dimensional skeleton that correctly represents the shape of the cloud. This paper compares different algorithms that solve the above skeletonisation problem for any point cloud and guarantee a successful reconstruction. For example, given a highly noisy point sample of an unknown underlying graph, a reconstructed skeleton should be geometrically close and homotopy equivalent to (has the same number of independent cycles as) the underlying graph. One of these algorithms produces a Homologically Persistent Skeleton (HoPeS) for any cloud without extra parameters. This universal skeleton contains subgraphs that provably represent the 1-dimensional shape of the cloud at any scale. Other subgraphs of HoPeS reconstruct an unknown graph from its noisy point sample with a correct homotopy type and within a small offset of the sample. The extensive experiments on synthetic and real data reveal for the first time the maximum level of noise that allows successful graph reconstructions. [ABSTRACT FROM AUTHOR]

Kernel possibilistic fuzzy C-means with local information (KWPFLICM) has important research significance of image segmentation, but it is very sensitive to high noise or outliers. To enhance the segmentation performance of the algorithm, this paper proposes a kernelized total Bregman divergence-driven possibilistic fuzzy clustering with local information (TKWPFLICM). Firstly, a polynomial kernel function is introduced to kernelize total Bregman divergence (TBD), and local neighborhood information of the pixel is used to modify it, which overcomes the shortcomings of Bregman divergence (BD) with rotation variability; Secondly, the modified kernelized TBD and possibilistic typicality are combined to further enhance the anti-noise ability of the algorithm; Finally, the modified kernelized TBD is introduced into the objective function of KWPFLICM algorithm, then a novel robust fuzzy clustering algorithm is derived by optimization theory. Experimental results show that compared with existing fuzzy clustering-related algorithms, the average SA improvement on TKWPFLICM algorithm is in the range of 0.791% to 33.237%. Therefore, TKWPFLICM algorithm has better anti-noise robustness and segmentation accuracy. [ABSTRACT FROM AUTHOR]

• We propose incomplete multi-view clustering with cosine similarity (IMCCS) for partitioning incomplete multi-view data. • IMCCS calculates the cosine similarity of incomplete multi-view data in the original multi-view space. • Gradient descent with the multiplicative update rule is presented to solve the objective of IMCCS. • IMCCS outperforms state-of-the-art incomplete multi-view clustering methods. Incomplete multi-view clustering partitions multi-view data suffering from missing views, for which matrix factorization approaches seek the latent representation of incomplete multi-view data and constitute one effective category of methods. To exploit data properties further, manifold structure preserving is also incorporated into matrix factorization. However, previous methods optimized the data similarity matrix in the manifold structure preserving term as an unknown variable, which is not guaranteed to faithfully represent the similarities of the original multi-view data and also increases the computational difficulty. To overcome these drawbacks, in this paper, we propose Incomplete Multi-view Clustering with Cosine Similarity (IMCCS). In IMCCS, we directly calculate the cosine similarity in the original multi-view space to strengthen the ability of preserving the manifold structure of the original multi-view data. There is no need to introduce the additional variable. The manifold structure preserving term with cosine similarity and the matrix factorization term are integrated into a unified objective function. An iterative algorithm with gradient descent is designed to solve this objective. Extensive experiments on multi-view datasets show that IMCCS outperforms state-of-the-art incomplete multi-view clustering methods. [ABSTRACT FROM AUTHOR]

• We propose a systematic approach to find bases of moment invariants with respect to orthogonal transformations using the generator method for scalar, vector, and tensor fields in two and three dimensions. • We show that it is always possible to construct a basis using all homo- 5 geneous invariants and simultaneous invariants with no more than two different moment tensors. • To the best of our knowledge, this results in the first 3D generator ap- proach that produces bases that are complete, independent, and exible, i.e., working for any input pattern. • We reveal so far unknown structural similarity between the 3D generator approach and its 2D counterpart as well as between the 3D generator approach and the 3D normalization approach. Moment invariants have been successfully applied to pattern detection tasks in 2D and 3D scalar, vector, and matrix valued data. However so far no flexible basis of invariants exists, i.e., no set that is optimal in the sense that it is complete and independent for every input pattern. In this paper, we prove that a basis of moment invariants can be generated that consists of tensor contractions of not more than two different moment tensors each under the conjecture of the set of all possible tensor contractions to be complete. This result allows us to derive the first generator algorithm that produces flexible bases of moment invariants with respect to orthogonal transformations by selecting a single non-zero moment to pair with all others in these two-factor products. Since at least one non-zero moment can be found in every non-zero pattern, this approach always generates a complete set of descriptors. [ABSTRACT FROM AUTHOR]

• Efficient kNN search based on feature learning, clustering, and adaptive k values. • Several proposals to automatically optimize the search parameters. • Comprehensive experimentation with 10 datasets of different typology and size. • Results demonstrate that the proposal outperforms state-of-the-art methods. The k -Nearest Neighbor (k NN) algorithm is widely used in the supervised learning field and, particularly, in search and classification tasks, owing to its simplicity, competitive performance, and good statistical properties. However, its inherent inefficiency prevents its use in most modern applications due to the vast amount of data that the current technological evolution generates, being thus the optimization of k NN-based search strategies of particular interest. This paper introduces the caKD+ algorithm, which tackles this limitation by combining the use of feature learning techniques, clustering methods, adaptive search parameters per cluster, and the use of pre-calculated K-Dimensional Tree structures, and results in a highly efficient search method. This proposal has been evaluated using 10 datasets and the results show that caKD+ significantly outperforms 16 state-of-the-art efficient search methods while still depicting such an accurate performance as the one by the exhaustive k NN search. [ABSTRACT FROM AUTHOR]

This paper introduces a method for achieving online Bayesian learning: • for time series data of indefinite (possibly infinite) length, • where the model is statistically non-stationary over time, and • the model includes a circular variable. The integrals required for full Bayesian inference are intractable, so we use a varational approximation method and place priors over precision hyperparameters to ensure that • the posterior probability distributions do not become overly tight, which would impede its ability to recognise and track changes, and • no values in the system are able to continuously increase and hence exceed the numerical representation of the programming language. With only a single round of updates in the variational Bayesian scheme per observation, a processing rate of at least 11 kHz is achieved. We introduce an online variational Bayesian model for tracking changes in a non-stationary, multivariate, temporal signal, using as an example the changing frequency and amplitude of a noisy sinusoidal signal over time. The model incorporates each observation as it arrives and then discards it, and places priors over precision hyperparameters to ensure that (i) the posterior probability distributions do not become overly tight, which would impede its ability to recognise and track changes, and (ii) no values in the system are able to continuously increase and hence exceed the numerical representation of the programming language. It is thus able to perform truly online processing for an infinitely long set of observations. Only a single round of updates in the variational Bayesian scheme per observation is used, and the complexity of the algorithm is constant in time. The proposed method is demonstrated on a large number of synthetic datasets, comparing the results from the full model (with precision hyperparameters as variables with priors) with those from the base model where the precision hyperparameters are fixed values. The full model is also demonstrated on a set of real climate data. [ABSTRACT FROM AUTHOR]

• A novel non-parametric clustering algorithm based on the concept of divide-and-merge. • Ability to discover both convex and non-convex shaped clusters. • Ability to discover clusters different in densities. • Ability to detect and remove outliers/noise in the data. • Usability for high dimensional data and easily tunable or fixed hyperparameters. In this paper, a novel non-parametric clustering algorithm which is based on the concept of divide-and-merge is proposed. The proposed algorithm is based on two primary phases, after data cleaning: (i) the Division phase and (ii) the Merging phase. In the initial phase of division, the data is divided into an optimized number of small sub-clusters utilizing all the dimensions of the data. In the second phase of merging, the small sub-clusters obtained as a result of division are merged according to an advanced statistical metric to form the actual clusters in the data. The proposed algorithm has the following merits: (i) ability to discover both convex and non-convex shaped clusters, (ii) ability to discover clusters different in densities, (iii) ability to detect and remove outliers/noise in the data (iv) easily tunable or fixed hyperparameters (v) and its usability for high dimensional data. The proposed algorithm is extensively tested on 20 benchmark datasets including both, the synthetic and the real datasets and is found better/competing to the existing state-of-the-art parametric and non-parametric clustering algorithms. [ABSTRACT FROM AUTHOR]

• A novel unsupervised feature selection method is proposed which aims at preserving the local invariance of feature subspace by local linear embedding and manifold regularization simultaneously. • An alternating algorithm based on alternative direction method of multiplier s with computational complexity analysis is designed to handle the resulting optimization problem. • A variety of experiments on several publicly available real world datasets are conducted to demonstrate the effectiveness and superiority of the proposed method. As one of the important dimensionality reduction techniques, unsupervised feature selection (UFS) has enjoyed amounts of popularity over the last few decades, which can not only improve learning performance, but also enhance interpretability and reduce computational costs. The existing UFS methods often model the data in the original feature space, which cannot fully exploit the discriminative information. In this paper, to address this issue, we investigate how to strengthen the relationship between UFS and the feature subspace, so as to select relevant features more straightforwardly and effectively. Methodologically, a novel UFS approach, referred to as Graph Regularized Local Linear Embedding (GLLE), is proposed by integrating local linear embedding (LLE) and manifold regularization constrained in feature subspace into a unified framework. To be more specific, we explicitly define a feature selection matrix composed of 0 and 1, which can realize the process of UFS. For the purpose of modelling the feature selection matrix, we propose to preserve the local linear reconstruction relationship among neighboring data points in the feature subspace, which corresponds to LLE constrained in the feature subspace. To make the feature selection matrix more accurate, we propose to use manifold regularization as an assistant of LLE to find the relevant and representative features such that the selected features can make each sample under the feature subspace be accordance with the manifold assumption. A tailored iterative algorithm based on Alternative Direction Method of Multipliers (ADMM) is designed to solve the proposed optimization problem. Extensive experiments on twelve real-world benchmark datasets are conducted, and the more promising results are achieved compared with the state-of-the-arts approaches. [ABSTRACT FROM AUTHOR]

• An innovatively improved SIFT structure that addresses the thorough feature extraction in all forgery cases. • A fast keypoint-label matching (FKLM) that creates groups of keypoint-label so that every high-dimensional feature is assigned into one of these groups. • A coarse-to-fine filtering relying on intrinsic attributes of exact keypoint-matches is designed to reduce false keypoint-matches effectively. • The adaptive block filling relying on true keypoint-matches contributes to the accurate and efficient suspicious region filling. Video copy-move forgery detection (VCMFD) is a significant and greatly challenging task due to a variety of difficulties, including a huge amount of video information, diverse forgery types, rich forgery objects, and homogenous forgery sources. These difficulties raise four unresolved key challenges in VCMFD: i) ineffective detection in some popular forgery cases; ii) inefficient matching in processing numerous video pixels with hundred-dimensional features under dozens of matching iterations; iii) high false positive (F P) in detecting forgery videos; iv) low trade-off of efficiency and effectiveness in filling forgery region, and even failing in indicating forgeries at the pixel level. In this paper, a novel VCMFD method is proposed to address these issues: i) an innovatively improved SIFT structure that can address the thorough feature extraction in all video copy-move forgery cases; ii) a novel fast keypoint-label matching (FKLM) algorithm is proposed that creates some keypoint-label groups so that every high-dimensional feature is assigned into one of these groups. As a result, matching of video pixels can be directly done on a small number of keypoint-label groups only, leading to a nearly 500% raise in matching efficiency; iii) a new coarse-to-fine filtering relying on intrinsic attributes of exact keypoint-matches is designed to more effectively reduce the false keypoint-matches; iv) the adaptive block filling relying on true keypoint-matches contributes to the accurate and efficient suspicious region filling, even at the pixel level. Finally, the suspicious region locations with the forgery vision persistence concept indicate forgery videos. Compared to the state-of-art methods, the experiments show that our proposed method achieves the best detection accuracy, lowest F P , and improved at least 16% and 8% of F 1 scores on the GRIP 2.0 dataset and a combination of SULFA 2.0 & REWIND datasets. Furthermore, the proposed method is with low computational time (4.45 s/Mpixels), which is about 1/2-1/3 times of the latest DFMI-BM (8.02 s/Mpixels) and PM-2D (13.1 s/Mpixels) methods. [ABSTRACT FROM AUTHOR]

• Presents a simple and efficient black-box adversarial attack strategy. • The adversarial perturbation can be dense or sparse. • Both universal and image dependent adversarial attack can be performed. • Employs a simple variant of Differential Evolution capable of optimizing the high dimensional problem under concern. Constructing adversarial perturbations for deep neural networks is an important direction of research. Crafting image-dependent adversarial perturbations using white-box feedback has hitherto been the norm for such adversarial attacks. However, black-box attacks are much more practical for real-world applications. Universal perturbations applicable across multiple images are gaining popularity due to their innate generalizability. There have also been efforts to restrict the perturbations to a few pixels in the image. This helps to retain visual similarity with the original images making such attacks hard to detect. This paper marks an important step that combines all these directions of research. We propose the DEceit algorithm for constructing effective universal pixel-restricted perturbations using only black-box feedback from the target network. We conduct empirical investigations using the ImageNet validation set on the state-of-the-art deep neural classifiers by varying the number of pixels to be perturbed from a meager 10 pixels to as high as all pixels in the image. We find that perturbing only about 10% of the pixels in an image using DEceit achieves a commendable and highly transferable Fooling Rate while retaining the visual quality. We further demonstrate that DEceit can be successfully applied to image-dependent attacks as well. In both sets of experiments, we outperform several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A graph matching method based on fast normalized cut and multiplicative update mapping is proposed. • The discrete constraint is incorporated into the optimization step and the multiplicative update algorithm is utilized in the mapping process. • Both synthetic points and real-world images comparisons with the state-of-the-art methods validate the effectiveness of the proposed method. Point correspondence is a fundamental problem in pattern recognition and computer vision, which can be tackled by graph matching. Since graph matching is basically an NP-complete problem, some approximate methods are proposed to solve it. Continuous relaxation offers an effective approximate method for graph matching problem. However, the discrete constraint is not taken into consideration in the optimization step. In this paper, a fast normalized cut based graph matching method is proposed, where the discrete constraint is introduced into the optimization step. Specifically, first a semidefinite positive affinity matrix based form objective function is constructed by introducing a regularization term which is related to the discrete constraint. Then the fast normalized cut algorithm is utilized to find the continuous solution. Last, the discrete solution of graph matching is obtained by a multiplicative update algorithm. Experiments on both synthetic points and real-world images validate the effectiveness of the proposed method by comparing it with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Modeling the market evolution for start-up projects via a graph-based model. • Modeling the competitiveness pressure by aggregating funding states of rivals. • Propagation tree structure with hierarchical updating can track the market evolution. • The proposed model achieved the best result compared with all powerful baselines. In the online innovation market, the fund-raising performance of the start-up project is a concerning issue for creators, investors and platforms. Unfortunately, existing studies always focus on modeling the fund-raising process after the publishment of a project but the predicting of a project attraction in the market before setting up is largely unexploited. Usually, this prediction is always with great challenges to making a comprehensive understanding of both the start-up project and market environment. To that end, in this paper, we present a focused study on this important problem from a market graph perspective. Specifically, we propose a Graph-based Market Environment (GME) model for predicting the fund-raising performance of the unpublished project by exploiting the market environment. In addition, we discriminatively model the project competitiveness and market preferences by designing two graph-based neural network architectures and incorporating them into a joint optimization stage. Furthermore, to explore the information propagation problem with dynamic environment in a large-scale market graph, we extend the GME model with parallelizing competitiveness quantification and hierarchical propagation algorithm. Finally, we conduct extensive experiments on real-world data. The experimental results clearly demonstrate the effectiveness of our proposed model. [ABSTRACT FROM AUTHOR]

• We propose a residual generator (Rg-GAN) served as a better approximation divergence minimization framework for GAN, and prove that residual generator for standard and least-squares GAN are equivalent to the minimization of reverse-KL and a new instance of f-divergence, respectively. • We prove that Rg-GAN can be reduced to IPMs based GAN and bridge the gap between IPMs and f-divergence. • We propose a new loss function for the discriminator of Rg-GAN that manifests a better discriminative property and therefore improved on Rg-GAN generalisation ability. • We conduct experiments on multiple benchmark data sets and demonstrate that our proposed framework can mitigate the mode collapse issue and facilitate GAN to generate higher-quality images with negligible additional computation cost. GAN is a generative modelling framework which has been proven as able to minimise various types of divergence measures under an optimal discriminator. However, there is a gap between the loss function of GAN used in theory and in practice. In theory, the proof of the Jensen divergence minimisation involves the min-max criterion, but in practice the non-saturating criterion is instead used to avoid gradient vanishing. We argue that the formulation of divergence minimization via GAN is biased and may yield a poor convergence of the algorithm. In this paper, we propose the Residual Generator for GAN (Rg-GAN), which is inspired by the closed-loop control theory, to bridge the gap between theory and practice. Rg-GAN minimizes the residual between the loss of the generated data to be real and the loss of the generated data to be fake from the perspective of the discriminator. In this setting, the loss terms of the generator depend only on the generated data and therefore contribute to the optimisation of the model. We formulate the residual generator for standard GAN and least-squares GAN and show that they are equivalent to the minimisation of reverse-KL divergence and a novel instance of f-divergence, respectively. Furthermore, we prove that Rg-GAN can be reduced to Integral Probability Metrics (IPMs) GANs (e.g., Wasserstein GAN) and bridge the gap between IPMs and f-divergence. Additionally, we further improve on Rg-GAN by proposing a loss function for the discriminator that has a better discrimination ability. Experiments on synthetic and natural images data sets show that Rg-GAN is robust to mode collapse, and improves the generation quality of GAN in terms of FID and IS scores. [ABSTRACT FROM AUTHOR]

• Unified approach to recognize egocentric hand gesture and detect fingertips. • Binary output of learning network as visible fingers in a gesture. • Introducing an ensemble of direct regression to predict fingertip position. • Experiments on known database and real-life images along with VR application. • Achieved highly accurate gesture classification and fingertip position. Head-mounted device-based human-computer interaction often requires egocentric recognition of hand gestures and fingertips detection. In this paper, a unified approach of egocentric hand gesture recognition and fingertip detection is introduced. The proposed algorithm uses a single convolutional neural network to predict the probabilities of finger class and positions of fingertips in one forward propagation. Instead of directly regressing the positions of fingertips from the fully connected layer, the ensemble of the position of fingertips is regressed from the fully convolutional network. Subsequently, the ensemble average is taken to regress the final position of fingertips. Since the whole pipeline uses a single network, it is significantly fast in computation. Experimental results show that the proposed method outperforms the existing fingertip detection approaches including the Direct Regression and the Heatmap-based framework. The effectiveness of the proposed method is also shown in-the-wild scenario as well as in a use-case of virtual reality. [ABSTRACT FROM AUTHOR]

• An exclusivity term is introduced to enforce the diversity of different views. • Multi-view subspace representation is constrained to find the cluster structure. • An effective optimization algorithm is proposed to solve the objective function. • Experiments demonstrate that our algorithm outperforms state-of-the-art approaches. Multi-view subspace clustering algorithms have recently been developed to process multi-view dataset clustering by accurately depicting the essential characteristics of multi-view data. Most existing methods focus on conduct self-representation property using a consistent representation and a set of specific representations with well-designed regularization to learn the common and specific knowledge among different views. However, specific representations only contain the unique information of each individual view, which limits their ability to fully excavate the diversity of multi-view data to enhance the complementarity among different views. Moreover, when conducting multi-view subspace clustering, the learned subspace self-representation and clustering are sequential and independent, which lacks consideration of the interaction between representation learning and the final clustering calculation. In this paper, a novel method termed consistent and diverse multi-view subspace clustering with structure constraint (CDMSC 2) is proposed to overcome the above-described deficiencies. (1) An exclusivity constraint term is employed to enhance the diversity of specific representations among different views for modeling consistency and diversity in a unified framework. (2) A clustering structure constraint is imposed on the subspace self-representation by factorizing the learned subspace self-representation into the cluster centroids and the cluster assignments with the goal of obtaining a clustering-oriented subspace self-representation. In addition, we carefully designed an efficient optimization algorithm to solve the objective function through relaxation and alternating minimization. Extensive experiments on five benchmark datasets in terms of six evaluation metrics demonstrate that our method outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a novel regularized TWSVM called ITWSVM for indefinite kernel. • We introduce a smooth quadratic hinge loss function and a maximum margin regularization term to ITWSVM. • We theoretically analyze the nature of ITWSVM and expand ITWSVM to indefinite kernels and multi-class classification. • We introduce difference of convex functions (DC) to solve the non-convex problem in indefinite kernel settings and further propose ITWSVM-DC. • Extensive experiments demonstrate that ITWSVM-DC is a robust and prominent algorithm and can perform excellently in different situations. Twin support vector machine (TWSVM) is an efficient algorithm for binary classification. However, the lack of the structural risk minimization principle restrains the generalization of TWSVM and the guarantee of convex optimization constraints TWSVM to only use positive semi-definite kernels (PSD). In this paper, we propose a novel TWSVM for indefinite kernel called indefinite twin support vector machine with difference of convex functions programming (ITWSVM-DC). The indefinite TWSVM (ITWSVM) leverages a maximum margin regularization term to improve the generalization of TWSVM and a smooth quadratic hinge loss function to make the model continuously differentiable. The representer theorem is applied to the ITWSVM and the convexity of the ITWSVM is analyzed. In order to address the non-convex optimization problem when the kernel is indefinite, a difference of convex functions (DC) is used to decompose the non-convex objective function into the subtraction of two convex functions and a line search method is applied in the DC algorithm to accelerate the convergence rate. A theoretical analysis illustrates that ITWSVM-DC can converge to a local optimum and extensive experiments on indefinite and positive semi-definite kernels show the superiority of ITWSVM-DC. [ABSTRACT FROM AUTHOR]

• A novel explainable neural network is proposed for modeling main effects and structured interactions. • The GAMI-Net is a disentangled feedforward network with multiple additive subnetworks. • GAMI-Net takes into account three interpretability constraints: sparsity, heredity, marginal clarity. • An adaptive training algorithm is developed for training GAMI-Net efficiently. • GAMI-Net enjoys superior interpretability and outperforms benchmark methods. The lack of interpretability is an inevitable problem when using neural network models in real applications. In this paper, an explainable neural network based on generalized additive models with structured interactions (GAMI-Net) is proposed to pursue a good balance between prediction accuracy and model interpretability. GAMI-Net is a disentangled feedforward network with multiple additive subnetworks; each subnetwork consists of multiple hidden layers and is designed for capturing one main effect or one pairwise interaction. Three interpretability aspects are further considered, including a) sparsity, to select the most significant effects for parsimonious representations; b) heredity, a pairwise interaction could only be included when at least one of its parent main effects exists; and c) marginal clarity, to make main effects and pairwise interactions mutually distinguishable. An adaptive training algorithm is developed, where main effects are first trained and then pairwise interactions are fitted to the residuals. Numerical experiments on both synthetic functions and real-world datasets show that the proposed model enjoys superior interpretability and it maintains competitive prediction accuracy in comparison to the explainable boosting machine and other classic machine learning models. [ABSTRACT FROM AUTHOR]

• A multi-task fully deep convolutional neural network is proposed for contactless fingerprint minutiae extraction. • A new loss function is proposed to jointly learn the tasks of minutiae detection and its direction regression from the whole fingerprints. • The proposed method operates directly on the gray scale contactless fingerprints without any preprocessing. • Experiments on three datasets have shown the effectiveness of the proposed algorithm. With the outbreak and wide spread of novel coronavirus (COVID-19), contactless fingerprint recognition has attracted more attention for personal recognition because it can provide significantly higher user convenience and hygiene than the traditional contact-based fingerprint recognition. However, it is still challenging to achieve a highly accurate recognition due to the low ridge-valley contrast and pose variances of contactless fingerprints. Minutiae points are a kind of ridge flow discontinuities, and robust and accurate extraction is an important step for most automatic fingerprint recognition algorithms. Most of existing methods are based on two stages which locate the minutiae points first and then compute their directions. The two-stage method cannot make full use of location and direction information. In this paper, we propose a multi-task fully deep convolutional neural network for jointly learning the minutiae location detection and its corresponding direction computation which operates directly on the whole gray scale contactless fingerprints. The proposed method consists of offline training and online testing stages. In the training stage, a fully deep convolutional neural network is built for the tasks of minutiae detection and its direction regression, with an attention mechanism to make the direction regression branch concentrate on the minutiae points. A new loss function is proposed to jointly learn the tasks of minutiae detection and its direction regression from the whole fingerprints. In the testing stage, the trained network is applied on the whole contactless fingerprint to generate the minutiae location and direction maps. The proposed multi-task leaning method performs better than the individual single task and it operates directly on the raw gray-scale contactless fingerprints without preprocessing. The results on three contactless fingerprint datasets show the proposed algorithm performs better than other minutiae extraction algorithms and the commercial software. [ABSTRACT FROM AUTHOR]

• Properties that help solve the clustering users dilemma. • Weighted properties formally differentiate clustering methods. • A formal classification highlights advantages of center-based clustering techniques. This paper makes a major step towards addressing a long-standing challenge in cluster analysis, known as the user's dilemma , which is the problem of selecting an appropriate clustering algorithm for a specific task. A formal approach for addressing this challenge relies on the identification of succinct, user-friendly properties that capture formal differences amongst clustering techniques. While helpful for gaining insight into the nature of clustering paradigms, there is a theory-practice gap that has so far limited the utility of this approach: Formal properties typically highlight advantages of classical linkage-based algorithms, while practical experience shows that center-based methods are preferable for many applications. We present simple new properties that delineate core differences between common clustering paradigms and overcome this theory-practice gap. The properties we present give a formal understanding of the advantages of center-based approaches for some applications and insight into when different clustering paradigms should be used. These properties address how sensitive algorithms are to changes in element frequencies, which we capture in a generalized setting where every element is associated with a real-valued weight. To complement extensive formal analysis, we discuss how these properties can be applied in practice. [ABSTRACT FROM AUTHOR]