Search

Adversarial training has been considered to be one of the most effective strategies to defend against adversarial attacks. Most existing adversarial training methods have shown a trade-off between training cost and robustness. This paper explores a new optimization direction from training data to reduce the computational cost of adversarial training without scarifying robustness. First, we show that some adversarial examples are less important, meaning that removing them does not hurt the robustness. Second, we propose a method to identify insignificant adversarial examples at a minimal cost. Third, we demonstrate that our approach can be integrated with other adversarial training frameworks with few modifications. The experimental results show that combined with previous works, our approach not only reduces about 20% of computational cost on the CIFAR10 and CIFAR100 datasets but also improves about 1.5% natural accuracy. With less computational cost, it achieves 58.22%, 30.68%, and 41.92% robust accuracy on CIFAR10, CIFAR100, and ImageNet datasets respectively, which are higher than those of the original methods. • Analyze the effectiveness of adversarial examples for adversarial training. • Design a method to remove inefficient adversarial examples but maintain the accuracy. • Design an algorithm to detect inefficient adversarial examples without extra cost. • Conduct experiments to demonstrate the compatibility with other frameworks. [ABSTRACT FROM AUTHOR]

Cluster analysis is one of the most important research issues in data mining and machine learning. To date, numerous clustering algorithms have been proposed to tackle the fixed-length vector data. In many real applications, we need to detect clusters from a set of discrete sequences in which each sequence is an ordered list of items. Due to the sequential and discrete nature, the discrete sequence clustering problem is more challenging and most of existing vector data clustering algorithms cannot be directly employed. In this paper, we present a stochastic algorithm for clustering discrete sequences. Our method first quickly generates a set of random partitions over the sequential data set and then merges these random clustering results via weighted graph construction and partition. We perform extensive empirical comparisons on real data sets to show that our method is comparable to those state-of-the-art clustering algorithms with respect to both accuracy and efficiency. • A randomized algorithm is proposed to solve the discrete sequence clustering issue. • Our algorithm provides high success rate for sequence linkage in clusters. • Experimental results demonstrate the feasibility and effectiveness of our method. [ABSTRACT FROM AUTHOR]

This paper investigates algorithms for matrix factorization when some or many components are missing, a problem that arises frequently in computer vision and pattern recognition. We demonstrate that the Jacobian used in the damped Wiberg (DW) method is exactly the same as that of Kaufman's simplified variable projection (VP) algorithm. Our analysis provides a novel perspective on the efficiency of VP algorithms by improving the strong convexity of the approximate function. To enhance numerical stability, we set a lower bound on the damping parameter instead of adding a null space like the DW algorithm. Another challenge of low-rank matrix decomposition with missing data is the existence of many sharp local minima, which are often distributed in narrow valleys of the landscape of objection functions. Falling into such minima tends to result in poor reconstruction results. To address this issue, we design a non-monotonic VP algorithm, which can facilitate the algorithm to escape from sharp minima and converge to flatter minima. Numerical experiments confirm the effectiveness and efficiency of the proposed nonmonotone VP algorithm. • We provide a new perspective to examine the superiority of the VP algorithm from a convexity point of view. • We propose a nonmonotone VP algorithm, which can facilitate the algorithm to escape from sharp minima. • We conduct extensive experiments to validate the effectiveness of the proposed nonmonotone algorithm. [ABSTRACT FROM AUTHOR]

Wide-angle video frames typically contain more information, but they also exhibit distortion that degrades the visual quality, especially at the edges. To eliminate this distortion from videos, we propose a self-supervised iterative optimization method in this paper. Specifically, we construct a motion parameter estimation model utilizing two consecutive distorted frames, where motion parameters comprise affine transform and distortion parameters. We apply the Gauss–Newton algorithm to minimize the sum-of-squares error between frames and update parameters. Treating inter-frame motion as undistort-affine-distort transformations, frame alignment is achieved by continuously adjusting transform parameters. Ultimately, frames are corrected using the converged parameters. We generated a synthetic dataset with various distortion parameters for evaluation. Experiments demonstrate superior performance versus state-of-the-art methods on synthetic and real wide-angle videos. Our algorithm also achieves higher parameter estimation accuracy without sacrificing efficiency. • proposes a algorithm for video distortion correction based on iterative optimization. • the distort transform and undistort transform are derived separately and the range of parameters and the experimental error are given. • Based on this, we transform the distortion correction into a parameter estimation problem in image alignment. • optimize the parameter estimation problem using Gauss-Newton algorithm. • Experimental results show that the proposed algorithms outperform some state-of-art algorithms in accuracy, without reducing robustness and efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we investigate L 0 -regularized logistic regression models, and design two fast and efficient algorithms for high-dimensional correlated data and massive data, respectively. Our first algorithm, the Variable Sorted Active Set (VSAS) algorithm, is based on the local quadratic approximation of the KKT conditions for L 0 -penalized maximum log-likelihood function in high-dimensional correlated data. We establish an L ∞ error upper bound for the estimator obtained by the VSAS algorithm and prove its optimal convergence rate. Moreover, when the target signal exceeds the detectable level, the estimator obtained by the VSAS algorithm can achieve the oracle estimator with high probability. Our second algorithm, Communication Effective Variable Sorted Active Set (CEVSAS), aims to solve high-dimensional and large-sample L 0 -regularized logistic regression models by reduce computational and communication costs, while maintaining estimation efficiency. Finally, simulations and real data demonstrate the effectiveness of our proposed VSAS and CEVSAS algorithms. • We develop a fast VSAS algorithm for L 0 regularized logistic regression involving high-dimensional correlated data. • An L ∞ upper bound is established on the estimation error of the solution sequence generated by VSAS algorithm. • The CEVSAS altorithm is proposed to reduce computational costs and ensure higher estimation efficiency. • We propose two adaptive versions of the VSAS and CEVSAS algorithms for selecting the optimal tuning parameters. [ABSTRACT FROM AUTHOR]

• The first attempt to design a GEP-based codematrix generation algorithm. The individual structure of GEP allows the codematrix to be optimized in a much faster and more effective way. • The feature selection scheme is combined with the design of codematrix, to accomplish the feature selection and codematrix optimization simultaneously. • A greedy strategy-based adaptive optimization method to further improve the individual's quality to deal with hard classes by appending proper columns to the codematrix. The multi-class classification task is one of the most common tasks in machine learning. As a typical solution based on a partitioning strategy, Error-Correcting Output Codes (ECOC) can transform a multi-class classification problem into multiple binary classification problems. The key of ECOC is to construct an effective codematrix to represent a set of class decomposition schemes, which transforms a multiclass problem into a group of binary class problems. Consequently, the design of a fast and effective ECOC codematrix generation method is of great research significance and value for solving multi-class classification problems. In ECOC algorithms, the design of codematrix is treated as a combination problem between different code columns, in which the evolutionary algorithm shows a great advantage. Based on this consideration, the Gene Expression Programming (GEP) is applied to search for the codematrix with high performance because its expressive tree structure makes it well represent codematrcies for subsequent optimization operations. This paper proposes an adaptive ECOC algorithm based on Gene Expression Programming (GEP) and similarity measurement matrix, named GEP-ECOC. In our GEP, each individual represents a set of columns to form a random ECOC codematrix, which is optimized in the evolutionary process. Meanwhile, the crossover and mutation operations are modified to include a legality checking process to ensure that the generated codematrix satisfies the ECOC constraints. The GEP-based ECOC codematrix generation algorithm can quickly produce a codematrix with better performance, which ensures the efficiency of the algorithm to a certain extent. In addition, an adaptive algorithm based on a similarity measurement matrix is proposed to add new columns to the current codematrix, aiming to better handle hard classes. Our algorithm is compared with other algorithms on various data sets, and the experimental results confirm that our GEP-ECOC can balance the efficiency and performance of the algorithm and achieve higher performance. [ABSTRACT FROM AUTHOR]

Class imbalance problem commonly exists in multi-label classification (MLC) tasks. It has non-negligible impacts on the classifier performance and has drawn extensive attention in recent years. Borderline oversampling has been widely used in single-label learning as a competitive technique in dealing with class imbalance. Nevertheless, the borderline samples in multi-label data sets (MLDs) have not been studied. Hence, this paper deeply discussed the borderline samples in MLDs and found they have different neighboring relationships with class borders, which makes their roles different in the classifier training. For that, they are divided into two types named the self-borderline samples and the cross-borderline samples. Further, a novel MLDs resampling approach called Multi-Label Borderline Oversampling Technique (MLBOTE) is proposed for multi-label imbalanced learning. MLBOTE identifies three types of seed samples, including interior, self-borderline, and cross-borderline samples, and different oversampling mechanisms are designed for them, respectively. Meanwhile, it regards not only the minority classes but also the classes suffering from one-vs-rest imbalance as those in need of oversampling. Experiments on eight data sets with nine MLC algorithms and three base classifiers are carried out to compare MLBOTE with some state-of-art MLDs resampling techniques. The results show MLBOTE outperforms other methods in various scenarios. • A new borderline oversampling technique for multi-label imbalanced learning. • Defining self-borderline and cross-borderline samples in multi-label data sets. • Handling one-vs-rest imbalance in multi-label imbalanced learning. • Performing competitively in the experiments with C4.5, RBF kernel SVM and linear SVM. [ABSTRACT FROM AUTHOR]

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Encouraging results are also observed when applying PEEL to compress the semantic segmentation model. Our code is available at https://github.com/cardwing/Codes-for-PEEL. • Previous methods typically perform iterative pruning and are tedious and expensive. • A simple yet effective method is proposed to quickly produce a slim model with negligible cost. • Our method constructs a predefined backbone and conducts resource reallocation on it. • Resource reallocation shifts parameters from less informative layers to more important ones in one round. • Extensive experiments on ImageNet with various models validate the effectiveness. [ABSTRACT FROM AUTHOR]