Search

• We propose an improved differential evolution (DE) algorithm by composite binary crossover and Pareto refining operations to the DE. • DE solves the quantifying fraudulent transactions problem by minimizing the selected feature cost while maximizing the quantification ability. • We show that the proposed DE outperforms four other variants of DE and five state-of-the-art EAs on various credit datasets. • The results are also statically validated through the Friedman test and Wilcoxon post-hoc analysis. [Display omitted] Identification of fraudulent credit card transactions is a complex problem mainly due to the following factors: 1) The relative behavior of customers and fraudsters may alter over time. 2) The ratio of legitimate to fraudulent transactions is highly imbalanced, and 3) Investigators examine a small segment of transactions in a reasonable time frame. Researchers have proposed various algorithms to identify potential fraud in a new incoming transaction. However, these approaches require significant human investigator effort and are sometimes misleading. To address this issue, this paper proposes an improved multiobjective differential evolution (DE) algorithm to estimate the distribution of fraudulent transactions in a set of new incoming transactions, referred to as quantifying fraudulent transactions. Our paper has three major novelties. First, we present the problem formulation of cost-based feature selection with maximum quantification ability. Second, we improve the DE by applying effective trial vector generation algorithms to the random control parameter settings to exploit the advantage of individual DE variants. Third, we develop the maximum-relevancy-minimum-redundancy-based Pareto refining operator to enhance the self-learning ability of individuals in Pareto solutions. We compare our approach against four other modifications of DE and five state-of-the-art evolutionary algorithms on real-time credit datasets in streaming and non-streaming frameworks using hyper-volume, two-set coverage, and spread performance metrics. [ABSTRACT FROM AUTHOR]

Adversarial training has been considered to be one of the most effective strategies to defend against adversarial attacks. Most existing adversarial training methods have shown a trade-off between training cost and robustness. This paper explores a new optimization direction from training data to reduce the computational cost of adversarial training without scarifying robustness. First, we show that some adversarial examples are less important, meaning that removing them does not hurt the robustness. Second, we propose a method to identify insignificant adversarial examples at a minimal cost. Third, we demonstrate that our approach can be integrated with other adversarial training frameworks with few modifications. The experimental results show that combined with previous works, our approach not only reduces about 20% of computational cost on the CIFAR10 and CIFAR100 datasets but also improves about 1.5% natural accuracy. With less computational cost, it achieves 58.22%, 30.68%, and 41.92% robust accuracy on CIFAR10, CIFAR100, and ImageNet datasets respectively, which are higher than those of the original methods. • Analyze the effectiveness of adversarial examples for adversarial training. • Design a method to remove inefficient adversarial examples but maintain the accuracy. • Design an algorithm to detect inefficient adversarial examples without extra cost. • Conduct experiments to demonstrate the compatibility with other frameworks. [ABSTRACT FROM AUTHOR]

Cluster analysis is one of the most important research issues in data mining and machine learning. To date, numerous clustering algorithms have been proposed to tackle the fixed-length vector data. In many real applications, we need to detect clusters from a set of discrete sequences in which each sequence is an ordered list of items. Due to the sequential and discrete nature, the discrete sequence clustering problem is more challenging and most of existing vector data clustering algorithms cannot be directly employed. In this paper, we present a stochastic algorithm for clustering discrete sequences. Our method first quickly generates a set of random partitions over the sequential data set and then merges these random clustering results via weighted graph construction and partition. We perform extensive empirical comparisons on real data sets to show that our method is comparable to those state-of-the-art clustering algorithms with respect to both accuracy and efficiency. • A randomized algorithm is proposed to solve the discrete sequence clustering issue. • Our algorithm provides high success rate for sequence linkage in clusters. • Experimental results demonstrate the feasibility and effectiveness of our method. [ABSTRACT FROM AUTHOR]

Highlights • This paper proposes a novel connected components labeling (CCL) approach, a gamma-signal-regulated approach, for higher efficiency. • A new and more efficient block-based two-scan CCL algorithm, Eight-Connected Gamma-Signal-regulated (ECGS) algorithm, is proposed. • Experiment results have demonstrated that ECGS can outperform current state-of-the-art CCL algorithms for a number of digital images. Abstract This paper proposes a novel connected components labeling (CCL) approach that introduces a gamma signal to record certain mask pixels' values to eliminate duplicated pixel checking and regulate the labeling process for higher efficiency. A new block-based two-scan CCL algorithm, Eight-Connected Gamma-Signal-regulated (ECGS) algorithm, is designed and developed by applying this approach to evaluate a block of 2 × 2 pixels (with just 6 mask pixels) in each iteration such that the total number of operations is considerably reduced and the labeling efficiency is significantly improved. The experiments conducted on a public benchmark, YACCLAB (Yet Another Connected Components Labeling Benchmark), have demonstrated that the proposed ECGS algorithm can outperform current state-of-the-art CCL algorithms for a number of digital images. [ABSTRACT FROM AUTHOR]

Highlights • We find an observation important to the LSN subspace segmentation. • We provide the theoretical support of the mentioned observation. • Our paper is the first to adopt infinite norm in subspace segmentation. • The computational complexity of our ADM algorithm is lower than O (n 3). • Our method achieve the state-of-the-art results in extensive experiments. Abstract Spectral-clustering based methods have recently attracted considerable attention in the field of subspace segmentation. The approximately block-diagonal graphs achieved by this kind of methods usually contain some noise, i.e., nonzero elements in the off-diagonal region, due to outlier contamination or complex intrinsic structure of the dataset. In the experiment of most previous work, the number of the subspaces is often no more than 10. In this situation, this kind of noise almost has no influence on the segmentation results. However, the segmentation performance could be negatively affected by the noise when the number of subspaces is large, which is quite common in the real-world applications. In this paper, we address the problem of LSN subspace segmentation, i.e., large subspace number subspace segmentation. We first show that the approximately block-diagonal graph with the smaller difference in its diagonal blocks will be more robust to the off-diagonal noise mentioned above. Then, by using the infinity norm to control the bound of the difference in the diagonal blocks, we propose infinity norm minimization for LSN subspace segmentation. Experimental results demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Highlights • We propose a linear classifier design method in the weight space based on the observation that a training sample represents a plane in the weight space. • We proposed search algorithms to find the optimal subspace, though the optimal solution may not be guaranteed since the full search cannot be used. • Experimental results show that the proposed classifier performed better than or equivalent to existing linear classifiers (linear SVM, LDA). Abstract In this paper, we propose a linear classifier design method in the weight space. A linear classifier is completely determined by a weight vector. To design a linear classifier is equivalent to finding a weight vector. When there are a number of training samples, each training sample represents a plane in the weight space. On one side of the plane, the training sample is correctly classified while it is incorrectly classified on the other side. Thus, finding the optimal linear classifier can be formulated as finding a subspace that provides the best classification accuracy. In this paper, we have developed an algorithm to find such optimal subspaces in the weight space. Although we could not find the optimal linear classifier due to prohibitive computational complexity, the proposed design method produced noticeable improvements in some cases. Experimental results show that the proposed linear classifier performed better than or equivalent to existing linear classifiers. [ABSTRACT FROM AUTHOR]

Highlights • First system for Simultaneous Localization and Mapping in large environments from Squared Planar Markers. • Print markers with a regular printer and place them in environment. Use a camera an our system finds the camera position while estimating the pose of the markers at the same time. • Real time operation on CPUs (150 Hz) with a single execution thread. Abstract SLAM is generally addressed using natural landmarks such as keypoints or texture, but it poses some limitations, such as the need for enough textured environments and high computational demands. In some cases, it is preferable sacrificing the flexibility of such methods for an increase in speed and robustness by using artificial landmarks. The recent work [1] proposes an off-line method to obtain a map of squared planar markers in large indoor environments. By freely distributing a set of markers printed on a piece of paper, the method estimates the marker poses from a set of images, given that at least two markers are visible in each image. Afterwards, camera localization can be done, in the correct scale. However, an off-line process has several limitations. First, errors can not be detected until the whole process is finished, e.g., an insufficient number of markers in the scene or markers not properly spotted in the capture stage. Second, the method is not incremental, so, in case of requiring the expansion of the map, it is necessary to repeat the whole process from start. Finally, the method can not be employed in real-time systems with limited computational resources such as mobile robots or UAVs. To solve these limitations, this work proposes a real-time solution to the problems of simultaneously localizing the camera and building a map of planar markers. This paper contributes with a number of solutions to the problems arising when solving SLAM from squared planar markers, coining the term SPM-SLAM. The experiments carried out show that our method can be more robust, precise and fast, than visual SLAM methods based on keypoints or texture. [ABSTRACT FROM AUTHOR]

This paper proposes a new method of personal identification that introduces the analysis of lip prints. In spite of its important role in forensic and biometric applications, the results of previous investigations into lip prints are scanty. This is mainly due to the difficulties that accompany any analysis of the lips: lips are very flexible and pliable, and successive lip print impressions – even those obtained from the same person – may significantly differ from one other. Our article’s principal contribution is a proposal for a new personal identification methodology and application that uses, for the first time, a strategy for biometric and forensic analysis of lip print structure. As a result of our analysis we propose a lip print pattern for each individual. This pattern contains only such furrows that occur on the greatest number of lip prints obtained from the same person, where these furrows’ locations and inclinations remain similar across the lip prints obtained. It should be noted that in our approach, instead of lip photos we employ lip prints, such as can be obtained at a crime scene. It is worth noticing also that we propose a new method of personal identification where, instead of popular machine learning methods,the furrow-analysis of lip prints is introduced. Thus, no classifier learning phase is required. According to the authors’ convictions, based on reports in the literature, the proposed approach describes for the first time a strategy as to how lip print structures could be analyzed in biometric applications. The main novelty of this paper is its use of multiple lip print furrows from the same person to determine the most common lip furrow distribution in four different directions. On the basis of the pattern recognition of the lip furrows, the identity of the person will be determined. The proposed method’s effectiveness is 92.73%, determined using a database containing 350 lip prints. [ABSTRACT FROM AUTHOR]

This paper explores the problem of circle fitting for incomplete (partial arc) laser scanning point cloud data in the presence of outliers. In mobile laser scanning, data are commonly incomplete because of the orientation of the scanning unit to the surveying objects and the limited street-based positions. Also, multiple structures in the built environment often produce clustered outliers. To address these problems, this paper combines robust Principal Component Analysis (PCA) and robust regression with an efficient algebraic circle fitting method to develop two algorithms for circle fitting. Experimental efforts show that the proposed algorithms are statistically robust and can tolerate a high-percentage (exceeding 44%) of clustered outliers with insignificant error levels, while still achieving better shape recognition compared to existing competitive methods. For example, for a simulation of 1000 quarter circle datasets including 20% clustered outliers, RANSAC estimated the circle radius with a Mean Squared Error (MSE) of 172.10, whereas the proposed algorithms fit circles with an MSE of less than 0.42. The algorithms have potential in many areas including building information modeling, particle tracking, product quality control, arboreal assessment, and road asset monitoring. [ABSTRACT FROM AUTHOR]

This guest editorial introduces the twenty two papers accepted for this Special Issue on Articulated Motion and Deformable Objects (AMDO). They are grouped into four main categories within the field of AMDO: human motion analysis (action/gesture), human pose estimation, deformable shape segmentation, and face analysis. For each of the four topics, a survey of the recent developments in the field is presented. The accepted papers are briefly introduced in the context of this survey. They contribute novel methods, algorithms with improved performance as measured on benchmarking datasets, as well as two new datasets for hand action detection and human posture analysis. The special issue should be of high relevance to the reader interested in AMDO recognition and promote future research directions in the field. [ABSTRACT FROM AUTHOR]

Multiple instance learning (MIL) is a form of weakly supervised learning where training instances are arranged in sets, called bags, and a label is provided for the entire bag. This formulation is gaining interest because it naturally fits various problems and allows to leverage weakly labeled data. Consequently, it has been used in diverse application fields such as computer vision and document classification. However, learning from bags raises important challenges that are unique to MIL. This paper provides a comprehensive survey of the characteristics which define and differentiate the types of MIL problems. Until now, these problem characteristics have not been formally identified and described. As a result, the variations in performance of MIL algorithms from one data set to another are difficult to explain. In this paper, MIL problem characteristics are grouped into four broad categories: the composition of the bags, the types of data distribution, the ambiguity of instance labels, and the task to be performed. Methods specialized to address each category are reviewed. Then, the extent to which these characteristics manifest themselves in key MIL application areas are described. Finally, experiments are conducted to compare the performance of 16 state-of-the-art MIL methods on selected problem characteristics. This paper provides insight on how the problem characteristics affect MIL algorithms, recommendations for future benchmarking and promising avenues for research. Code is available on-line at https://github.com/macarbonneau/MILSurvey . [ABSTRACT FROM AUTHOR]

Ensemble methods have shown to be more effective than monolithic classifiers, in particular when diversity holds among their components. How to enforce diversity in classifier ensembles has received much attention from machine learning researchers, yielding a variety of different techniques and algorithms. In this paper, a novel algorithm for ensemble classifiers is proposed, in which ensemble components are trained with focus on different regions of the sample space. In so doing, diversity is mainly a consequence of the intention to limit the scope of base classifiers. The algorithm proposed in this paper shares roots with several ensemble paradigms, in particular with random forests, as it generates forests of decision trees as well. As decision trees are trained with focus on specific subsets of the sample space, the resulting ensemble is in fact a forest of “local” trees. Comparative experimental results highlight that, on average, these ensembles perform better than other relevant kinds of ensemble classifiers, including random forests. [ABSTRACT FROM AUTHOR]

This paper investigates algorithms for matrix factorization when some or many components are missing, a problem that arises frequently in computer vision and pattern recognition. We demonstrate that the Jacobian used in the damped Wiberg (DW) method is exactly the same as that of Kaufman's simplified variable projection (VP) algorithm. Our analysis provides a novel perspective on the efficiency of VP algorithms by improving the strong convexity of the approximate function. To enhance numerical stability, we set a lower bound on the damping parameter instead of adding a null space like the DW algorithm. Another challenge of low-rank matrix decomposition with missing data is the existence of many sharp local minima, which are often distributed in narrow valleys of the landscape of objection functions. Falling into such minima tends to result in poor reconstruction results. To address this issue, we design a non-monotonic VP algorithm, which can facilitate the algorithm to escape from sharp minima and converge to flatter minima. Numerical experiments confirm the effectiveness and efficiency of the proposed nonmonotone VP algorithm. • We provide a new perspective to examine the superiority of the VP algorithm from a convexity point of view. • We propose a nonmonotone VP algorithm, which can facilitate the algorithm to escape from sharp minima. • We conduct extensive experiments to validate the effectiveness of the proposed nonmonotone algorithm. [ABSTRACT FROM AUTHOR]

Wide-angle video frames typically contain more information, but they also exhibit distortion that degrades the visual quality, especially at the edges. To eliminate this distortion from videos, we propose a self-supervised iterative optimization method in this paper. Specifically, we construct a motion parameter estimation model utilizing two consecutive distorted frames, where motion parameters comprise affine transform and distortion parameters. We apply the Gauss–Newton algorithm to minimize the sum-of-squares error between frames and update parameters. Treating inter-frame motion as undistort-affine-distort transformations, frame alignment is achieved by continuously adjusting transform parameters. Ultimately, frames are corrected using the converged parameters. We generated a synthetic dataset with various distortion parameters for evaluation. Experiments demonstrate superior performance versus state-of-the-art methods on synthetic and real wide-angle videos. Our algorithm also achieves higher parameter estimation accuracy without sacrificing efficiency. • proposes a algorithm for video distortion correction based on iterative optimization. • the distort transform and undistort transform are derived separately and the range of parameters and the experimental error are given. • Based on this, we transform the distortion correction into a parameter estimation problem in image alignment. • optimize the parameter estimation problem using Gauss-Newton algorithm. • Experimental results show that the proposed algorithms outperform some state-of-art algorithms in accuracy, without reducing robustness and efficiency. [ABSTRACT FROM AUTHOR]

In this paper, we investigate L 0 -regularized logistic regression models, and design two fast and efficient algorithms for high-dimensional correlated data and massive data, respectively. Our first algorithm, the Variable Sorted Active Set (VSAS) algorithm, is based on the local quadratic approximation of the KKT conditions for L 0 -penalized maximum log-likelihood function in high-dimensional correlated data. We establish an L ∞ error upper bound for the estimator obtained by the VSAS algorithm and prove its optimal convergence rate. Moreover, when the target signal exceeds the detectable level, the estimator obtained by the VSAS algorithm can achieve the oracle estimator with high probability. Our second algorithm, Communication Effective Variable Sorted Active Set (CEVSAS), aims to solve high-dimensional and large-sample L 0 -regularized logistic regression models by reduce computational and communication costs, while maintaining estimation efficiency. Finally, simulations and real data demonstrate the effectiveness of our proposed VSAS and CEVSAS algorithms. • We develop a fast VSAS algorithm for L 0 regularized logistic regression involving high-dimensional correlated data. • An L ∞ upper bound is established on the estimation error of the solution sequence generated by VSAS algorithm. • The CEVSAS altorithm is proposed to reduce computational costs and ensure higher estimation efficiency. • We propose two adaptive versions of the VSAS and CEVSAS algorithms for selecting the optimal tuning parameters. [ABSTRACT FROM AUTHOR]

The term “iris” refers to the highly textured annular portion of the human eye that is externally visible. An iris recognition system exploits the richness of these textural patterns to distinguish individuals. Iris recognition systems are being used in a number of human recognition applications such as access control, national ID schemes, border control, etc. To capture the rich textural information of the iris pattern regardless of the eye color, traditional iris recognition systems utilize near-infrared (NIR) sensors to acquire images of the iris. This, however, restricts the iris image acquisition distance to close quarters (less than 1 m). Over the last several years, there have been numerous attempts to design and implement iris recognition systems that operate at longer standoff distances ranging from 1 m to 60 m. Such long range iris acquisition and recognition systems can provide high user convenience and improved throughput. This paper reviews the state-of-the-art design and implementation of iris-recognition-at-a-distance (IAAD) systems. In this regard, the design of such a system from both the image acquisition (hardware) and image processing (algorithms) perspectives are presented. The major contributions of this paper include: (1) discussing the significance and applications of IAAD systems in the context of human recognition, (2) providing a review of existing IAAD systems, (3) presenting a complete solution to the design problem of an IAAD system, from both hardware and algorithmic perspectives, (4) discussing the use of additional ocular information, along with iris, for improving IAAD accuracy, and (5) discussing the current research challenges and providing recommendations for future research in IAAD. [ABSTRACT FROM AUTHOR]

• The establishment of a new mathematical model to realize dense light field reconstruction. • The innovative application of optical flow method. • Simple algorithm structure and relatively easy to implement. • Algorithm strong adaptability, good dense reconstruction was completed. Dense light field sampling is an important basis for refocusing, depth estimation and 3-D imaging. It is difficult to obtain high resolution dense light field with a large-scale camera array and expensive equipment. At the same time, the current storage devices and transmission bandwidth also limit this technology's post-processing and application. In order to effectively reconstruct the angle domain of the light field based on the sparse light field data, this paper analyzes the correlation and constraint relationship between the optical flow field and the motion field of multi-view images in the same scene, extends the traditional optical flow constraint equation of two-dimensional imaging to the optical flow constraint equation of four-dimensional light field, and establishes an effective mathematical model. The coordinate position of the original pixel in the new angle image is determined by coordinate search, and its intensity is reconstructed. The experimental results of multi-scene dense reconstruction show that the proposed method can reconstruct the texture, shadow and color information in the light field of a long-baseline scene with high quality. The quantitative evaluation results show that the algorithm can be applied to dense light field reconstruction of complex scenes. The algorithm in this paper is only suitable for the case of optical flow constraint in a linear light field, and the follow-up research will focus on the case of nonlinear optical flow constraint. [ABSTRACT FROM AUTHOR]

In this paper, a robust method is presented to perform word spotting in degraded handwritten and printed document images. A new sequence matching technique, called the Flexible Sequence Matching (FSM) algorithm, is introduced for this word spotting task. The FSM algorithm was specially designed to incorporate crucial characteristics of other sequence matching algorithms (especially Dynamic Time Warping (DTW), Subsequence DTW (SSDTW), Minimal Variance Matching (MVM) and Continuous Dynamic Programming (CDP)). Along with the characteristics of multiple matching (many-to-one and one-to-many), FSM is strongly capable of skipping existing outliers or noisy elements, regardless of their positions in the target signal. More precisely, in the domain of word spotting, FSM has the ability to retrieve complete words or words that contain only a part of the query. Furthermore, due to its adaptable skipping capability, FSM is less sensitive to local variation in the spelling of words and to local degradation effects within the word image. The multiple matching capability (many-to-one, one-to-many) of FSM helps it addressing the stretching effects of query and/or target images. Moreover, FSM is designed in such a way that with little modification, its architecture can be changed into the architecture of DTW, MVM, and SSDTW and to CDP-like techniques. To illustrate these possibilities for FSM applied to specific cases of word spotting, such as incorrect word segmentation and word-level local variations, we performed experiments on historical handwritten documents and also on historical printed document images. To demonstrate the capabilities of sub-sequence matching, of noise skipping, as well as the ability to work in a multilingual paradigm with local spelling variations, we have considered properly segmented lines of historical handwritten documents in different languages and improperly as well as properly segmented words in printed and handwritten historical documents. From the comparative experimental results shown in this paper, it can be clearly seen that FSM can be equivalent or better than most DTW-based word spotting techniques in the literature while providing at the same time more meaningful correspondences between elements. [ABSTRACT FROM AUTHOR]

Ensemble pruning is an important task in supervised learning because of the performance and efficiency advantage it begets to predictive modelling. Performance based empirical comparison (primarily on accuracy) is the most common modus operandi for critical evaluation of ensembles pruned by different algorithms. Deep analysis of existing literature reveals that ensemble size is an ignored attribute while judging the quality of ensembles. In this paper, we argue that the cost-effectiveness of an ensemble is a function of both performance and size. Hence, equitable comparison of two ensembles must take into account both these attributes to judge their relative merits. Following this argument, we propose an objective function called accrual function which quantifies the difference in performance and size of two ensembles, to gauge their relative cost-effectiveness. The function can be parameterized and has nice mathematical properties. Semantic interpretations of these properties are delineated in the paper. Finally, we apply the accrual function on published results from selected publications and demonstrate its ability to beget clarity while comparing ensembles. [ABSTRACT FROM AUTHOR]

• The first attempt to design a GEP-based codematrix generation algorithm. The individual structure of GEP allows the codematrix to be optimized in a much faster and more effective way. • The feature selection scheme is combined with the design of codematrix, to accomplish the feature selection and codematrix optimization simultaneously. • A greedy strategy-based adaptive optimization method to further improve the individual's quality to deal with hard classes by appending proper columns to the codematrix. The multi-class classification task is one of the most common tasks in machine learning. As a typical solution based on a partitioning strategy, Error-Correcting Output Codes (ECOC) can transform a multi-class classification problem into multiple binary classification problems. The key of ECOC is to construct an effective codematrix to represent a set of class decomposition schemes, which transforms a multiclass problem into a group of binary class problems. Consequently, the design of a fast and effective ECOC codematrix generation method is of great research significance and value for solving multi-class classification problems. In ECOC algorithms, the design of codematrix is treated as a combination problem between different code columns, in which the evolutionary algorithm shows a great advantage. Based on this consideration, the Gene Expression Programming (GEP) is applied to search for the codematrix with high performance because its expressive tree structure makes it well represent codematrcies for subsequent optimization operations. This paper proposes an adaptive ECOC algorithm based on Gene Expression Programming (GEP) and similarity measurement matrix, named GEP-ECOC. In our GEP, each individual represents a set of columns to form a random ECOC codematrix, which is optimized in the evolutionary process. Meanwhile, the crossover and mutation operations are modified to include a legality checking process to ensure that the generated codematrix satisfies the ECOC constraints. The GEP-based ECOC codematrix generation algorithm can quickly produce a codematrix with better performance, which ensures the efficiency of the algorithm to a certain extent. In addition, an adaptive algorithm based on a similarity measurement matrix is proposed to add new columns to the current codematrix, aiming to better handle hard classes. Our algorithm is compared with other algorithms on various data sets, and the experimental results confirm that our GEP-ECOC can balance the efficiency and performance of the algorithm and achieve higher performance. [ABSTRACT FROM AUTHOR]

Class imbalance problem commonly exists in multi-label classification (MLC) tasks. It has non-negligible impacts on the classifier performance and has drawn extensive attention in recent years. Borderline oversampling has been widely used in single-label learning as a competitive technique in dealing with class imbalance. Nevertheless, the borderline samples in multi-label data sets (MLDs) have not been studied. Hence, this paper deeply discussed the borderline samples in MLDs and found they have different neighboring relationships with class borders, which makes their roles different in the classifier training. For that, they are divided into two types named the self-borderline samples and the cross-borderline samples. Further, a novel MLDs resampling approach called Multi-Label Borderline Oversampling Technique (MLBOTE) is proposed for multi-label imbalanced learning. MLBOTE identifies three types of seed samples, including interior, self-borderline, and cross-borderline samples, and different oversampling mechanisms are designed for them, respectively. Meanwhile, it regards not only the minority classes but also the classes suffering from one-vs-rest imbalance as those in need of oversampling. Experiments on eight data sets with nine MLC algorithms and three base classifiers are carried out to compare MLBOTE with some state-of-art MLDs resampling techniques. The results show MLBOTE outperforms other methods in various scenarios. • A new borderline oversampling technique for multi-label imbalanced learning. • Defining self-borderline and cross-borderline samples in multi-label data sets. • Handling one-vs-rest imbalance in multi-label imbalanced learning. • Performing competitively in the experiments with C4.5, RBF kernel SVM and linear SVM. [ABSTRACT FROM AUTHOR]

Channel pruning is broadly recognized as an effective approach to obtain a small compact model through eliminating unimportant channels from a large cumbersome network. Contemporary methods typically perform iterative pruning procedure from the original over-parameterized model, which is both tedious and expensive especially when the pruning is aggressive. In this paper, we propose a simple yet effective channel pruning technique, termed network Pruning via rEsource rEalLocation (PEEL), to quickly produce a desired slim model with negligible cost. Specifically, PEEL first constructs a predefined backbone and then conducts resource reallocation on it to shift parameters from less informative layers to more important layers in one round, thus amplifying the positive effect of these informative layers. To demonstrate the effectiveness of PEEL , we perform extensive experiments on ImageNet with ResNet-18, ResNet-50, MobileNetV2, MobileNetV3-small and EfficientNet-B0. Experimental results show that structures uncovered by PEEL exhibit competitive performance with state-of-the-art pruning algorithms under various pruning settings. Encouraging results are also observed when applying PEEL to compress the semantic segmentation model. Our code is available at https://github.com/cardwing/Codes-for-PEEL. • Previous methods typically perform iterative pruning and are tedious and expensive. • A simple yet effective method is proposed to quickly produce a slim model with negligible cost. • Our method constructs a predefined backbone and conducts resource reallocation on it. • Resource reallocation shifts parameters from less informative layers to more important ones in one round. • Extensive experiments on ImageNet with various models validate the effectiveness. [ABSTRACT FROM AUTHOR]

In this paper, a patch based change detection method for polarimetric synthetic aperture radar (PolSAR) data is proposed. Because of the inherent speckle interference in SAR system, a despeckling procedure is important for change detection. However, the residual speckle still exists in the filtered data and is unevenly distributed, which is not considered in the available change detectors. Besides that, the detection methods with despeckling procedure cannot be adjusted based on the constant false alarm rate (CFAR). Specifically, the available PolSAR change detection methods are inefficient for the strong speckle interference cases due to the absence of the despeckling procedure. In order to overcome these drawbacks, an improved change detection framework for PolSAR data is proposed in this paper. The main innovations in the new framework include: (1) a despeckling procedure is added for PolSAR change detection, which makes the algorithm robust to strong noise; (2) a new equivalent number of looks (ENL) estimator is proposed for the PolSAR/SAR data filtered by non-local means filter; (3) the proposed method which detects the changes based on the filtered data can be adjusted with different CFARs. The experiments on both synthetic and real PolSAR datasets have demonstrated that the effectiveness of the proposed patch based PolSAR change detector (PPCD). [ABSTRACT FROM AUTHOR]

Abstract: Verification of identity based on the analysis of dynamic signatures is an important problem of biometrics. The effectiveness of the verification significantly increases when the dynamic characteristics of signature (e.g. velocity of the pen) are considered. These characteristics are individual for each user and difficult to imitate. The effectiveness of the verification can be further improved by using partitioning. In this paper we propose a new method which uses partitioning. In our method partitions represent areas of high and low speed of signature and high and low pen׳s pressure. All selected partitions are used by our algorithm, but more important in the classification process are these partitions, in which the signatures of the user acquired during training phase are more stable. Moreover, final classification is interpretable. In this paper we present the simulation results of the proposed method for the two databases: SVC2004 and BioSecure Database. [Copyright &y& Elsevier]

Highlights • A generalized 2D multi-level thresholding criterion function is proved rigorously by mathematical induction method. • A multi-level thresholding scheme for a RGB color image is proposed. • PSO algorithm is applied to seek to optimal threshold values in a very reasonable computational time. • The segmented image is compared with the human segmentation from BSDS300 to evaluate the experiment results quantitatively and objectively. Abstract In this paper, we propose a multi-level thresholding model based on gray-level & local-average histogram (GLLA) and Tsallis–Havrda–Charvát entropy for RGB color image. We validate the multi-level thresholding formulation by using the mathematical induction method. We apply particle swarm optimization (PSO) algorithm to obtain the optimal threshold values for each component of a RGB image. By assigning the mean values from each thresholded class, we obtain three segmented component images independently. We conduct the experiments extensively on The Berkeley Segmentation Dataset and Benchmark (BSDS300) and calculate the average four performance indices (BDE, PRI, GCE and VOI) to show the effectiveness and reasonability of the proposed method. [ABSTRACT FROM AUTHOR]

• For multi-view input, we propose the global-aware fusion to adapt the global state when fusing features from different views. • Our algorithm includes a powerful but relatively lightweight network, novel loss function, and specified training strategy. • The reconstructor exceeds existing SOTA methods while the amount of parameters is far less than the same type of algorithm. • An extra view-reduction method based on maximizing diversity for our model can be used to tradeoff cost and performance. Deep learning technology has made great progress in multi-view 3D reconstruction tasks. At present, the mainstream solutions adopt different ways to fusion the features from several views. Among them, attention-based aggregation function performs relatively well and stably, however, it still has an obvious shortcoming the strong independence of each view during predicting the weights for merging leads to a lack of adaption of the global state. In this paper, we propose a global-aware attention-based fusion approach that builds the correlation between each branch and the global feature to provide a comprehensive foundation for weights inference. On the basis of this, we design a complete reconstruction algorithm. Experiments on ShapeNet verify that our method outperforms existing SOTA methods. Furthermore, we propose a view-reduction method based on maximizing diversity and discuss the cost-performance tradeoff of our model to achieve a better performance when facing a heavy input amount and limited computational cost. [ABSTRACT FROM AUTHOR]

• A novel rank-based approach is proposed for regression tasks; • The method allows to combine predictions from a base regressor and neighborhood information; • Re-ranking strategies and weighted functions are used for better exploiting neighborhood information; • Several experiments considering various datasets, base regressors and re-ranking methods; • Significant decrease of prediction errors considering traditional and state-of-the-art regressors. Several approaches based on regression have been developed in the past few years with the goal of improving prediction results, including the use of ranking strategies. Re-ranking has been exploited and successfully employed in several applications, improving rankings by encoding the manifold structure and redefining distances among elements from a dataset. Despite the promising results observed, re-ranking has not been evaluated in regressions tasks. This paper proposes a novel, generic, and customizable framework entitled Regression by Re-ranking (RbR) , which explores the ability of re-ranking algorithms in determining relevant rankings of objects in prediction tasks. The framework relies on the integration of a base regressor, unsupervised re-ranking learning techniques, and predictions associated with nearest neighbours weighted according to their ranking positions. The RbR framework was evaluated under a rigorous experimental protocol and presented significant results in improving the prediction when compared to state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• Human activities recognition (HAR) and human intent recognition (HIR) are important for medical diagnosis and human-robot interaction. HAR and HIR usually rely on the signals of some wearable sensors, such as inertial measurement unit (IMU), but these signals may be user-dependent, which degrades the performance of the recognition algorithm on new subjects. Traditional supervised learning methods require labeling signals and training specific classifiers for each new subject, which is burdensome. To deal with this problem, this paper proposes a novel non-adversarial cross-subject adaptation method called Gaussian-guided feature alignment (GFA). The proposed GFA metric quantifies the discrepancy between the labeled features of source subjects and the unlabeled features of target subjects so that minimizing the GFA metric leads to the alignment of the source and target features. The GFA metric is estimated by calculating the divergence between the feature distribution and Gaussian distribution, as well as the mean squared error of the mean and variance between source and target features. This paper analytically proves the effect of the GFA metric and validates its performance using three public human activity datasets. Experimental results show that the proposed GFA achieves 1% higher target classification accuracy and 0.5% lower variance than state-of-the-art methods in case of cross-subject validation. These results indicate that the proposed GFA is feasible for improving the generalization of the HAR and HIR. • The key contributions of the present paper include: The cross-subject validation problems are resolved by proposing a novel Gaussian-guided Feature Alignment (GFA) metric to extract the features of the two domains in a Gaussian distribution space without projecting the features to another vector space, e.g., Hilbert space. A better alignment is achieved by adjusting the mean and variance of the Gaussian distribution based on the mean and variance of the corresponding feature distribution and sorting the data sampled from the feature and Gaussian distributions. The effect of the GFA metric on aligning the target distribution with the source distribution and decreasing the upper bound of the target error is analytically proved. Moreover, the feature extractor is optimized to decrease the variance of the target classification accuracy on the training set to further improve the performance of the cross-subject adaptation. The experimental results show that the proposed method outperforms state-of-the-art methods in the case of the cross-subject validation on one motion-sensor-based HIR dataset, one motionsensor- based HAR dataset, and one skeleton-based HAR dataset. Human activities recognition (HAR) and human intent recognition (HIR) are important for medical diagnosis and human-robot interaction. HAR and HIR usually rely on the signals of some wearable sensors, such as inertial measurement unit (IMU), but these signals may be user-dependent, which degrades the performance of the recognition algorithm on new subjects. Traditional supervised learning methods require labeling signals and training specific classifiers for each new subject, which is burdensome. To deal with this problem, this paper proposes a novel non-adversarial cross-subject adaptation method called Gaussian-guided feature alignment (GFA). The proposed GFA metric quantifies the discrepancy between the labeled features of source subjects and the unlabeled features of target subjects so that minimizing the GFA metric leads to the alignment of the source and target features. The GFA metric is estimated by calculating the divergence between the feature distribution and Gaussian distribution, as well as the mean squared error of the mean and variance between source and target features. This paper analytically proves the effect of the GFA metric and validates its performance using three public human activity datasets. Experimental results show that the proposed GFA achieves 1% higher target classification accuracy and 0.5% lower variance than state-of-the-art methods in case of cross-subject validation. These results indicate that the proposed GFA is feasible for improving the generalization of the HAR and HIR. [ABSTRACT FROM AUTHOR]

• A directed convolutional neural network model is proposed • Partial dilation-Global erosion generates resistance against information loss • Several recent approaches for the segmentation of graphemes/characters are discussed • The over-traces issue is addressed for the first time in the literature • MMOs can provide real solutions if a convenient strategy is adopted Due to the nature of Arabic handwriting, segmenting words into characters/graphemes is the most difficult and critical task of the recognition system. The present paper proposes an approach to segment handwritten Arabic words into graphemes based on a directed Convolutional Neural Network (CNN) and Mathematical Morphology Operations (MMO). Arabic script is cursive, which means that almost all graphemes are connected via horizontal links; therefore, a technique to remove links will facilitate the segmentation of graphemes. In general, an MMO such as erosion seems suitable for getting the job done, but since Arabic handwriting is difficult, MMOs cause information loss and suffer from many issues such as diacritics and over-traces, which lead to over/under/bad segmentations. To overcome limitations, the present paper addresses these issues in the following order: the over-traces issue is addressed for the first time in the literature; a robust algorithm for diacritics extraction is provided; and finally, the main segmentation algorithm adopts a strategy based on a Partial Dilation (PD)-Global Erosion (GE) technique to combat the information loss issue. The PD phase amplifies important regions, while GE eliminates links between graphemes. The complementarity between PD and GE facilitates the extraction of graphemes and creates resistance against information loss. To properly tackle these difficult problems, this article exploits the robustness of CNNs, so a new directed CNN model is suggested. The idea is to draw the model's attention to certain targeted features, which are selected according to the nature of the problem addressed. The proposed directed CNN is used in all phases of the segmentation process. The experimental results are very encouraging and show that the proposed directed CNN model outperformed basic CNN in many experiments. The results also reveal that the followed strategy improved the ability of MMOs to perform segmentation and to compete with other approaches in this research area. [ABSTRACT FROM AUTHOR]

• In this paper, an object-based clustering validity index with densities referred to as OCVD is proposed. This index uses densities of clusters to capture the exclusive contribution of each data object in both separation and compactness of clusters. • OCVD is superior to many existing clustering validity indices that capture the properties of clusters by using representative statistics such as mean, variance, diameter, etc. The reason is that those indices might not perform well in capturing the whole characteristics of clusters with arbitrary shapes but OCVD is well capable of doing so. • Although there are some existing density-based validity indices, studies show that they have problems such as poor performance on clusters with arbitrary shapes which are not necessarily perfectly separated and poor performance due to relying only on some representative data objects in clusters. Clustering validity indices are typically used as tools to find the correct number of clusters in a data set and/or to evaluate the quality of the clusters formed by clustering algorithms. Clustering validity indices measure separation and compactness of clusters. Typically, when applying a clustering algorithm, the input includes the number of clusters. After applying the algorithm with several different numbers of clusters, we determine the number of clusters to be the one with the best validity index. There are two types of clustering validity indices: external indices that are supervised, and internal indices that are unsupervised. The focus of this paper is on internal validity indices. Some existing internal validity indices capture the properties of the clusters by using representative statistics such as mean, variance, diameter, etc., however, these do not perform well when clusters have arbitrary shapes. One approach to overcome this issue is to use the density of the data objects in each cluster. That provides the advantage of capturing the full characteristics of the cluster which is most beneficial when there are clusters with arbitrary shapes. In the literature, a few density-based clustering validity indices have been proposed. However, some of them show poor performance when the clusters are not perfectly separated. Some others perform poorly because they use only representative objects from each cluster instead of all objects. The contribution of this paper is an internal validity index named the object-based clustering validity index with densities (OCVD). OCVD is a single number that averages the density-based contribution of individual data objects to both separation and compactness of clusters. The methodology behind calculating the density-based contributions of the objects is kernel density estimation. We show through several experiments that OCVD performs well in detecting the correct number of clusters in data sets with different cluster shapes including arbitrary shapes. [ABSTRACT FROM AUTHOR]

• Convolutional Neural Networks can be improved in terms of the classification performance and robustness by using variable weight structures. • Analysis of different data processing methods, models' robustness and statistical properties. • Comparative analysis of variable weight convolutional neural networks and other widely used machine learning techniques. • Medical applications to the classification of seizure phases and types. Deep learning techniques have recently achieved impressive results and raised expectations in the domains of medical diagnosis and physiological signal processing. The widely adopted methods include convolutional neural networks (CNNs) and recurrent neural networks (RNNs). However, the existing models possess static connection weights between layers, which might limit the generalization capability and the classification performance of the models as the weights of different layers are fixed after training. Furthermore, to deal with a large amount of data, a neural network with a sufficiently large size is required. This paper proposes the variable weight convolutional neural networks (VWCNNs), which are a type of network structure employing dynamic weights instead of static weights in their convolutional layers and fully-connected layers. VWCNNs are able to adapt to different characteristics of input data and can be viewed as an infinite number of traditional, fixed-weight CNNs. We will show that the proposed VWCNN structure outperforms the conventional CNN in terms of the classification accuracy, generalization capability, and robustness when the inputs are contaminated by noise. In this paper, VWCNNs are applied to the classification of three seizure phases (seizure-free, pre-seizure and seizure) based on measured electroencephalography (EEG) data. VWCNNs achieve 100% test accuracy and show strong robustness in the classification of the three seizure phases, and thus show the potential to be a useful classification tool for medical diagnosis. Furthermore, the classification of seven types of seizures is investigated in this paper using the world's largest open source database of seizure recordings, TUH EEG seizure corpus. Comparisons with conventional CNNs, RNN, MobileNet, ResNet, DenseNet and traditional machine learning methods including random forest, decision tree, support vector machine, K-nearest neighbours, standard neural networks, and Naïve Bayes are being conducted using realistic test data sets. The results demonstrate that VWCNNs have advantages over other classifiers in terms of classification accuracy and robustness. [ABSTRACT FROM AUTHOR]

The paper presents numerical experiments and some theoretical developments in prediction with expert advice (PEA). One experiment deals with predicting electricity consumption depending on temperature and uses real data. As the pattern of dependence can change with season and time of the day, the domain naturally admits PEA formulation with experts having different "areas of expertise". We consider the case where several competing methods produce online predictions in the form of probability distribution functions. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). A popular example of scoring rule for continuous outcomes is Continuous Ranked Probability Score (CRPS). In this paper the problem of combining probabilistic forecasts is considered in the PEA framework. We show that CRPS is a mixable loss function and then the time-independent upper bound for the regret of the Vovk aggregating algorithm using CRPS as a loss function can be obtained. Also, we incorporate a "smooth" version of the method of specialized experts in this scheme which allows us to combine the probabilistic predictions of the specialized experts with overlapping domains of their competence. [ABSTRACT FROM AUTHOR]

• In the paper we propose a novel approach for solving puzzles whose pieces are of general shape, which need not necessarily match precisely due to erosion, and which have degraded unevenly over time. • We define the notion of valid transformations and propose a method of sampling them. This method may find uses in other domains. • We define a novel dissimilarity score that takes into account the special characteristics of the domain. • We introduce the notion of confidence in the dissimilarity scores, which depends not only on the value of the dissimilarity, but also on those of the competing matches. Confidence may be used not only in re-assembly, but also in other applications that are based on dissimilarity. This paper focuses on the re-assembly of an archaeological artifact, given images of its fragments. This problem can be considered as a special challenging case of puzzle solving. The restricted case of re-assembly of a natural image from square pieces has been investigated extensively and was shown to be a difficult problem in its own right. Likewise, the case of matching "clean" 2D polygons/splines based solely on their geometric properties has been studied. But what if these ideal conditions do not hold? This is the problem addressed in the paper. Three unique characteristics of archaeological fragments make puzzle solving extremely difficult: (1) The fragments are of general shape; (2) They are abraded, especially at the boundaries (where the strongest cues for matching should exist); and (3) The domain of valid transformations between the pieces is continuous. The key contribution of this paper is a fully-automatic and general algorithm that addresses puzzle solving in this intriguing domain. We show that our approach manages to correctly reassemble dozens of broken artifacts and frescoes. [ABSTRACT FROM AUTHOR]

The paper is devoted to two problems: (1) reinforcement of SVM algorithms, and (2) justification of memorization mechanisms for generalization. (1) Current SVM algorithm was designed for the case when the risk for the set of nonnegative slack variables is defined by l 1 norm. In this paper, along with that classical l 1 norm, we consider risks defined by l 2 norm and l ∞ norm. Using these norms, we formulate several modifications of the existing SVM algorithm and show that the resulting modified SVM algorithms can improve (sometimes significantly) the classification performance. (2) Generalization ability of existing learning algorithms is usually explained by arguments involving uniform convergence of empirical losses to the corresponding expected losses over a given set of functions. However, along with bounds for uniform convergence of empirical losses to the expected losses, the VC theory also provides bounds for relative uniform convergence. These bounds lead to a more accurate estimate of the expected loss. Advanced methods of estimating of expected risk of error have to leverage these bounds, which also support mechanisms of training data memorization, which, as the paper demonstrates, can improve classification performance. [ABSTRACT FROM AUTHOR]

Highlights • Always considered graph edit distance (GED) is a metric if edit functions are a metric. • We discern between GED computed through edit path and graph bijection. • Triangle inequality of edit functions not necessary if GED defined by graph bijection. • Important: usually recognition ratio is maximized in non-metric edit functions. Abstract In the presentation of the graph edit distance in 1983 and other newer bibliography, authors state that it is necessary to apply the distance restrictions (non-negativity, identity of indiscernible elements, symmetry and triangle inequality) to each of the edit functions (insertion, deletion and substitution of nodes and edges) involved in the process of computing the graph edit distance to make the graph edit distance a true distance. Moreover, graph edit distance algorithms presented in the last three decades have been based on mapping the edit path that transforms a graph into the other one into a bijection of the graphs in which some null nodes have been added. In this paper, we show that the triangle inequality does not need to be imposed in each edit function if the graph edit distance is defined through an edit path; however, it is necessary if it is defined as a graph bijection. This is an important finding since the triangle inequality is the only restriction that relates different edit functions and concerns the process of tuning the edit functions given a specific application. Hence, on one hand, it would encourage research to define new algorithms based on the edit path instead of the graph bijection and, on the other hand, use edit functions without the restriction, for instance, that the sum of the costs of insertion and deletion of a pair of nodes has to be larger or equal than the cost of substituting them, which could increase the recognition ratio of a concrete application. [ABSTRACT FROM AUTHOR]

Highlights • The proposed method extends grid-based DBSCAN scalable to higher-dimensional dataset. • Cluster forest is devised to alleviate redundancies in the merging step. • HyperGrid Bitmap is used to index non-empty grids for efficient neighbor grid queries. • Experiments show performance superiority of proposed method on real/synthetic data. Abstract DBSCAN is one of clustering algorithms which can report arbitrarily-shaped clusters and noises without requiring the number of clusters as a parameter (unlike the other clustering algorithms, k -means, for example). Because the running time of DBSCAN has quadratic order of growth, i.e. O (n 2), research studies on improving its performance have been received a considerable amount of attention for decades. Grid-based DBSCAN is a well-developed algorithm whose complexity is improved to O (n log n) in 2D space, while requiring Ω(n 4/3) to solve when dimension ≥ 3. However, we find that Grid-based DBSCAN suffers from two problems: neighbour explosion and redundancies in merging, which make the algorithms infeasible in high dimensional space. In this paper we first propose a novel algorithm called GDCF which utilizes bitmap indexing to support efficient neighbour grid queries. Second, based on the concept of union-find algorithm we devise a forest-like structure, called cluster forest, to alleviate the redundancies in the merging. Moreover, we find that running the cluster forest in different orders can lead to a different number of merging operations needed to perform in the merging step. We propose to perform the merging step in a uniform random order to optimize the number of merging operations. However, for high-density database, a bottleneck could be occurred, we further propose a low-density-first order to alleviate this bottleneck. The experiments resulted on both real-world and synthetic datasets demonstrate that the proposed algorithm outperforms the state-of-the-art exact/approximate DBSCAN and suggests a good scalability. [ABSTRACT FROM AUTHOR]

Highlights • Unlike common multi-label feature selection, the proposed method derives from different cognitive standpoint. • Feature selection process is directly done on multi-label data, and there is no need to data transformation. • The proposed method tries to find label-specific features which are the most discriminative features for each label. • Also, an extension of our method is presented which selects a pre-defined number of features. • The proposed method is appropriate to both numerical and nominal features. • The proposed method is effective and fast. Abstract In multi-label data, each instance is associated with a set of labels, instead of one label. Similar to single-label data, feature selection plays an important role in improving classification performance. In multi-label classification, each class label might be specified by some particular characteristics of its own which are called label-specific features. In this paper, a fast accurate filter-based feature selection method is exclusively designed for multi-label datasets to find label-specific features. It maps the features to a multi-dimensional space based on a filter method, and selects the most salient features with the help of Pareto-dominance concepts from multi-objective optimization domain. Our proposed method can be used as online feature selection that deals with problems in which features arrive sequentially while the number of data samples is fixed. In this method, the number of features to be selected is specified during the process of feature selection. However, sometimes it is desired to predefine the number of features. For this reason, an extension of the proposed method is presented to solve this problem. To prove the performance of the proposed methods, several experiments are conducted on some multi-label datasets and the results are compared to five well-established multi-label feature selection methods. The results show the superiority of the proposed methods in terms of different multi-label classification criteria and execution time. [ABSTRACT FROM AUTHOR]

Highlights • Almost all existing attribute weighting approaches to naive Bayes are class-independent. • We propose a new class-specific attribute weighting paradigm for naive Bayes. • The resulting model is called class-specific attribute weighted naive Bayes (CAWNB). • To learn CAWNB, we propose two gradient-based learning algorithms. • The experimental results validate the effectiveness of the proposed algorithms. Abstract Due to its easiness to construct and interpret, along with its good performance, naive Bayes (NB) is widely used to address classification problems in real-world applications. In order to alleviate its conditional independence assumption, a mass of attribute weighting approaches have been proposed. However, almost all these approaches assign each attribute a same (global) weight for all classes. In this paper, we call them the general attribute weighting and argue that for NB attribute weighting should be class-specific (class-dependent). Based on this premise, we propose a new paradigm for attribute weighting called the class-specific attribute weighting, which discriminatively assigns each attribute a specific weight for each class. We call the resulting model class-specific attribute weighted naive Bayes (CAWNB). CAWNB selects class-specific attribute weights to maximize the conditional log likelihood (CLL) objective function or minimize the mean squared error (MSE) objective function, and thus two different versions are created, which we denote as CAWNBCLL and CAWNBMSE, respectively. Extensive empirical studies show that CAWNBCLL and CAWNBMSE all obtain more satisfactory experimental results compared with NB and other existing state-of-the-art general attribute weighting approaches. We believe that for NB class-specific attribute weighting could be a more fine-grained attribute weighting approach than general attribute weighting. [ABSTRACT FROM AUTHOR]

Highlights • Optimization is an important issue in machine learning because many machine learning models are reformulated as optimization problems. Different kinds of machine learning algorithms mainly focus on minimizing their emiprical loss like deep learning, logisitic regression, and support vector machine. Because data is explosively growing, it is challenging to deal with a large-scale optimization problem. We propose a fast and most condition number free algorithm for train machine learning algorithms. This algorithm has several advantages. • Contrast to previous work which approximates Hessian matrix by sketched or subsampled Hessian directly, we use the sketched or subsampled Hessian matrix as preconditioner and get the direction vector by preconditioned conjugate gradient method. Accordingly, we propose a novel stochastic second-order algorithm called Preconditioned Newton Conjugate Gradient with Sketched Hessian (PNCG). This strategy makes our algorithm almost independent on the condition number. • We numerically demonstrate the effectiveness and robustness of our algorithms in widely used machine learning algorithm Ridge Logistic Regression on several real datasets with different condition number. Our algorithms show good performance compared with existing second order methods in all cases, especially when condition number is large. Abstract Optimization is an important issue in machine learning because many machine learning models are reformulated as optimization problems. Different kinds of machine learning algorithms mainly focus on minimizing their empirical loss like deep learning, logistic regression, and support vector machine. Because data is explosively growing, it is challenging to deal with a large-scale optimization problem. Recently, stochastic second-order methods have emerged to attract much attention due to their efficiency in each iteration. These methods show good performance on training machine learning algorithms like logistic regression and support vector machine. However, the computational complexity of existing stochastic second-order methods heavily depends on the condition number of the Hessian. In this paper, we propose a new Newton-like method called Preconditioned Newton Conjugate Gradient with Sketched Hessian (PNCG). The runtime complexity of PNCG is at most logarithmic in the condition number of the Hessian. PNCG exhibits advantages over existing subsampled Newton methods especially when the Hessian matrix in question is ill-conditioned. We also show that our method has good performance on training machine learning algorithm empirically. The results show consistent improvements in computational efficiency. [ABSTRACT FROM AUTHOR]

Highlights • For Alzheimer's disease (AD) diagnosis, we consider the integration of multi-modality imaging and genetic data which encode different level of knowledge. • With a focus on taking advantage of both phenotype and genotype information, a novel structured sparsity, defined by ℓ 1, p -norm (p > 1), regularized multiple kernel learning method is designed. • An efficient block coordinate descent algorithm applicable to any case with p > 1 was derived to solve the proposed formulation. Abstract Multimodal data fusion has shown great advantages in uncovering information that could be overlooked by using single modality. In this paper, we consider the integration of high-dimensional multi-modality imaging and genetic data for Alzheimer's disease (AD) diagnosis. With a focus on taking advantage of both phenotype and genotype information, a novel structured sparsity, defined by ℓ 1, p -norm (p > 1), regularized multiple kernel learning method is designed. Specifically, to facilitate structured feature selection and fusion from heterogeneous modalities and also capture feature-wise importance, we represent each feature with a distinct kernel as a basis, followed by grouping the kernels according to modalities. Then, an optimally combined kernel presentation of multimodal features is learned in a data-driven approach. Contrary to the Group Lasso (i.e., ℓ 2, 1 -norm penalty) which performs sparse group selection, the proposed regularizer enforced on kernel weights is to sparsely select concise feature set within each homogenous group and fuse the heterogeneous feature groups by taking advantage of dense norms. We have evaluated our method using data of subjects from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. The effectiveness of the method is demonstrated by the clearly improved prediction diagnosis and also the discovered brain regions and SNPs relevant to AD. [ABSTRACT FROM AUTHOR]

Highlights • We give theoretical guarantees for the block sparse representation based classifier. • A RBSRC (robust block sparse representation based classifier) framework is given. • We devise an efficient half-quadratic optimization algorithm for RBSRC. • Using RBSRC as a general platform, we propose three robust classifiers. Abstract By exploiting the low-dimensional structure of high-dimensional data, sparse representation based classifiers (SRC) has recently attracted massive attention in pattern recognition. In this paper, we study a natural generalization of SRC, i.e., block sparse representation based classifiers (BSRC), which takes into account the block structure of the dictionary. Our contributions are two-fold: (1) we provide theoretical guarantees for BSRC and theoretically show that BSRC performs perfect classification for any test sample under both cases of independent subspaces and arbitrary subspaces settings; (2) we extend BSRC and propose three robust BSRC methods based on M-estimators originating in robust statistics. This is motivated by the observation that many previous representation based classifiers utilize the mean square error (MSE) criterion as the loss function, which is sensitive to outliers and complicated noises in reality. In contrast, M-estimators has shown much stronger robustness than MSE against gross corruptions. We demonstrate the efficacy of the proposed methods through experiments on both synthetic and real-world databases for block sparse recovery, handwritten digit recognition and robust face recognition. [ABSTRACT FROM AUTHOR]

Highlights • We design a semi-supervised feature representation framework for person Re-Identification which effectively utilizes both labeled and unlabeled training data to learn a discriminative representation so that person images across disjoint camera views can be reliably matched. • A multi-view clustering method is proposed to integrate features from multiple Convolutional Neural Networks for clustering, which can give more accurate label estimation for unlabeled data. • Each of our Convolutional Neural Networks utilizes a siamese network that simultaneously computes the identification loss and verification loss, which simultaneously learns a discriminative Convolutional Neural Network embedding and a similarity metric, and thus improving pedestrian retrieval accuracy. Extensive experiments on large-scale person Re-Id datasets demonstrate the effectiveness of our method. Abstract Person Re-Identification (Re-Id) is a challenging task focusing on identifying the same person among disjoint camera views. A number of deep learning algorithms have been reported for this task in fully-supervised fashion which requires a large amount of labeled training data, while obtaining high quality labels for Re-Id is extremely time consuming. To address this problem, we propose a semi-supervised Re-Id framework by using only a small portion of labeled data and some additional unlabeled samples. This paper approaches the problem by constructing a set of heterogeneous Convolutional Neural Networks (CNNs) fine-tuned using the labeled portion, and then propagating the labels to the unlabeled portion for further fine-tuning the overall system. In this work, label estimation is a key component during the propagation process. We propose a novel multi-view clustering method, which integrates features of multiple heterogeneous CNNs to cluster and generate pseudo labels for unlabeled samples. Then we fine-tune each of the multiple heterogeneous CNNs by minimizing an identification loss and a verification loss simultaneously, using training data with both true labels and pseudo labels. The procedure is iterated until the estimation of pseudo labels no longer changes. Extensive experiments on three large-scale person Re-Id datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract In streaming data classification, most of the existing methods assume that all arrived evolving data are completely labeled. One challenge is that some applications where only small amount of labeled examples are available for training. Incremental semi-supervised learning algorithms have been proposed for regularizing neural networks by incorporating various side information, such as pairwise constraints or user-provided labels. However, it is hard to put them into practice, especially for non-stationary environments due to the effectiveness and parameter sensitivity of such algorithms. In this paper, we propose a novel incremental semi-supervised learning framework on streaming data. Each layer of model is comprised of a generative network, a discriminant structure and the bridge. The generative network uses dynamic feature learning based on autoencoders to learn generative features from streaming data which has been demonstrated its potential in learning latent feature representations. In addition, the discriminant structure regularizes the network construction via building pairwise similarity and dissimilarity constraints. It is also used for facilitating the parameter learning of the generative network. The network and structure are integrated into a joint learning framework and bridged by enforcing the correlation of their parameters, which balances the flexible incorporation of supervision information and numerical tractability for non-stationary environments as well as explores the intrinsic data structure. Moreover, an efficient algorithm is designed to solve the proposed optimization problem and we also give an ensemble method. Particularly, when multiple layers of model are stacked, the performance is significantly boosted. Finally, to validate the effectiveness of the proposed method, extensive experiments are conducted on synthetic and real-life datasets. The experimental results demonstrate that the performance of the proposed algorithms is superior to some state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Abstract Datasets are often collected from different resources or comprised of multiple representations (i.e., views). Multi-view clustering aims to analyze the multi-view data in an unsupervised way. Owing to the efficiency of uncovering the hidden structures of data, graph-based approaches have been investigated widely for various multi-view learning tasks. However, similarity measurement in these methods is challenging since the construction of similarity graph is impacted by several factors such as the scale of data, neighborhood size, choice of similarity metric, noise and outliers. Moreover, nonlinear relationships usually exist in real-world datasets, which have not been considered by most existing methods. In order to address these challenges, a novel model which simultaneously performs multi-view clustering task and learns similarity relationships in kernel spaces is proposed in this paper. The target optimal graph can be directly partitioned into exact c connected components if there are c clusters. Furthermore, our model can assign ideal weight for each view automatically without additional parameters as previous methods do. Since the performance is often sensitive to the input kernel matrix, the proposed model is further extended with multiple kernel learning ability. With the proposed joint model, three subtasks including construct the most accurate similarity graph, automatically allocate optimal weight for each view and find the cluster indicator matrix can be simultaneously accomplished. By this joint learning, each subtask can be mutually enhanced. Experimental results on benchmark datasets demonstrate that our model outperforms other state-of-the-art multi-view clustering algorithms. [ABSTRACT FROM AUTHOR]

Abstract Traditional collaborative representation based classification (CRC) method usually faces the challenge of data uncertainty hence results in poor performance, especially in the presence of appearance variations in pose, expression and illumination. To overcome this issue, this paper presents a CRC-based face classification method by jointly using block weighted LBP and analysis dictionary learning. To this end, we first design a block weighted LBP histogram algorithm to form a set of local histogram-based feature vectors instead of using raw images. By this means we are able to effectively decrease data redundancy and uncertainty derived from image noises and appearance variations. Second, we adopt an analysis dictionary learning model as the projection transform to construct an analysis subspace, in which a new sample is characterized with the improved sparsity of its reconstruction coefficient vector. The crucial role of the analysis dictionary learning method in CRC is revealed by its capacity of the collaborative representation in an analytic coefficient space. Extensive experimental results conducted on a set of well-known face databases demonstrate the merits of the proposed method. [ABSTRACT FROM AUTHOR]

Highlights • We promote an adaptive weighting method based on structure tensor that considers the contribution rate of each spectral band. • We propose a novel method to extract spatial details from an HSI that is more appropriate in terms of the significance of its physics and avoids spectral distortion. • We propose a deep saliency enhancement method that uses CNN to learn the feature representation from input pixels and is trained in an end-to-end manner. • We utilize the NMF algorithm to extract the spatial features provided by the enhanced saliency and spectral features provided by the original HSI and then merge these features to obtain a high-quality HSI. Abstract Limited by the existing imagery hardware and contaminated with noise or shading, spatial deterioration and spectral distortion exist in hyperspectral images (HSIs). Spectral-spatial quality enhancement was seldom addressed in a clear way albeit of first importance in HSI interpretation. In this paper, we present a promising quality enhancement method in a spectral-spatial combination framework for removing unwanted components and enhancing useful features. Our approach, called saliency detection and feature enhancement (SDFE), combines the theory of structure tensor with a deep convolutional neural network (CNN) to solve an HSI quality enhancement problem that has rarely been identified. Considering the different contribution rates of each band, an adaptive weighting method based on the eigenvalues of structure tensor is proposed to fuse the selected key band group. Then, a saliency detection method is presented to extract edge areas and corners. Owning to the success of CNN in visual-based issues, we utilize it to further enhance the saliency and obtain high-quality spatial information. To extract high-quality spectral features, the nonnegative matrix factorization (NMF) algorithm is used to extract spectral information from the original HSI. The experimental result enjoys a fact of identical materials with the similar signatures, which is useful for the subsequent application. Furthermore, our approach has a powerful influence on target detection. [ABSTRACT FROM AUTHOR]

Highlights • A geosensor data forecasting tensor framework (GDFTF) for significant societal events is proposed. • A rank increasing strategy and a sliding window strategy is used to improve the prediction accuracy. • Extensive experimental evaluations illustrate the superiority of our approach compared with the state-of-the-arts. Abstract Geosensor data forecasting has high practical value in government affairs such as prompt response and decision making. However, the spatial correlation across distinct sites and the temporal correlation within each site pose challenges to accurate forecasting. In this paper, a geosensor data forecasting tensor framework for significant societal events is proposed. Specifically, a tensor pattern is used to model the geosensor data, based on which a tensor decomposition algorithm is then developed to estimate future values of geosensor data. The proposed approach not only combines and utilizes the multi-mode correlations, but also well extracts the underlying factors in each mode of tensor and mines the multi-dimensional structures of geosensor data. In addition, a rank increasing strategy is used to determine tensor rank automatically, and a sliding window strategy is used to improve the prediction accuracy. Extensive experimental evaluations illustrate the superiority of our approach compared with the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Highlights • Found and molded the relationship between the high-frequency signal of image carrier and steganalysis accuracy. • Posterior accuracy is proposed to evaluate the real accuracy of steganalysis and it can be used to improve steganalysis performance in the actual environment, because we can find a subset with a high accuracy(such as 100%) from the prediction images which have a relatively low accuracy (such as 80%). • Actually, the relationship between image carrier and steganalysis accuracy can even be used to improve steganography performance significantly. Abstract Steganalysis research is committed to distinguishing the steganography media from the normal one correctly. However, individual differences of carriers disturb the detection inevitably and greatly. Existing methods treat all detection results with the same confidence level, or prior accuracy, which may make the prior accuracy overestimate or underestimate the real result. This paper presents a novel performance evaluation method of steganalysis based on posterior accuracy. Adaptive Convolution Feature (ACF) is calculated by the adaptive convolution, then a quantitative value S based on the ACF is modeled to posterior testify and estimate the detection confidence of the image under test. By clustering of carrier noise, we classify the images which we believe they have a similar confidence of the same cluster. The distribution of ACF from each cluster indicates that the S value works well for confidence evaluation. The experimental results show that the S value can distinguish and identify high-confidence samples from the low-confidence, which will greatly improve the real performance of steganalysis. Besides, it improves the steganalysis accuracy of the whole image set by matching S values between training and prediction samples. [ABSTRACT FROM AUTHOR]

Highlights • Networks and graphs are naturally embedded in Euclidean hyperspheres. • Communicability embedding of networks/graphs reveals clusters in networks. • Nonmetric multidimensional scaling allows visualization of networks in 3D communicability space. • Communicability clusters of papers in a citation network reveal levels of mathematization. • Communicability clusters in a gene-gene network reveal genes that co-participate in cancer and other diseases. Abstract A complex network is a condensed representation of the relational topological framework of a complex system. A main reason for the existence of such networks is the transmission of items through the entities of these complex systems. Here, we consider a communicability function that accounts for the routes through which items flow on networks. Such a function induces a natural embedding of a network in a Euclidean high-dimensional sphere. We use one of the geometric parameters of this embedding, namely the angle between the position vectors of the nodes in the hyperspheres, to extract structural information from networks. First we propose a simple method for visualizing networks by reducing the dimensionality of the communicability space to 3-dimensional spheres. Secondly, we use clustering analysis to cluster the nodes of the networks based on their similarities in terms of their capacity to successfully deliver information through the network. After testing these approaches in benchmark networks and compare them with the most used clustering methods in networks we analyze two real-world examples. In the first, consisting of a citation network, we discover citation groups that reflect the level of mathematics used in their publications. In the second, we discover groups of genes that coparticipate in human diseases, reporting a few genes that coparticipate in cancer and other diseases. Both examples emphasize the potential of the current methodology for the discovery of new patterns in relational data. [ABSTRACT FROM AUTHOR]

Highlights • A Bayesian denoising hashing method is proposed for robust image retrieval. • We impose extra constraints in the variational Bayes framework to learn robust hash code. • Our method can be combined with many traditional hashing algorithms to achieve state-of-the-art performance on real-world datasets. Abstract Learning to hash is one of the most popular techniques in image retrieval, but few work investigates its robustness to noise corrupted images in which the unknown pattern of noise would heavily deteriorate the performance. To deal with this issue, we present in this paper a Bayesian denoising hashing algorithm whose output can be regarded a denoised version of the input hash code. We show that our method essentially seeks to reconstruct a new but more robust hash code by preserving the original input information while imposing extra constraints so as to correct the corrupted bits. We optimized this model in variational Bayes framework which has a closed-form update in each iteration that is more efficient than numerical optimization. Furthermore, our method can be added at the top of any original hashing layer, serving as a post-processing denoising layer with no change to previous training procedure. Experiments on three popular datasets demonstrate that the proposed method yields robust and meaningful hash code, which significantly improves the performance of state-of-the-art hash learning methods on challenging tasks such as large-scale natural image retrieval and retrieval with corrupted images. [ABSTRACT FROM AUTHOR]