Search

This work develops a biologically inspired neural network for contour detection in natural images by combining the nonclassical receptive field modulation mechanism with a deep learning framework. The input image is first convolved with the local feature detectors to produce the classical receptive field responses, and then a corresponding modulatory kernel is constructed for each feature map to model the nonclassical receptive field modulation behaviors. The modulatory effects can activate a larger cortical area and thus allow cortical neurons to integrate a broader range of visual information to recognize complex cases. Additionally, to characterize spatial structures at various scales, a multiresolution technique is used to represent visual field information from fine to coarse. Different scale responses are combined to estimate the contour probability. Our method achieves state-of-the-art results among all biologically inspired contour detection models. This study provides a method for improving visual modeling of contour detection and inspires new ideas for integrating more brain cognitive mechanisms into deep neural networks. [ABSTRACT FROM AUTHOR]

• We propose a joint image restoration and matching method to deal with the problem of blurred image matching. • The approach utilizes distance-weighted sparse representation prior to exploit the correlation between restoration and matching. • The approach applies a coarse to fine strategy to improve the matching accuracy and reduce the search time. Image matching is a classic problem in the field of image processing, aims to locate the unique position of the real-time images and has been widely used in visual-based navigation systems. However, most of the works on image matching simply assume the ideal inputs without considering the degradation of the real world, such as image blur. In the presence of such a situation, the traditional matching methods usually first resort to image restoration and then perform image matching with the restored image. However, by treating the restoration and matching separately, the accuracy of blurred image matching will be reduced by the defective output of the image restoration. In this paper, we propose a joint blurred image restoration and matching method based on distance-weighted sparse representation (JRM-DSR), which utilizes the sparse representation prior to exploit the correlation between restoration and matching. This prior assumes that the blurred image, if correctly restored, can be well represented as a sparse linear combination of the dictionary constructed by the reference image. In order to achieve more accurate matching results to help image restoration, we consider both local and sparse information as well as adopt distance-weighted sparse representation to obtain better representation coefficients. By iterative restoring the input image in pursuit of the sparest representation, our approach can achieve restoration and match simultaneous, and these two tasks can benefit greatly from each other. Moreover, a coarse-to-fine matching strategy is proposed to further improve the matching accuracy and search efficiency. Experiments demonstrate the effectiveness of our method compared with conventional methods. [ABSTRACT FROM AUTHOR]

Highlights • Motion blur kernel size is a significant input parameter of deblurring algorithms. • A regression CNN is proposed to estimate the kernel size of a blurry image. • The kernel size for training is labeled in a relative way. • The kernel size to be estimated can be arbitrary. • The estimated kernel size helps existing deblurring algorithms yielding satisfactory results. Abstract Deblurring is to restore a latent clear image as well as to estimate an underlying blur kernel from a single blurry image. Motion blur kernel size is a significant input parameter of existing deblurring algorithms. Setting the size manually, which is a tedious trial-and-error process, can hardly obtain a satisfactory result directly and effectively. In this paper, we formulate the kernel size estimation as a regression problem and construct a convolutional neural network (CNN) to resolve it. After training the CNN by a large number of simulated blurry images with labels, which are produced by a relative way, we can estimate the kernel sizes of a given blurry image without size limitation. Both quantitative and qualitative results of experiments show that our method can not only estimate the motion blur kernel size accurately but also help to yield better pictures with estimated kernel sizes as input parameters. [ABSTRACT FROM AUTHOR]

Constructing a good graph that can capture intrinsic data structures is critical for graph-based semi-supervised learning methods, which are widely applied for hyperspectral image (HSI) classification with small amount of labeled samples. Among the existing graph construction methods, sparse representation (SR)-based methods have shown impressive performance on semi-supervised HSI classification tasks. However, most SR-based algorithms fail to consider the rich spatial information of HSI, which has been shown beneficial for classification tasks. In this paper, we propose a spatial and class structure regularized sparse representation (SCSSR) graph for semi-supervised HSI classification. Specifically, spatial information has been incorporated into SR model via the graph Laplacian regularization, it assumes that the spatial neighbors should have similar representation coefficients, the obtained coefficient matrix thus can reflect the similarity between samples more accurately. Besides, we also incorporate probabilistic class structure, which implies the probabilistic relationship between each sample and each class, into SR model to further improve discriminability of graph. The experimental results on Hyperion and AVIRIS hyperspectral data show that our method outperforms state of the art methods. [ABSTRACT FROM AUTHOR]

Several dimensionality reduction methods based on the max-min idea have been proposed in recent years and can obtain good classification performance. In this paper, inspired by the idea, we develop max-min linear discriminant analysis (MMLDA), which maximizes the minimum ratio of each two-class scatter measure to the within-class scatter measure. However, the method ignores equal emphasis on the distances between class centers and there may be room to improve the classification performance. We then propose regularized max-min linear discriminant analysis (RMMLDA), which introduces the Shannon entropy and the corresponding distance difference regularization terms based on MMLDA. The changing trends of distances between class centers can be precisely controlled in optimization and the separation between classes can be emphasized approximately equally. As a result, RMMLDA may obtain better classification performance. Experiments on synthetic data sets and three publicly available data sets demonstrate its effectiveness. [ABSTRACT FROM AUTHOR]

Graph-based semi-supervised learning (SSL), which performs well in hyperspectral image classification with a small amount of labeled samples, has drawn a lot of attention in the past few years. The key step of graph-based SSL is to construct a good graph to represent original data structures. Among the existing graph construction methods, sparse representation (SR) based methods have shown impressive performance on graph-based SSL. However, most SR based methods fail to take into consideration the class structure of data. In SSL, we can obtain a probabilistic class structure, which implies the probabilistic relationship between each sample and each class, of the whole data by utilizing a small amount of labeled samples. Such class structure information can help SR model to yield a more discriminative coefficients, which motivates us to exploit this class structure information in order to learn a discriminative graph. In this paper, we present a discriminative graph construction method called probabilistic class structure regularized sparse representation (PCSSR) approach, by incorporating the class structure information into the SR model, PCSSR can learn a discriminative graph from the data. A class structure regularization is developed to make use of the probabilistic class structure, and therefore to improve the discriminability of the graph. We formulate our problem as a constrained sparsity minimization problem and solve it by the alternating direction method with adaptive penalty (ADMAP). The experimental results on Hyperion and AVIRIS hyperspectral data show that our method outperforms state of the art. [ABSTRACT FROM AUTHOR]

The goal of this work is to present a computational model for contour detection, based on the surround suppression mechanisms of the primary visual cortex, in which the strength of surround suppression can adaptively vary with contrast—the surround modulation tends to be clearly suppressive at high contrast and less suppressive at low contrast, which may help to achieve the tradeoff between reducing cluttered edges and retaining object structures with weak responses, thus improving performance of contour detection under different circumstances; on the other hand, for similar texels can engender stronger suppression effects, more than orientation feature, texton is introduced to homogeneity measurement, which can well character multiple properties of texture regions, thereby yielding better contour detection results than other suppression models. The study can provide useful suggestions for contour detection algorithm in computer vision, but may also contribute to understanding contrast influence and feature representation in non-classical receptive field inhibition. [ABSTRACT FROM AUTHOR]

We present a method for object detection combining the effectiveness of a set of mid-level parts. These parts are learned weak-supervised from object bounding box annotations. The approach based part models can handle the detection of objects across changes in viewpoint, intraclass variability and object deformation. The objects are localized by the detected parts with learned information of location and scale. We evaluate the detection method on the standard PASCAL VOC 2007 dataset. Our system is competitive with the state of art in localizing the object. [ABSTRACT FROM AUTHOR]

This paper presents a novel tracking-by-detection approach for multitarget tracking. There are two major steps in our framework: data association to form global tracklet association, followed by trajectory estimation to deal with the remaining gaps. In the first step, we formulate tracklet association as an inference problem in a Hough forest random field, which combines Hough forest and conditional random field and allows us to model both local and global tracklet relationships in one unified model. In the second step, we improve the reversible-jump Markov chain Monte Carlo particle filtering method with explicit mutual-occlusion reasoning to fill in the remaining gaps from the first step and increase the overall tracking precision. Extensive experiments have been conducted on five public data sets, and the performance is comparable to that of the state-of-the-art method, if not better. [ABSTRACT FROM AUTHOR]

In this paper, we propose a simple yet effective local color image descriptor, completed local similarity pattern (CLSP), for face recognition. We represent the color image as the co-occurrence of its image pixel color quantization information and the local color image textural information. Specifically, CLSP consists of two complementary components: color label and local similarity pattern. Expressed by soft color label for every image pixel, we adopt the k-means clustering method and the soft-assignment coding method to summarize and encode the image pixel color information globally ignoring local neighboring pixels. While based on the similarity of color information between central pixel and its neighbors, local similarity pattern (LSP) is used to encode the local spatial textural feature of the color image ignoring their color value. Therefore, LSP has the merits of robustness and compactness of coding based feature extraction method. The joint distribution (2-D histogram), which unifies the color label and LSP, is used for a given image representation. Moreover, seven different color spaces are selected and fused to compensate each other for color image feature extraction. Experimental results on GeorgiaTech, AR, NUST-RWFR and LFW face database show that CLSP outperforms state-of-the-art color image feature extraction methods. [ABSTRACT FROM AUTHOR]

A good image representation is critical to face recognition task. Recently, eight-direction Kirsch masks based image descriptors, e.g. local directional pattern (LDP), local sign directional pattern (LSDP), have been devised and shown competitive results than conventional LBP descriptor. However, these methods may lose or do not fully explore valuable texture information of the image. To remedy this drawback, a novel discriminative image descriptor, namely local edge direction and texture descriptor (LEDTD) is proposed in this paper. LEDTD differs from the existing Kirsch based methods in a manner that it not only considers image edge direction information but also extracts image texture feature by encoding the edge response directions of center and its neighborhood pixels by employing local XOR binary coding strategy. Finally, edge direction and texture features are integrated to form the image feature vector. Extensive performance evaluations on four benchmark face databases show that the proposed approach yields a better performance in terms of the recognition rate as well as robustness to the noise compared with the state of the art methods. [ABSTRACT FROM AUTHOR]

This paper presents a two-level hierarchical multi-resolution image fusion method based on attention, which focus on combining multi-sensor images from visible and infrared spectrums into an image while maintaining the salient information necessary for human visual. Attention-based one-level fusion employs adaptive weighting fusion rules to obtain the initial fused image which may being the boundary of saliency regions, then two-level fusion based on nonsubsampled contourlet transform (NSCT) is used to hide possible imprints of the initial fused image. Firstly, for the given visible and infrared images from the same scene, salient regions are detected by visual attention model. Secondly, salient sizes are acquired by calculating maximum entropy. Thirdly, one-level fusion is achieved by the adoptive weighing fusion rules. Various weight values are adopted according to the visual saliency order from high to low. The initial fused image contains not only the salient information of visible image, but also the salient thermal target information of infrared image. Finally, two-level fusion based on NSCT is employed to fuse the visible source image, the infrared source image and the initial fused image, and the final fused image is obtained. Two different data sets from TNO, Human Factors are employed and the experimental results show that the proposed approach is efficient and also performed better than some existing methods. [ABSTRACT FROM AUTHOR]

Text information plays a significant role in many applications for providing more descriptive and abstract information than other objects. In this paper, an approach based on the confidence map and context information is proposed to robustly detect texts in natural scenes. Most of the conventional methods design sophisticated texture features to describe the text regions, while we focus on building a confidence map model by integrating the seed candidate appearance and the relationships with its adjacent candidates to highlight the texts from the backgrounds, and the candidates with low confidence value will be removed. In order to improve the recall rate, the text context information is adopted to regain the missing text regions. Finally, the text lines are formed and further verified, and the words are obtained by calculating the threshold to separate the intra-word letters from the inter-word letters. Experimental results on the three public benchmark datasets, i.e., ICDAR 2005, ICDAR 2011 and ICDAR 2013, show that the proposed approach has achieved the competitive performances by comparing with the other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper presents an effective and efficient approach based on simulating the information processing procedure of the biological visual system to solve the occlusion problem in facial expression recognition. The proposed method is composed of three components. First, Histograms of Oriented Gradients (HOG) and Local Binary Patterns (LBP) are used to extract features, which imitate the responding to stimuli on visual cortex. Second, Sparse Representation based Classification (SRC) is used due to its robustness to occlusions. Finally, since the recognition results of HOG+SRC and LBP+SRC are complimentary because HOG mainly extracts shape information while LBP primarily represents texture information, a strategy of combining HOG+SRC and LBP+SRC is implemented. Experiments on the Cohn–Kanade database show that the proposed method achieves better performance than many existing methods, and it is robust to both random occlusions and the major component occlusions. [ABSTRACT FROM AUTHOR]

A method based on color information and contour fragments was developed to identify citrus fruits in variable illumination conditions within tree canopy, in order to guide the robots for harvesting citrus fruits. The color properties of fruit targets within citrus-grove scene were analyzed, a preliminary segmentation method was put forward by fusing the chromatic aberration information and normalized RGB model. The set of contour fragments was constructed by detecting the significant edges of chromatic aberration map and the corners within these edges. The valid subset was chosen out by three indicators of every fragment: length, bending degree, and concavity or convexity. The combination analysis was done for these valid contour fragments, and the ellipse fitting was used for every subset of valid fragments to recover the occluded fruits. The partial order relationship was derived based on the distribution of the edge within the overlapped area. The results showed that occluded fruits were effectively recovered under natural outdoor light conditions using the proposed method, and the relative error was 5.27%. The partial order relation of fruit targets provide key cues for path planning of harvesting robot. [ABSTRACT FROM AUTHOR]

Max-min distance analysis (MMDA) for dimensionality reduction has been presented to guarantee class separation. However, class centers may be nonuniformly distributed and thus optimal classification accuracy may not be obtained. In this paper, we first give a novel method based on MMDA, called fractional-step max-min distance analysis (FMMDA), which relaxes max-min pairwise distances in fractional steps. The method can make a relatively uniform distribution of class centers and approximately maintain class separation. Then we present a more efficient method, called regularized max-min distance analysis (RMMDA), which achieves the same effect as FMMDA by integrating the Fisher criterion into MMDA. Moreover, we present the speedup and kernel versions of the methods to accelerate an optimization procedure and deal with the data distribution problem, respectively. Finally, we analyze the computational complexities of our methods. Empirical studies demonstrate that our methods can outperform or be comparable to some state-of-the-art discriminant analysis methods in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

Abstract: The double-random phase-encoding (DRPE) technique is a typical optical image encryption technique, which can also be used for image hiding. Usually, the secret image is encrypted with the DRPE technique and the encoded image is hidden into the host image via superimposition to obtain the stego-image. The attack technique on the DRPE-based image hiding method was proposed in this paper. Firstly, a randomly selected superimposition coefficient was used to approximate the original superimposition coefficient to extract the hidden encoded images from the stego-images approximately. Then, the chosen-plaintext attack technique on the DRPE-based optical image encryption technique was applied to recover the random phase masks used in the DRPE technique. The theoretical analysis indicated that, without considering the computational error, the recovered secret image via the proposed attack technique is identical to the original one. Even considering the computational error, it is identical to the secret image recovered with the original DRPE-based image hiding method, which demonstrates that the attack on the DRPE-based image hiding method is successfully achieved. The numerical simulation results demonstrated the correctness of the theoretical analysis. [Copyright &y& Elsevier]

Abstract: In this paper, we present a novel method to detect license plates in video sequences automatically. The framework mainly integrates the cascade detectors method and the Tracking Learning Detection (TLD) algorithm. The cascade detectors are used to detect license plates, and the TLD algorithm is adopted to track the license plate regions. The license plates in the first frame image are detected by the cascade detectors to build the original tracking list, the tracking results and the detection results in following frames will be compared, and the newly appearing license plate information will be added to the tracking list. Meanwhile, the tracking results existing in the current tracking list would be replaced by the corresponding detection results with higher degree of confidence. We demonstrate the effectiveness of our algorithm for license plate detection task on a road intersection dataset, and the experimental evaluation shows the detection performance has been greatly improved by synthetically using the detection and tracking strategy. [Copyright &y& Elsevier]

This paper presents a novel object detection method using a single instance from the object category. Our method uses biologically inspired global scene context criteria to check whether every individual location of the image can be naturally replaced by the query instance, which indicates whether there is a similar object at this location. Different from the traditional detection methods that only look at individual locations for the desired objects, our method evaluates the consistency of the entire scene. It is therefore robust to large intra-class variations, occlusions, a minor variety of poses, low-revolution conditions, background clutter etc., and there is no off-line training. The experimental results on four datasets and two video sequences clearly show the superior robustness of the proposed method, suggesting that global scene context is important for visual detection/localization. [ABSTRACT FROM AUTHOR]

Abstract: Artificial vision systems are powerful tools for the automatic guiding of fruit harvesting robots, a novel method based on chromatic aberration map and luminance map is developed to identify citrus fruits with highlight within a tree canopy. Twenty images of citrus-grove scene under direct sunlight are taken, the color properties of target objects are analyzed. First, parts of citrus fruits from background are segmented from background by thresholding the CAM (chromatic aberration map), then the highlight region of citrus fruits could be detected correctly from the tree canopy by thresholding the LM (luminance map), at last the citrus fruits can be detected integrally by fusing the segmented results of CAM and LM. The results showed that the fruits under direct sunlight can be segmented wholly using this algorithm, the detection accuracy by fusion method is up to 86.81%, and the false alarm rate of fusion method is 2.25%. [Copyright &y& Elsevier]

Abstract: An adaptive enhancement algorithm is proposed in the paper for low illumination color image which is low brightness and low contrast. The algorithm is realized through three steps: the global brightness adaptive adjustment, locally adaptive contrast enhancement and color restore. Experiments show that the brightness and local contrast of low illumination color image can be effectively enhanced. The details, especially in the dark region of enhanced image are more prominent and the enhanced image is more vivid than some other methods for low illumination color image enhancement. [Copyright &y& Elsevier]

Abstract: In the object recognition process of the remote sensing image, as the object data is influenced by the imaging scale, intensity and the shape of the object, the distribution of the object data is very different with that of known training data which leads to the low reliability of the object recognition. Aimed at the problem, an object recognition method based on transfer learning framework for the remote sensing image is proposed in this paper. The feature vectors of the object data are extracted by an improved LBP firstly, and then the transfer learning is used to find the common parameters among the feature spaces of the object data with the different distributions. The transfer learning method can transfer knowledge from the old object data to the new object data and improve the performance of the object recognition. According to the experiments in the satellite remote sensing images, it shows that the accuracy of object recognition has been greatly improved by our proposed method compared with the other classical methods. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a license plate detection method based on the gradient information and a cascade detection framework. The proposed method consists of three main modules: image preprocessing, license plate detection and license plate confirmation. Considering the diversities of Chinese license plate, we first get the gradient images to unify plate forms through image preprocessing. The license plate will be detected roughly by using the cascade AdaBoost classifier in the next step. Finally, several heuristic judgment strategies and a voting-based method are used to verify the candidate license plates. The effectiveness of our method is validated with an experimental study in license plate detection. [Copyright &y& Elsevier]

Abstract: Recently the sparse representation based classification (SRC) is successfully used to automatically recognize facial expression, well-known for its ability to solve occlusion and corruption problems. The results of those methods which using different features conjunction with SRC framework show state of the art performance on clean or noised facial expression images. Therefore, the role of feature extraction for SRC framework will greatly affect the success of facial expression recognition (FER). In this paper, we select a new feature which called LBP map. This feature is generated using local binary pattern (LBP) operator. It is not only robust to gray-scale variation, but also extracts sufficient texture information for SRC to deal with FER problem. Then we proposed a new method using the LBP map conjunction with the SRC framework. Firstly, we compared our method with state of the art published work. Then experiments on the Cohn–Kanade database show that the LBP map+SRC can reach the highest accuracy with the lowest time-consuming on clean face images than those methods which use different features such as raw image, Downsample image, Eigenfaces, Laplacianfaces and Gabor conjunction with SRC. We also experiment the LBP map+SRC to recognize face image with partial occluded and corrupted, the result shows that this method is more robust to occlusion and corruption than existing methods based on SRC framework. [Copyright &y& Elsevier]

Recently, the local binary patterns (LBP) have been widely used in the texture classification. The LBP methods obtain the binary pattern by comparing the gray scales of pixels on a small circular region with the gray scale of their central pixel. The conventional LBP methods only describe microstructures of texture images, such as edges, corners, spots and so on, although many of them show good performances on the texture classification. This situation still could not be changed, even though the multi-resolution analysis technique is adopted by LBP methods. Moreover, the circular sampling region limits the ability of the conventional LBP methods in describing anisotropic features. In this paper, we change the shape of sampling region and get an extended LBP operator. And a multi-structure local binary pattern (Ms-LBP) operator is achieved by executing the extended LBP operator on different layers of an image pyramid. Thus, the proposed method is simple yet efficient to describe four types of structures: isotropic microstructure, isotropic macrostructure, anisotropic microstructure and anisotropic macrostructure. We demonstrate the performance of our method on two public texture databases: the Outex and the CUReT. The experimental results show the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract: We propose a new method to estimate image bias field by an intuitive assumption that the overall trend of image surface is associated with bias field and the true bias field can evolve from image surface. To constrain the evolution, an energy function is designed based on two diffusion constraints. One is that the bias-free image has concise representation with image gradients. The other constraint is that model fit on the bias field which is more relaxed than previous methods. The new algorithm enables adaptive specifications of the estimated bias field's smoothness. The experimental results show our new technique can be applied to intensity inhomogeneity correction of medical image effectively. [Copyright &y& Elsevier]

Abstract: In this paper, we propose an Instance Transfer Boosting (ITB) framework for object tracking. The proposed tracking framework tries to transfer prior knowledge from the first frame and frame t-2 regarded as source instance to frame t-1 approximately as target instance. Those instances build the online training classifier used in tracking-by-detection for frame t. The novel method presents the tracking task in the current frame from the knowledge transferred by the first frame and the previous two frames, resulting in a more robust tracker for distinguishing the object from the background. Experimental results on several public video sequences demonstrated promising performance of the proposed tracking framework in both tracking accuracy and stability. [Copyright &y& Elsevier]

The local binary pattern (LBP) approach has been widely used in texture description. In this paper, we build a new framework to extract the binary patterns and propose a robust texture descriptor: multi-ring local binary pattern (MrLBP). The MrLBP algorithm creates patterns from several ringed areas and mainly contains two parts. One is the extra-ring local binary pattern operator that gets patterns from the mean values of different ringed areas. The other is the intra-ring local binary pattern operator that obtains patterns by counting the majority of binary values in every single ringed area. Moreover, the binary formation of each part of the MrLBP is obtained from two different aspects. The MrLBP method not only considers the binary relationship among pixels in a local region, but also focuses on the relationship between pixels in a local region and the whole image. This is a little different from the conventional LBP methods that only get the binary formation from the local gray scales differences. The experimental results on two public databases have validated the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract: Semi-supervised classification has become an active topic recently and a number of algorithms, such as Self-training, have been proposed to improve the performance of supervised classification using unlabeled data. In this paper, we propose a semi-supervised learning framework which combines clustering and classification. Our motivation is that clustering analysis is a powerful knowledge-discovery tool and it may reveal the underlying data space structure from unlabeled data. In our framework, semi-supervised clustering is integrated into Self-training classification to help train a better classifier. In particular, the semi-supervised fuzzy c-means algorithm and support vector machines are used for clustering and classification, respectively. Experimental results on artificial and real datasets demonstrate the advantages of the proposed framework. [Copyright &y& Elsevier]

Abstract: Automatic image segmentation is always a fundamental but challenging problem in computer vision. The simplest approach to image segmentation may be clustering feature vectors of pixels at first, then labeling each pixel with its corresponding cluster. This requires that the clustering on feature space must be robust. However, most of popular clustering algorithms could not obtain a robust clustering result yet, if the clusters in feature space have a complex distribution. Generally, for most of clustering-based segmentation methods, it still needs more constraints of positional relations between pixels in image lattice to be utilized during the procedure of clustering. Our works in this paper address the problem of image segmentation under the paradigm of pure clustering-then-labeling. A robust clustering algorithm which could maintain good coherence of data in feature space is proposed and utilized to do clustering on the L∗a∗b∗ color feature space of pixels. Image segmentation is straightforwardly obtained by setting each pixel with its corresponding cluster. Further, based on the theory of Minimum Description Length, an effective approach to automatic parameter selection for our segmentation method is also proposed. We test our segmentation method on Berkeley segmentation database, and the experimental results show that our method compares favorably against some state-of-the-art segmentation methods. [Copyright &y& Elsevier]

Abstract: Despite of its great success, two key problems are still unresolved for AdaBoost algorithms: how to select the most discriminative weak learners and how to optimally combine them. In this paper, a new AdaBoost algorithm is proposed to make improvement in the two aspects. First, we select the most discriminative weak learners by minimizing a novel distance related criterion, i.e., error-degree-weighted training error metric (ETEM) together with generalization capability metric (GCM), rather than training error rate only. Second, after getting the coefficients that are set empirically, we combine the weak learners optimally by tuning the coefficients using kernel-based perceptron. Experiments with synthetic and real scene data sets show our algorithm outperforms conventional AdaBoost. [Copyright &y& Elsevier]

Abstract: The paper presents a versatile nonlinear diffusion method to visually enhance the angiogram images for improving the clinical diagnosis. Traditional nonlinear diffusion has been shown very effective in edge-preserved smoothing of images. However, the existing nonlinear diffusion models suffer several drawbacks: sensitivity to the choice of the conductance parameter, limited range of edge enhancement, and the sensitivity to the selection of evolution time. The new anisotropic diffusion we proposed is based on facet model which can solve the issues mentioned above adaptively according to the image content. This method uses facet model for fitting the image to reduce noise, and uses the sum square of eigenvalues of Hessian as the standard of the conductance parameter selection synchronously. The capability of dealing with noise and conductance parameter can also change adaptively in the whole diffusion process. Moreover, our method is not sensitive to the choice of evolution time. Experimental results show that our new method is more effective than the original anisotropic diffusion. [Copyright &y& Elsevier]

Abstract: The responses of neurons in the primary visual cortex (V1) to stimulus inside the receptive field (RF) can be markedly modulated by stimuli outside the classical receptive field. The modulation, relying on contextual configurations, yields excitatory and inhibitory activities. The V1 neurons compose a functional network by lateral interactions and accomplish specific visual tasks in a dynamic and flexible fashion. Well-organized structures and conspicuous image locations are more salient and thus can pop out perceptually from the background. The excitatory and inhibitory activities give different visual physiological interpretations to the two kinds of saliencies. A model of contour extraction, inspired by visual cortical mechanisms of perceptual grouping, is presented. We unify the dual processes of spatial facilitation and surround inhibition to extract salient contours from complex scenes, and in this way coherent spatial configurations and region boundaries could stand out from their surround. The proposed method can selectively retain object contours, and meanwhile can dramatically reduce non-meaningful elements resulting from a texture background. This work gives a clear understanding for the roles of the inhibition and facilitation in grouping, and provides a biologically motivated computational strategy for contour extraction in computer vision. [Copyright &y& Elsevier]

Vessel segmentation is the base of three dimensional reconstruction on digital subtraction angiography (DSA) images. In this paper we propose two simple but efficient methods of vessel segmentation for DSA images. The original DSA image is divided into several appropriate subimages according to a prior knowledge of the diameter of vessels. We introduce the vessels existence measure to determine whether each subimage contains vessels and then choose an optimal threshold, respectively, for every subimage previously determined to contain vessels. Finally, an overall binarization of the original image is achieved by combining the thresholded subimages. Experiments are implemented on cerebral and hepatic DSA images. The results demonstrate that our proposed methods yield better binary results than global thresholding methods and some other local thresholding methods do. [Copyright &y& Elsevier]

A contour detection model, inspired by the behavior of the primary visual cortex, is presented. The response of a central stimulus in the receptive field is affected by the presence of surrounding stimuli – for some stimulus conditions, the response is suppressed and for other conditions the response is enhanced. The visual mechanisms of contextual influences are utilized to extract “coherent” configurations. This is mainly due to the following two reasons: (1) on the one hand, a smooth contour can yield collinear excitation, which highlights smooth contours from irregularly textured surround; (2) on the other hand, similar orientation textures receive iso-orientation surround suppression and region boundary is subjected to the less inhibition, which makes boundary more salient for perceptual pop-out. Accordingly, smooth contours progressively stand out from their surround and at the same time textures are gradually suppressed by their surround through dynamic fine-tuning of contextual information. The proposed method which distinguishes between contours and texture edges is more effective for contour-based object recognition tasks. Initial experiments show that the model can be successfully applied to contour detection. Especially, when object contours are lumped together with unwantedly cluttered surround, the advantage of our approach is more prominent. This study provides a biological scheme for contour detection in computer vision. [Copyright &y& Elsevier]

Transition region based thresholding is a newly developed approach for image segmentation in recent years. Gradient-based transition region extraction methods (G-TREM) are greatly affected by noise. Local entropy in information theory represents the variance of local region and catches the natural properties of transition regions. In this paper, we present a novel local entropy-based transition region extraction method (LE-TREM), which effectively reduces the affects of noise. Experimental results demonstrate that LE-TREM significantly outperforms the conventional G-TREM. [Copyright &y& Elsevier]

Learning from large-scale contrastive language-image pre-training like CLIP has shown remarkable success in a wide range of downstream tasks recently, but it is still under-explored on the challenging few-shot action recognition (FSAR) task. In this work, we aim to transfer the powerful multimodal knowledge of CLIP to alleviate the inaccurate prototype estimation issue due to data scarcity, which is a critical problem in low-shot regimes. To this end, we present a CLIP-guided prototype modulating framework called CLIP-FSAR, which consists of two key components: a video-text contrastive objective and a prototype modulation. Specifically, the former bridges the task discrepancy between CLIP and the few-shot video task by contrasting videos and corresponding class text descriptions. The latter leverages the transferable textual concepts from CLIP to adaptively refine visual prototypes with a temporal Transformer. By this means, CLIP-FSAR can take full advantage of the rich semantic priors in CLIP to obtain reliable prototypes and achieve accurate few-shot classification. Extensive experiments on five commonly used benchmarks demonstrate the effectiveness of our proposed method, and CLIP-FSAR significantly outperforms existing state-of-the-art methods under various settings. The source code and models are publicly available at https://github.com/alibaba-mmai-research/CLIP-FSAR. [ABSTRACT FROM AUTHOR]

In this article, we propose a novel compressed latent distribution representation for 3D hand pose estimation from monocular RGB images to alleviate the channel correspondence problem. The channel correspondence problem occurs when the 2D and depth coordinates are estimated from independent feature maps, which means the 2D and depth channel sequences may not match during the cross-dataset inference. In contrast, we propose a compressed latent distribution representation that the 2D and depth feature maps for each joint are interconnected and inter-constrained more directly, effectively alleviating the channel correspondence problem and improving cross-dataset performance. Moreover, we design an efficient encoder-decoder network that can maintain the resolution of feature maps to enable better hand feature extraction from monocular RGB images. In this work, the overall pipeline contains two branches: one is the 2D hand pose estimation branch based on a latent heatmap representation (LHR); the other is the 3D hand pose estimation branch based on our proposed latent distribution representation (LDR). In this way, the 2D estimation branch serves as guidance for the 3D branch, which simplifies the optimization of the overall network and results in a more rapid convergence during training. The results on several benchmark datasets (including STB, RHD, and the most recently released InterHand2.6M) demonstrate that our proposed method achieves state-of-the-art (SOTA) performance. [ABSTRACT FROM AUTHOR]

Existing retinex-based low-light image enhancement strategies focus heavily on crafting complex networks for Retinex decomposition but often result in imprecise estimations. To overcome the limitations of previous methods, we introduce a straightforward yet effective strategy for Retinex decomposition, dividing images into colormaps and graymaps as new estimations for reflectance and illumination maps. The enhancement of these maps is separately conducted using a diffusion model for improved restoration. Furthermore, we address the dual challenge of perturbation removal and brightness adjustment in illumination maps by incorporating brightness guidance. This guidance aids in precisely adjusting the brightness while eliminating disturbances, ensuring a more effective enhancement process. Extensive quantitative and qualitative experimental analyses demonstrate that our proposed method improves the performance by approximately 4.4 % on the LOL dataset compared to other state-of-the-art diffusion-based methods, while also validating the model's generalizability across multiple real-world datasets. [ABSTRACT FROM AUTHOR]

Person re-identification is a challenging problem that aims at matching persons across multiple non-overlapping cameras. Previous works on person re-identification mainly focus on solving the problems caused by pose variations, backgrounds, and occlusion while ignoring the viewpoint variations. To address the viewpoint problem, we propose a Viewpoint Transform Matching (VTM) model, which reduces the inference of viewpoints by feature-level viewpoint transformation. Overall, we design a framework by using viewpoint specific branches to separately extract the representation of different viewpoints. Specifically, we first select the branch for an input image, according to its viewpoint information. In the branch, we transform the feature to other viewpoints by establishing a mutually transformable connection between the features of different viewpoints. To suppress the interference among viewpoints, we propose a viewpoint transform classifier module to independently train a classifier for each viewpoint. To improve the effectiveness of the transform, we propose a viewpoint transform loss to guarantee the consistency of the original features and the transformed features. Experiments conducted on Market-1501, DukeMTMC-reID and CUHK03 show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Visual Question Answering (VQA) is a multimodal task, which requires understanding the information in the natural language questions and paying attention to the useful information in the images. So far, the solution of VQA tasks can be divided into grid-based methods and bottom-up-based methods. The grid-based method directly extracts the semantic features of the image by leveraging the convolution neural network (CNN), so it has a praiseworthy computational efficiency, but the global convolution feature ignores the information of the key area and causes the performance bottleneck. The bottom-up-based method needs to detect potentially problem-related objects by using some object detection frameworks, e.g. Faster RCNN, so it has better performance, but the computational efficiency is reduced due to the computation of Region Proposal Network (RPN) and Non-Maximum Suppression (NMS). Based on the aforementioned reasons, we propose a fully transformer-based network (FTN) that can maintain a balance between computational efficiency and accuracy, which can be trained end-to-end and consists of three components: question module, image module, and fusion module. Meanwhile, the image module and the question module are visualized to explore the operating rules of the transformer. The experiment results demonstrate that the FTN can focus on key information and objects in the question module and the image module, and our single model has reached 69.01% accuracy on the VQA2.0 dataset, which is superior to the grid-based methods. Although FTN fails to surpass a few state-of-the-art bottom-up-based methods, the FTN has obvious advantages in computational efficiency. The code will be released at https://github.com/weixiang-xu/FTN-VQA.git. [ABSTRACT FROM AUTHOR]

Video synopsis is an intelligent condensation approach to solve fast video browsing and retrieval for surveillance cameras. However, collision caused by unsatisfied tube rearrangement in traditional methods brings uncomfortable visual effect to users and how to mitigate the collision still remains an attracting topic. Unlike conventional methods that deal with tube rearrangement by minimizing a global energy function, we propose a novel approach by formulating it as a graph coloring problem. In our approach, all the tubes are firstly mapped into the spatial domain for analyzing their potential collision relationship. The input tube set is then represented by a graph structure, where each node stands for a tube and the edge between two nodes represents the potential collision relationship. To mitigate the collision artifacts, our method finds the mapping of tubes from original video to synopsis video by L ( q )- coloring the graph, which separates tubes from their collision points. The parameter q is left tunable to make a compromise between collision artifacts and synopsis length, which can better meet users’ demand of freely adjusting the compactness of synopsis video. The shifted objects are finally composited with the background image to obtain the high-quality video synopsis. Extensive experimental results show that the proposed method can generate more compact video synopsis with less collision artifacts than the existing methods. [ABSTRACT FROM AUTHOR]

Skin color provides a useful cue for vision based human-computer interaction (HCI). However, rapidly changing illumination conditions under HCI application environment make skin color detection a challenging task, as skin colors in an image highly depend on the illumination under which the image was taken. This paper presents a method for skin color detection under rapidly changing illumination conditions. Skin colors are modeled under the Bayesian decision framework. Face detection is employed to online sample skin colors and a dynamic thresholding technique is used to update the skin color model. When there is no face detected, color correction strategy is employed to convert the colors of the current frame to those as they appear under the same illuminant of the last model updated frame. Skin color detection is then applied on the color corrected image. Face detection is time-consuming and hence should not be applied to every frame in real-time applications on general consumer hardware. To improve efficiency, a novel method is proposed to detect illumination changes, and face detection is used to update the skin color model only if the illumination has changed. Experimental results show that the proposed method can achieve satisfactory performance for skin color detection under rapidly changing illumination conditions in real-time on general consumer hardware. [ABSTRACT FROM AUTHOR]

Few-shot classification (FSC) is a highly challenging task, as only a small number of labeled samples are available when identifying new categories. Distance metric learning-based methods have emerged as a prominent approach to FSC, which typically use a distance function to measure the difference between query and support samples for identifying the class membership of the query sample. However, these methods simply treat each channel difference between query and support features equally when computing the class scores. Since different channels in the learned feature seek for different patterns, these distance metrics fail to consider that different channels are of different importance to FSC, and thus cannot accurately measure the similarity between samples. To address this issue, we propose a Q uery- C entric D istance M odulator (QCDM) to generate query-related weights for each channel difference adaptively. Specifically, since the distribution of difference between a query sample and all support samples in a particular channel can reflect the importance of this channel to the classification of the query sample, we take this difference vector as input and generate a query-specific channel weight through a meta-network. QCDM can guide FSC models to focus on discriminative channel differences and achieve better generalization. QCDM is a plug-and-play module that can be seamlessly integrated with existing distance metric learning-based FSC methods. Extensive experimental results indicate that our method can effectively improve the performance of distance metric learning-based FSC methods. The source code is available in https://github.com/Wu-Wenxiao/QCDM. • A plug-and-play query-centric distance modulator for distance metric learning-based FSC methods. • Weight generation through data inconsistency within each channel from different classes. • Decoupled from the process of feature extraction and integrated with any given backbone. [ABSTRACT FROM AUTHOR]

The deep learning based methods have improved the visual tracking precision significantly. However, the background distraction and the high precise localization remain challenging problems. Despite that some methods have fused the deep and shallow layer features to solve these problems, the existing fusion methods, like simply concatenating or adding the features from the different layers, cannot take the advantage of both the deep and shallow layer features fully. In this paper, we propose a new adaptive feature fusion method, called the instance-based feature pyramid (IBFP) to obtain the discriminative high-resolution feature, which not only inherits the discriminative information from the deep layer feature, but also keeps the high precision localization information of the shallow layer feature. For utilizing the deep and shallow features effectively, we design an instance-based upsampling (IBU) module to fuse them, and a compressed space channel selection (CSCS) module to re-weight the feature channels adaptively. We insert the IBU and CSCS modules in the Siamese tracker for end-to-end training and testing. By using the proposed IBU and CSCS modules, we fuse the deep and shallow features in a series manner. Experiments on large-scale benchmark datasets demonstrate that the proposed modules boost the capabilities of distinguishing the targets and the similar distractors and perform favorably against the state-of-the-art. [ABSTRACT FROM AUTHOR]

Few-shot action recognition is a challenging but practical problem aiming to learn a model that can be easily adapted to identify new action categories with only a few labeled samples. However, existing attempts still suffer from two drawbacks: (i) learning individual features without considering the entire task may result in limited representation capability, and (ii) existing alignment strategies are sensitive to noises and misaligned instances. To handle the two limitations, we propose a novel Hybrid Relation guided temporal Set Matching (HyRSM++) approach for few-shot action recognition. The core idea of HyRSM++ is to integrate all videos within the task to learn discriminative representations and involve a robust matching technique. To be specific, HyRSM++ consists of two key components, a hybrid relation module and a temporal set matching metric. Given the basic representations from the feature extractor, the hybrid relation module is introduced to fully exploit associated relations within and cross videos in an episodic task and thus can learn task-specific embeddings. Subsequently, in the temporal set matching metric, we carry out the distance measure between query and support videos from a set matching perspective and design a bidirectional Mean Hausdorff Metric to improve the resilience to misaligned instances. Furthermore, we extend the proposed HyRSM++ to deal with the more challenging semi-supervised few-shot action recognition and unsupervised few-shot action recognition tasks. Experimental results on multiple benchmarks demonstrate that our method consistently outperforms existing methods and achieves state-of-the-art performance under various few-shot settings. The source code is available at https://github.com/alibaba-mmai-research/HyRSMPlusPlus. • A new temporal coherence regularization on videos is proposed. • Capturing the intra- and inter-relations inside the episodic task. • Reformulating the query-support metric as a set matching problem. [ABSTRACT FROM AUTHOR]

Abstract: A double-random phase-encoding technique based information hiding scheme was analyzed and improved. By encoding the hidden image with the double-random phase-encoding technique, and embedding the encoded image into the enlarged host image to generate the composed image, the original scheme can recover the hidden image from the composed image directly (i.e., without using the original host image) with satisfying quality. For the proposed improved scheme, 3 values (other than 2 values as in the original scheme) are embedded into each 2×2 block within the enlarged host image, resulting in increasing the data hiding capacity around 50%, while not influence the qualities of the composed image and the reconstructed hidden image seriously. In addition, the security of the original scheme is enhanced markedly. Theoretical analysis and experimental results demonstrate the effectiveness of the improved scheme. [Copyright &y& Elsevier]