Search

Organizing multimodal Web pages into hot topics is the core step to grasp trends on the Web. However, the less-constrained social media generate noisy user-generated content, which makes a detected topic be less coherent and less interpretable. In this paper, we address this problem by proposing a coupled Poisson deconvolution to jointly handle topic detection and topic description. For the topic detection, the interestingness of a topic is estimated from the similarities refined by the description of topics; for the topic description, the interestingness of topics is leveraged to describe topics. Two processes cyclically detect interesting topics and generate the multimodal description of topics. This is the innovation of this paper, which just likes killing two birds with one stone. Experiments not only show the significantly improved accuracies for the topic detection but also demonstrate the interpretable descriptions for the topic description on two public data sets. [ABSTRACT FROM AUTHOR]

Image representation and annotation are two key tasks in practical applications such as image search. Existing methods have tried to learn an effective representation or to predict tags directly using multi-view low-level visual features, which usually contain redundant information. However, these two tasks are closely related and interact on each other. A suitable image representation can yield better image annotation results, which in turn can effectively guide the image representation learning. In this paper, we propose to jointly conduct multi-view representation and image annotation via optimal predictive subspace learning, making the two tasks promote each other. Specifically, for subspace learning, visual structure and semantic information of images are exploited to make the learned subspace more discriminative and compact. For tag prediction, support vector machines (SVM) is adopted to obtain better tag prediction results. Then to simultaneously learn image representation, tag predictors and projection function, the three subproblems are combined into a unified optimization objective function and an alternative optimization algorithm is derived to solve it. Experimental results on four image datasets illustrate that our method is superior to the other image annotation methods. [ABSTRACT FROM AUTHOR]

Image classification refers to the task of automatically classifying the categories of images based on the contents. This task is typically solved using visual features with the histogram based classification scheme. Although effective, this strategy has two drawbacks. On one hand, histogram based representation often disregards the object layout which is very important for classification. On the other hand, visual features are unable to fully separate different images due to the semantic gap. To solve these two problems, in this paper, we propose a novel image classification method by explicitly and implicitly representing the images with searching strategy. First, to make use of object layouts, we randomly select a number of regions and then use these regions for image representations. Second, we generate the explicitly semantic representations using a number of pre-learned semantic models. Third, we measure the visual similarities with the Internet images and use the text information for implicitly semantic representations. Since Internet images are contaminated with noise, the resulting representations only implicitly reflect the contents of images. Finally, both the explicitly and implicitly semantic representations are jointly modeled for image classifications by training bi-linear classifiers. We evaluate the effectiveness of the proposed image classification by search with explicitly and implicitly semantic representations method (EISR) on the Scene-15 dataset, the MIT-Indoor dataset, the UIUC-Sports dataset and the PASCAL VOC 2007 dataset. The experimental results prove the usefulness of the proposed method. [ABSTRACT FROM AUTHOR]

Weakly supervised object detection (WSOD) is a challenging task that requires simultaneously learning object detectors and estimating object locations under the supervision of image category labels. Many WSOD methods that adopt multiple instance learning (MIL) have nonconvex objective functions and, therefore, are prone to get stuck in local minima (falsely localize object parts) while missing full object extent during training. In this article, we introduce classical continuation optimization into MIL, thereby creating continuation MIL (C-MIL) with the aim to alleviate the nonconvexity problem in a systematic way. To fulfill this purpose, we partition instances into class-related and spatially related subsets and approximate MIL’s objective function with a series of smoothed objective functions defined within the subsets. We further propose a parametric strategy to implement continuation smooth functions, which enables C-MIL to be applied to instance selection tasks in a uniform manner. Optimizing smoothed loss functions prevents the training procedure from falling prematurely into local minima and facilities learning full object extent. Extensive experiments demonstrate the superiority of CMIL over conventional MIL methods. As a general instance selection method, C-MIL is also applied to supervised object detection to optimize anchors/features, improving the detection performance with a significant margin. [ABSTRACT FROM AUTHOR]

We propose an end-to-end place recognition model based on a novel deep neural network. First, we propose to exploit the spatial pyramid structure of the images to enhance the vector of locally aggregated descriptors (VLAD) such that the enhanced VLAD features can reflect the structural information of the images. To encode this feature extraction into the deep learning method, we build a spatial pyramid-enhanced VLAD (SPE-VLAD) layer. Next, we impose weight constraints on the terms of the traditional triplet loss (T-loss) function such that the weighted T-loss (WT-loss) function avoids the suboptimal convergence of the learning process. The loss function can work well under weakly supervised scenarios in that it determines the semantically positive and negative samples of each query through not only the GPS tags but also the Euclidean distance between the image representations. The SPE-VLAD layer and the WT-loss layer are integrated with the VGG-16 network or ResNet-18 network to form a novel end-to-end deep neural network that can be easily trained via the standard backpropagation method. We conduct experiments on three benchmark data sets, and the results demonstrate that the proposed model defeats the state-of-the-art deep learning approaches applied to place recognition. [ABSTRACT FROM AUTHOR]

Images can be represented by multiple views and each view describes a specific visual appearance. Compared with single view learning method, multi-view methods can integrate information of different views to generate better clustering performance. Most of the existing multi-view methods assume that the importance of each view is the same to all the images. However, since visual appearance of images are different, the description abilities of different features vary with images. To solve this problem, a group-aware multi-view fusion approach is proposed in this paper. Specifically, images are partitioned into groups according to their visual appearance, and different fusion weights are assigned to different groups. We develop two paradigms under our group-aware fusion framework: pair-wise fusion and center-wise fusion. The former focuses on generating more accurate fusion results, while the latter achieves lower computational complexity. We design an optimization objective function which combines consensus and discrimination criterion to select more reliable and discriminative views for multi-view fusion. The clustering results and the fusion weights are learned by an iterative optimization algorithm. Experiments on four real-world image datasets indicate that our approach achieves promising image clustering performance over the existing methods. [ABSTRACT FROM AUTHOR]

Existing multi-label learning approaches mainly utilize an identical data representation composed of all the features in the discrimination of all the labels, and assume that all the class labels are observed for each training sample. However, in multi-label learning, each class label might be determined by some specific features of its own, and only a partial label set of each example can be obtained for some real applications. This paper proposes a new method to learn L abel- S pecific features for multi-label classification with M issing L abels, named LSML. First, a new supplementary label matrix is augmented from the incomplete label matrix by learning high-order label correlations. Then, a label-specific data representation for each class label is learned, and the multi-label classifier is constructed simultaneously based on it by incorporating the learned high-order label correlations. A comparative study with the state-of-the-art approaches manifests the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Inspired by the photometric invariance of color space, this paper proposes a simple yet powerful descriptor for object detection and recognition, called Rotative Maximal Pattern (RMP). The effectiveness of RMP comes from the two components: Rotatable Couple Templates (RCTs) with max pooling, and Normalized Histogram Intersection (NHI) with the theoretical guarantee. More concretely, RCTs are the combination of two templates to code the possible rotations. NHI serves as the similarity between two color histograms. We have conducted extensive experiments on INRIA pedestrian and Pascal VOC2007 data sets for object detection tasks; we also show that our approach leads to a promising performance on Caltech 101, Scene 15, UIUCsport and Stanford 40 action data sets. [ABSTRACT FROM AUTHOR]

This paper tries to separate fine-grained images by jointly learning the encoding parameters and codebooks through low-rank sparse coding (LRSC) with general and class-specific codebook generation. Instead of treating each local feature independently, we encode the local features within a spatial region jointly by LRSC. This ensures that the spatially nearby local features with similar visual characters are encoded by correlated parameters. In this way, we can make the encoded parameters more consistent for fine-grained image representation. Besides, we also learn a general codebook and a number of class-specific codebooks in combination with the encoding scheme. Since images of fine-grained classes are visually similar, the difference is relatively small between the general codebook and each class-specific codebook. We impose sparsity constraints to model this relationship. Moreover, the incoherences with different codebooks and class-specific codebooks are jointly considered. We evaluate the proposed method on several public image data sets. The experimental results show that by learning general and class-specific codebooks with the joint encoding of local features, we are able to model the differences among different fine-grained classes than many other fine-grained image classification methods. [ABSTRACT FROM AUTHOR]

Visual information has been widely used for image representation. Although proven very effective, the visual representation lacks explicit semantics. However, how to generate a proper semantic space for image representation is still an open problem that needs to be solved. To jointly model the visual and semantic representations of images, we propose a boosted random contextual semantic space based image representation method. Images are initially represented using local feature’s distribution histograms. The semantic space is generated by randomly selecting training images. Images are then mapped into the semantic space accordingly. Semantic context is explored to model the correlations of different semantics which is then used for classification. The classification results are used to re-weight training images in a boosted way. The re-weighted images are used to construct new semantic space for classification. In this way, we are able to jointly consider the visual and semantic information of images. Image classification experiments on several public datasets show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The combination of local features with sparse technique has improved image classification performance dramatically in recent years. Although very effective, this strategy still has two shortcomings. First, local features are often extracted in a pre-defined way (e.g. SIFT with dense sampling) without considering the classification task. Second, the codebook is generated by sparse coding or its variants by minimizing the reconstruction error which has no direct relationships with the classification process. To alleviate the two problems, we propose a novel boosted local features method with random orientation and location selection. We first extract local features with random orientation and location using a weighting strategy. This randomization process makes us to extract more types of information for image representation than pre-defined methods. These extracted local features are then encoded by sparse representation. Instead of generating the codebook in a single process, we construct a series of codebooks and the corresponding encoding parameters of local features using a boosting strategy. The weights of local features are determined by the classification performances of learned classifiers. In this way, we are able to combine the local feature extraction and encoding with classifier training into a unified framework and gradually improve the image classification performance. Experiments on several public image datasets prove the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]