Search

Cross-modal hashing has received growing interest to facilitating efficient retrieval across large-scale multi-modal data, and existing methods still face three challenges: 1) Most offline learning works are unsuitable for processing and training with streaming multi-modal data. 2) Current online learning methods rarely consider the potential interdependency between the label categories. 3) Existing supervised methods often utilize pairwise label similarities or adopt relaxation scheme to learn hash codes, which, respectively, require much computation time or accumulate large quantization loss during the learning process. To alleviate these challenges, this paper presents an efficient Online Label Consistent Hashing (OLCH) approach for streaming cross-modal retrieval. The proposed approach first exploits the relative similarity of semantic labels and utilizes the multi-class classification to derive the common semantic vector. Then, an online semantic representation learning framework is adaptively designed to preserve the semantic similarity across different modalities, and a mini-batch online gradient descent approach associated with forward–backward splitting is developed to discriminatively optimize the hash functions. Accordingly, the hash codes are incrementally learned with high discriminative capability, while avoiding high computation complexity to process the streaming data. Extensive experiments highlight the superiority of the proposed approach and show its very competitive performance in comparison with the state-of-the-arts. • Present an efficient online label consistent hashing approach to benefit cross-modal retrieval for streaming data. • Develop an online semantic representation learning framework to preserve the semantic similarity. • Propose a mini-batch online gradient descent approach to incrementally optimize the hash functions. • Exploit a forward–backward splitting scheme to learn the discriminative hash codes. [ABSTRACT FROM AUTHOR]

Image captioning requires not only accurate recognition of objects and corresponding relationships, but also full comprehension of the scene information. However, existing models suffer from partial understanding and object hallucination. In this paper, a Cross-modal retrievAl and viSual condiTioning model (CAST) is proposed to address the above issues for image captioning with three key modules: an image–text retriever, an image & memory comprehender and a dual attention decoder. Aiming at a comprehensive understanding, we propose to exploit cross-modal retrieval to mimic human cognition, i.e. , to trigger retrieval of contextual information (called episodic memory) about a specific event. Specifically, the image–text retriever searches the top n relevant sentences which serve as episodic memory for each input image. Then the image & memory comprehender encodes an input image and enriches episodic memory by self-attention and relevance attention respectively, which can encourage CAST to comprehend the scene thoroughly and support decoding more effectively. Finally, such image representation and memory are integrated into our dual attention decoder, which performs visual conditioning by re-weighting image and text features to alleviate object hallucination. Extensive experiments are conducted on MS COCO and Flickr30k datasets, which demonstrate that our CAST achieves state-of-the-art performance. Our model also has a promising performance even in low-resource scenarios (i.e. 0.1%, 0.5% and 1% of MS COCO training set). • An image–text retriever is proposed to search contextual information for captioning. • An image & memory comprehender is proposed for further understanding the scene. • A dual attention decoder is proposed to alleviate object hallucination. • The cross-modal retrieval and visual conditioning model achieves SOTA performance. [ABSTRACT FROM AUTHOR]

Online hashing technology has attracted extensive attention owing to its effectiveness and efficiency in processing large-scale streaming data. However, there are still some limitations: (1) In practical applications, the observed labels of multimedia data are obtained through manual annotation, which may inevitably introduce some noises into labels. This may lead to retrieval performance degradation when the noisy labels are directly applied to retrieval tasks. (2) The potential semantic correlation of multi-labels cannot be fully explored. To overcome these limitations, in this paper, we propose robust online hashing with label semantic enhancement (ROHLSE). Specifically, ROHLSE seeks to recover the clean labels from the provided noisy labels by imposing low-rank and sparse constraints. Meanwhile, it employs the representation of samples in the feature space to predict the labels via the dependency between sample instances and labels. To efficiently handle streaming data, ROHLSE preserves the similarity between new data, and establishes the semantic relationships between new and old data through chunk similarity, simultaneously. Furthermore, ROHLSE can fully utilize the semantic correlations between multiple labels of each instance. Extensive experiments are conducted on three benchmark datasets to demonstrate the superiority of the proposed ROHLSE approach. • We propose a novel robust OCMH approach for real retrieval problems. It can learn the hash codes of new data and update the hash functions via online learning way, effectively handling the streaming media data. Therefore, ROHLSE achieves more retrieval efficiency than traditional offline hashing technology. • To the best of our knowledge, this is the first work to retrieval multi-modal data with noisy labels among online hashing approaches. Specifically, ROHLSE recovers the clean labels from the observed labels by imposing the low-rank and sparse constraints, and then adaptively learns the latent semantic correlations between the clean labels to guide the generation of hash codes. • In ROHLSE, the learning of hash codes is guided by pairwise similarity based on new data. In addition, chunk similarity is constructed based on the semantic relationship between new data and the results of the previous round and it can avoid semantic forgetting. An efficient optimization algorithm is developed to solve the proposed model. Experimental results on several benchmark datasets have demonstrated the advantage of our ROHLSE approach. [ABSTRACT FROM AUTHOR]

Multi-label classification problems usually occur in tasks related to information retrieval, like text and image annotation, and are receiving increasing attention from the machine learning and pattern recognition fields. One of the main issues under investigation is the development of classification algorithms capable of maximizing specific accuracy measures based on precision and recall. We focus on the widely used F measure, defined for binary, single-label problems as the weighted harmonic mean of precision and recall, and later extended to multi-label problems in three ways: macro-averaged, micro-averaged and instance-wise. In this paper we give a comprehensive survey of theoretical results and algorithms aimed at maximizing F measures. We subdivide it according to the two main existing approaches: empirical utility maximization, and decision-theoretic. Under the former approach, we also derive the optimal (Bayes) classifier at the population level for the instance-wise and micro-averaged F , extending recent results about the single-label F . In a companion paper we shall focus on the micro-averaged F measure, for which relatively fewer solutions exist, and shall develop novel maximization algorithms under both approaches. [ABSTRACT FROM AUTHOR]

Highlights • A Densely Semantic Embedding Network (DSEN) is constructed, which is able to embed the attributes into the DSE-LSTM with each of other inputs at each step of word generation. • An enhancement to the representation of the inputs like image feature, text feature and the hidden state by the modulation of the attributes. • An activation function is proposed to compose together the attributes. Typically, it is designed as a Threshold ReLU (TReLU). With this TReLU, the attributes can be modulated to be sparser with enough discriminative power. • The comprehensive evaluations demonstrate the effectiveness of our method for both image captioning and image-text cross modal retrieval tasks. Abstract Recently, attributes that contain high-level semantic information of image are always used as a complementary knowledge to improve image captioning performance. However, the use of attributes in prior works cannot excavate the latent visual concepts effectively. At each time step, the semantic information which is sensitive to the predicted word could be different. In this paper, we propose a Dense Semantic Embedding Network (DSEN) for this task. The distinct operation of this network is to densely embed the attributes with the multi-modal of image and text at each step of word generation. The discriminative semantic information hidden in these attributes is formatted in form of global likelihood probabilities. As a result, this dense embedding can modulate the feature distributions of the image, text modals and the hidden states to explicit semantic representation. Furthermore, to improve the discrimination of attributes, a Threshold ReLU (TReLU) is proposed. In addition, a bidirectional LSTM structure is incorporated into the DSEN to capture both the previous and future contexts. Extensive experiments on the COCO and Flickr30K datasets achieve superior results when compared with the state-of-the-art models for the tasks of both image captioning and image-text cross modal retrieval. Most remarkably, our method obtains outstanding performance on the retrieval task, compared with the state-of-the-art models. [ABSTRACT FROM AUTHOR]

Highlights • Important application areas such as signature extraction, ink or document aging, information retrieval from historical document images, paintings analysis and forensic analysis of documents are reviewed. • Challenges involved in the hyperspectral document image acquisition are discussed. • Challenges involved in the pre-processing of HS document images are discussed. • A detailed review and discussion of the commercial HSI systems for document image analysis is presented. • The survey will provide useful information to researchers and forensic scientists towards this exciting technology and help them to solve many challenging problems of document image analysis. Abstract Automatic image analysis is a crucial component of many intelligent systems designed for high-level understanding of documents. Most document image understanding systems are usually based on applying pattern recognition techniques to conventional three channel RGB images. Airborne and satellite based macro scale Hyperspectral Imaging (HSI) systems are well established for geosciences. Recently, owing to advancements in imaging speed and reduced camera costs, micro scale HSI systems are also gaining importance in ground based applications such as hyperspectral document image analysis. HSI is non destructive and offers new opportunities via measuring richer information along spectral dimension by imaging the document in contiguous bands across the electromagnetic spectrum. Hyperspectral document imaging has shown potential for solving many challenging problems of document image analysis including signature extraction, ink or document aging, information retrieval from historical document images, paintings and forensic analysis of documents. In this paper, we explore the potential of HSI for document image analysis and present a comprehensive review of the literature and future prospects. We highlight and discuss the challenges involved in the acquisition and processing of hyperspectral document images. A review of commercial HSI systems for document image analysis is also presented. [ABSTRACT FROM AUTHOR]

Highlights • The network diffusion based embedding method solves the limitations of random walks. • Diffusion driven process is employed to capture both depth and breadth information. • The time dimension attached to node sequences strengthens information preserving. • The network inference technique based on cascades captures the global information. • Our method is more robust to low sampling frequency and highly unbalanced networks. Abstract In network embedding, random walks play a fundamental role in preserving network structures. However, random walk methods have two limitations. First, they are unstable when either the sampling frequency or the number of node sequences changes. Second, in highly biased networks, random walks are likely to bias to high-degree nodes and neglect the global structure information. To solve the limitations, we present in this paper a network diffusion embedding method. To solve the first limitation, our method uses a diffusion driven process to capture both depth and breadth information in networks. Temporal information is also included into node sequences to strengthen information preserving. To solve the second limitation, our method uses the network inference method based on information diffusion cascades to capture the global network information. Experiments show that the new proposed method is more robust to highly unbalanced networks and well performed when sampling under each node is rare. [ABSTRACT FROM AUTHOR]

Highlights • We design a semi-supervised feature representation framework for person Re-Identification which effectively utilizes both labeled and unlabeled training data to learn a discriminative representation so that person images across disjoint camera views can be reliably matched. • A multi-view clustering method is proposed to integrate features from multiple Convolutional Neural Networks for clustering, which can give more accurate label estimation for unlabeled data. • Each of our Convolutional Neural Networks utilizes a siamese network that simultaneously computes the identification loss and verification loss, which simultaneously learns a discriminative Convolutional Neural Network embedding and a similarity metric, and thus improving pedestrian retrieval accuracy. Extensive experiments on large-scale person Re-Id datasets demonstrate the effectiveness of our method. Abstract Person Re-Identification (Re-Id) is a challenging task focusing on identifying the same person among disjoint camera views. A number of deep learning algorithms have been reported for this task in fully-supervised fashion which requires a large amount of labeled training data, while obtaining high quality labels for Re-Id is extremely time consuming. To address this problem, we propose a semi-supervised Re-Id framework by using only a small portion of labeled data and some additional unlabeled samples. This paper approaches the problem by constructing a set of heterogeneous Convolutional Neural Networks (CNNs) fine-tuned using the labeled portion, and then propagating the labels to the unlabeled portion for further fine-tuning the overall system. In this work, label estimation is a key component during the propagation process. We propose a novel multi-view clustering method, which integrates features of multiple heterogeneous CNNs to cluster and generate pseudo labels for unlabeled samples. Then we fine-tune each of the multiple heterogeneous CNNs by minimizing an identification loss and a verification loss simultaneously, using training data with both true labels and pseudo labels. The procedure is iterated until the estimation of pseudo labels no longer changes. Extensive experiments on three large-scale person Re-Id datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract Most current fingerprint indexing systems are based on minutiae-only local structures and index local features directly. For minutiae local structure, missing and spurious neighboring minutiae significantly degrade the retrieval accuracy. To overcome this issue, we employs deep convolutional neural network to learn a minutia descriptor representing the local ridge structures. Instead of indexing local features, we aggregate various number of learned Minutia-centred Deep Convolutional (MDC) features of one fingerprint into a fixed-length feature vector to improve retrieval efficiency. In this paper, a novel aggregating method is proposed, which employs 1-D convolutional neural network to learn a discriminative and compact representation of fingerprint. In order to understand the MDC feature, a steerable fingerprint generation method is proposed to verify that it describes the attributes of minutiae and ridges. Comprehensive experimental results on five benchmark databases show that the proposed method achieves better performance on accuracy and efficiency than other prominent approaches. [ABSTRACT FROM AUTHOR]

Highlights • We present a new framework of deep convolutional neural network architecture distillation, namely LightweightNet, for acceleration and compression. • We exploit the prior knowledge of pre-defined network architecture to guide the efficient design of acceleration/compression strategies, while not using pre-trained model. • The proposed framework consists of network parameter compression, network structure acceleration, and non-tensor layer improvement. • The proposed framework demonstrates a higher acceleration/compression rate than previous methods in experiments, including a large category handwritten Chinese character recognition task with state-of-the-art performance. Abstract In recent years, deep neural networks have achieved remarkable successes in many pattern recognition tasks. However, the high computational cost and large memory overhead hinder them from applications on resource-limited devices. To address this problem, many deep network acceleration and compression methods have been proposed. One group of methods adopt decomposition and pruning techniques to accelerate and compress a pre-trained model. Another group designs single compact unit to stack their own networks. These methods are subject to complicated training processes, or lack of generality and extensibility. In this paper, we propose a general framework of architecture distillation, namely LightweightNet, to accelerate and compress convolutional neural networks. Rather than compressing a pre-trained model, we directly construct the lightweight network based on a baseline network architecture. The LightweightNet, designed based on a comprehensive analysis of the network architecture, consists of network parameter compression, network structure acceleration, and non-tensor layer improvement. Specifically, we propose the strategy of low-dimensional features of fully-connected layers for substantial memory saving, and design multiple efficient compact blocks to distill convolutional layers of baseline network with accuracy-sensitive distillation rule for notable time saving. Finally, it can effectively reduce the computational cost and the model size by > 4 × with negligible accuracy loss. Benchmarks on MNIST, CIFAR-10, ImageNet and HCCR (handwritten Chinese character recognition) datasets demonstrate the advantages of the proposed framework in terms of speed, performance, storage and training process. In HCCR, our method even outperforms traditional handcrafted features-based classifiers in terms of speed and storage while maintaining state-of-the-art recognition performance. [ABSTRACT FROM AUTHOR]

Multi-label classification has been successfully applied to image annotation, information retrieval, text categorization, etc. When the number of classes increases significantly, the traditional multi-label learning models will become computationally impractical. Label space dimension reduction (LSDR) is then developed to alleviate the effect of the high dimensionality of labels. However, almost all the existing LSDR methods focus on single-view learning. In this paper, we develop a multi-view label embedding (MVLE) model by exploiting the multi-view correlations. The label space and feature space of each view are bridged by a latent space. To exploit the consensus among different views, multi-view latent spaces are correlated by Hilbert–Schmidt independence criterion(HSIC). For a test sample, it is firstly embedded to the latent space of each view and then projected to the label space. The prediction is conducted by combining the multi-view outputs. Experiments on benchmark databases show that MVLE outperforms the state-of-the-art LSDR algorithms in both multi-view settings and different multi-view learning strategies. [ABSTRACT FROM AUTHOR]

Mining negative sequential patterns (NSP) is an important tool for nonoccurring behavior analysis, and it is much more challenging than mining positive sequential patterns (PSPs) due to the high computational complexity and huge search space when obtaining the support of negative sequential candidates (NSCs). Very few NSP mining algorithms are available and most of them are very inefficient since they obtain the support of NSC by scanning the database repeatedly. Instead, the state-of-the-art NSP mining algorithm e-NSP only uses the PSP’s information stored in an array structure to ‘calculate’ the support of NSC by equations, without database re-scanning. This makes e-NSP highly efficient, particularly on sparse datasets. However, when datasets become dense, the key process to obtain the support of NSC in e-NSP becomes very time-consuming and needs to be improved. In this paper, we propose a novel and efficient data structure, a bitmap , to obtain the support of NSC. We correspondingly propose a fast NSP mining algorithm, f-NSP, which uses a bitmap to store the PSP’s information and then obtain the support of NSC only by bitwise operations, which is much faster than the hash method in e-NSP. Experimental results on real-world and synthetic datasets show that f-NSP is not only tens to hundreds of times faster than e-NSP, but also saves more than ten-fold the storage spaces of e-NSP, particularly on dense datasets with a large number of elements in a sequence or a small number of itemsets. Further, we find that f-NSP consumes more storage space than e-NSP when PSP’s support is less than a support threshold sdsup , a value obtained through our theoretical analysis of storage space. Accordingly, we propose a self-adaptive storage strategy and a corresponding algorithm f-NSP + to overcome this deficiency. f-NSP + can automatically choose a bitmap or an array structure to store PSP information according to PSP support. Experimental results show that f-NSP + saves more storage spaces of f-NSP, and has similar time efficiency as f-NSP. [ABSTRACT FROM AUTHOR]

Cross-modal hashing has drawn increasing research interests in multimedia retrieval due to the explosive growth of multimedia big data. It is such a challenging topic due to the heterogeneity gap and high storage cost. However, most of the previous methods based on conventional linear projections and relaxation scheme fail to capture the nonlinear relationship among samples and suffers from large quantization loss, which result in an unsatisfactory performance of cross-modal retrieval. To address these issues, this paper is dedicated to learning discrete nonlinear hash functions by deep learning. A novel framework of cross-modal deep neural networks is proposed to learn binary codes directly. We formulate the similarity preserving in the framework, and also bit-independent as well as binary constraints are imposed on the hash codes. Specifically, we consider intra-modality similarity preserving at each hidden layer of the networks. Inter-modality similarity preserving is formulated by the output of each individual network. By so doing, the cross correlation can be encoded into the network training (i.e. hash functions learning) by back propagation algorithm. The final objective is solved by alternative optimization in an iterative fashion. Experimental results on four datasets i.e. NUS-WIDE, MIR Flickr, Pascal VOC, and LabelMe demonstrate the effectiveness of the proposed method, which is significantly superior to state-of-the-art cross-modal hashing approaches. [ABSTRACT FROM AUTHOR]

Clustering is an important technique to deal with large scale data which are explosively created in internet. Most data are high-dimensional with a lot of noise, which brings great challenges to retrieval, classification and understanding. No current existing approach is “optimal” for large scale data. For example, DBSCAN requires O ( n 2 ) time, Fast-DBSCAN only works well in 2 dimensions, and ρ -Approximate DBSCAN runs in O ( n ) expected time which needs dimension D to be a relative small constant for the linear running time to hold. However, we prove theoretically and experimentally that ρ -Approximate DBSCAN degenerates to an O ( n 2 ) algorithm in very high dimension such that 2 D  >  >  n . In this paper, we propose a novel local neighborhood searching technique, and apply it to improve DBSCAN, named as NQ-DBSCAN, such that a large number of unnecessary distance computations can be effectively reduced. Theoretical analysis and experimental results show that NQ-DBSCAN averagely runs in O ( n * log ( n )) with the help of indexing technique, and the best case is O ( n ) if proper parameters are used, which makes it suitable for many realtime data. [ABSTRACT FROM AUTHOR]

• Existing Visual Semantic Embedding (VSE) networks are trained by a hard negatives loss function that learns an objective margin between the similarity of relevant and irrelevant image–description embedding pairs and ignores the semantic differences between the irrelevant pairs. • We propose a novel Semantically Enhanced Hard negatives Loss function (LSEH) for Cross-modal Information Retrieval that considers the semantic differences between irrelevant training pairs. • The proposed LSEH function dynamically adjusts the learning objectives of VSE networks to make their learning flexible and efficient. • Experiments with various benchmark datasets and VSE networks revealed that the proposed LSEH function reduces their training epochs by approximately 50% and also improves their retrieval performance. Visual Semantic Embedding (VSE) networks aim to extract the semantics of images and their descriptions and embed them into the same latent space for cross-modal information retrieval. Most existing VSE networks are trained by adopting a hard negatives loss function which learns an objective margin between the similarity of relevant and irrelevant image–description embedding pairs. However, the objective margin in the hard negatives loss function is set as a fixed hyperparameter that ignores the semantic differences of the irrelevant image–description pairs. To address the challenge of measuring the optimal similarities between image–description pairs before obtaining the trained VSE networks, this paper presents a novel approach that comprises two main parts: (1) finds the underlying semantics of image descriptions; and (2) proposes a novel semantically-enhanced hard negatives loss function, where the learning objective is dynamically determined based on the optimal similarity scores between irrelevant image–description pairs. Extensive experiments were carried out by integrating the proposed methods into five state-of-the-art VSE networks that were applied to three benchmark datasets for cross-modal information retrieval tasks. The results revealed that the proposed methods achieved the best performance and can also be adopted by existing and future VSE networks. [ABSTRACT FROM AUTHOR]

• MCGCN for the first time builds cross-modal graph and jointly learns modality-specific and modality-shared features for semi-supervised cross-modal hashing. • MCGCN provides a three-channel network architecture, including two modality-specific channels and a cross-modal channel to model cross-modal graph with heterogeneous image and text features. • To effectively reduce the modality gap, network training is guided by adversarial scheme. • MCGCN obtains state-of-the-art semi-supervised cross-modal hashing performance. Cross-modal hashing maps heterogeneous multimedia data into Hamming space for retrieving relevant samples across modalities, which has received great research interests due to its rapid retrieval and low storage cost. In real-world applications, due to high manual annotation cost of multi-media data, we can only make use of limited number of labeled data with rich unlabeled data. In recent years, several semi-supervised cross-modal hashing (SCH) methods have been presented. However, how to fully explore and jointly utilize the modality-specific (complementarity) and modality-shared (correlation) information for retrieval has not been well studied for existing SCH works. In this paper, we propose a novel SCH approach named Modality-specific and Cross-modal Graph Convolutional Networks (MCGCN). The network architecture contains two modality-specific channels and a cross-modal channel to learn modality-specific and shared representations for each modality, respectively. Graph convolutional network (GCN) is leveraged in these three channels to explore intra-modal and inter-modal similarity, and perform semantic information propagation from labeled data to unlabeled data. Modality-specific and shared representations for each modality are fused with attention scheme. To further reduce the modality gap, a discriminative model is designed, learning to classify the modality of representations, and network training is guided by adversarial scheme. Experiments on two widely used multi-modal datasets demonstrate MCGCN outperforms state-of-the-art semi-supervised/supervised cross-modal hashing methods. [ABSTRACT FROM AUTHOR]

Multi-view data with each view corresponding to a type of feature set are common in real world. Usually, previous multi-view learning methods assume complete views. However, multi-view data are often incomplete, namely some samples have incomplete feature sets. Besides, most data are unlabeled due to a large cost of manual annotation, which makes learning of such data a challenging problem. In this paper, we propose a novel subspace learning framework for incomplete and unlabeled multi-view data. The model directly optimizes the class indicator matrix, which establishes a bridge for incomplete feature sets. Besides, feature selection is considered to deal with high dimensional and noisy features. Furthermore, the inter-view and intra-view data similarities are preserved to enhance the model. To these ends, an objective is developed along with an efficient optimization strategy. Finally, extensive experiments are conducted for multi-view clustering and cross-modal retrieval, achieving the state-of-the-art performance under various settings. [ABSTRACT FROM AUTHOR]

Eye iris has been widely recognized as one of the strongest biometrics attributed to its high accuracy performance. However, templates in conventional iris recognition systems are unprotected and highly vulnerable to numerous security and privacy attacks. Despite a number of cancellable biometric schemes have been proposed but at the expense of substantially decreased accuracy performance. In this paper, we introduce a new cancellable iris scheme, coined as “Indexing-First-One” (IFO) hashing. IFO hashing is inspired from the Min-hashing that primarily used in text retrieval domain. However, IFO hashing has been further strengthened by two novel mechanisms, namely P-order Hadamard product and modulo threshold function. The IFO hashing scheme strikes the balance between accuracy performance and privacy/security protection. Comprehensive experiments on CASIA-v3 iris benchmark database and rigorous analysis demonstrate decent accuracy performance with respect to its original counterparts yet offer strong resilience against several major security and privacy attacks. [ABSTRACT FROM AUTHOR]

In many visual classification tasks finding semantically meaningful regions has been confirmed as an effective solution. This paper aims to improve the performance of action classification in still images by introducing a discriminative region selection method. We observed that humans have certain periodic or symmetric pairs and they are critical for recognition. We also demonstrate that in action classification semantically meaningful regions are close to their periodic or symmetric parts and propose a model called a Generalized Symmetric Pair Model. By learning a max margin classifier, this method could identify regions around periodic or symmetric pairs without detection techniques. The method utilizes both the characteristics of actions and knowledge regarding periodism and symmetry to improve the popular bag-of-words (BoW) framework. We evaluate our method on five challenging action classification datasets. Experiments show that our method outperforms the state-of-the-art on four datasets. Qualitative visualization also demonstrate that the proposed method indeed identify semantically meaningful regions. [ABSTRACT FROM AUTHOR]

• We achieve effective deep hash retrieval by mapping and reversely mapping feature in multiple-GANs framework to simultaneously reduce storage cost truly and to obtain satisfactory user acceptance on the basis of acceptable retrieval precision. • We propose supervised manifold similarity to obtain better retrieval performance including retrieval precision and user acceptance followed by detailed demonstration. • We prove that Poisson distribution induced by tremendous hash codes can be initialized as generative distribution to fit real distribution. As an extension, any additive distribution can be utilized to initialize generative distribution to fit real distribution. • Experiments show that BH2I-GAN yields competitive retrieval performance comparing with state-of-the-art hashing methods, and obtains significant storage reduction as well as high-quality reconstruction from hash code. Besides, all retrieved images locate in the neighborhood of queries, which makes satisfactory user acceptance. Given the benefits of high retrieval efficiency and low storage cost, hashing method has received an increasing attention. In particular, deep learning-based hashing has been widely used in data mining and information retrieval. However, almost all the existing methods only achieve the goal of high retrieval precision, and limit the evaluation of hashing methods to objective aspect. In this paper, we propose a novel bidirectional hash_code-to-image translation model by using multi-generative multi-adversarial nets to reduce storage cost truly and obtain satisfactory user acceptance on the basis of high retrieval precision. Firstly, we propose supervised manifold metric to reduce Hamming distance between similar instances while increasing the Hamming distance between dissimilar instances, which have been proved to be helpful for high retrieval precision and good user acceptance. Then, we utilize multi-generative and multi-adversarial networks to construct hash mapping and inverse hash generation. During inverse generation, theoretical analysis is conducted to show that inverse hash network can avoid unstable training and mode collapse. Besides, we prove that Poisson distribution induced by hash codes can be initialized as generative distribution to fit real distribution. Experimental results show that our method outperforms several state-of-the-art approaches on three popular datasets. Specifically, ours yields average about 9.3% increment in Mean Average Precision(MAP) on three datasets, and achieves over 90% user satisfaction. Besides, it successfully reduces storage cost by 1,634 times in COCO 2017 large-scale dataset. [ABSTRACT FROM AUTHOR]

Abstract: In computer vision, camera calibration is a necessary process when the retrieval of information such as angles and distances is required. This paper addresses the multi-camera calibration problem with a single dimension calibration pattern under general motions. Currently, the known algorithms for solving this problem are based on the estimation of vanishing points. However, this estimate is very susceptible to noise, making the methods unsuitable for practical applications. Instead, this paper presents a new calibration algorithm, where the cameras are divided into binocular sets. The fundamental matrix of each binocular set is then estimated, allowing to perform a projective calibration of each camera. Then, the calibration is updated for the Euclidean space, ending the process. The calibration is possible without imposing any restrictions on the movement of the pattern and without any prior information about the cameras or motion. Experiments on synthetic and real images validate the new method and show that its accuracy makes it suitable also for practical applications. [Copyright &y& Elsevier]

• In consideration of privacy protection, we are the first to explore the universal attack against deep video hashing based on the UAH. • We also consider the data-free scenario and propose a method to perform the universal attack without access to the original videos. • We conduct extensive experiments to verify the effectiveness of our method under both white-box and black-box settings. While online video sharing becomes more popular, it also causes unconscious leakage of personal information in the video retrieval systems like deep hashing. A snoop can collect more users' private information from the video database by querying similar videos. This paper focuses on bypassing the deep video hashing based retrieval to prevent information from being maliciously collected. We propose universal adversarial head (UAH), which crafts adversarial query videos by prepending the original videos with a sequence of adversarial frames to perturb the normal hash codes in the Hamming space. This adversarial head can be generated only with a few natural videos, and mislead the retrieval system to return irrelevant videos when it is applied to most query videos. Furthermore, to obey the principle of information protection, we expand the proposed method to a data-free paradigm to generate the UAH, without access to users' original videos. Extensive experiments demonstrate the effectiveness of our method in misleading deep video hashing under both white-box and black-box settings. [ABSTRACT FROM AUTHOR]

Abstract: Intra-shape deformations complicate 3D shape recognition and therefore need proper modeling. Thereto, an isometric deformation model is used in this paper. The method proposed does not need explicit point correspondences for the comparison of 3D shapes. The geodesic distance matrix is used as an isometry-invariant shape representation. Two approaches are described to arrive at a sampling order invariant shape descriptor: the histogram of geodesic distance matrix values and the set of largest singular values of the geodesic distance matrix. Shape comparison is performed by comparison of the shape descriptors using the as dissimilarity measure. For object recognition, the results obtained demonstrate the singular value approach to outperform the histogram-based approach, as well as the state-of-the-art multidimensional scaling technique, the ICP baseline algorithm and other isometric deformation modeling methods found in literature. Using the TOSCA database, a rank-1 recognition rate of 100% is obtained for the identification scenario, while the verification experiments are characterized by a 1.58% equal error rate. External validation demonstrates that the singular value approach outperforms all other participants for the non-rigid object retrieval contests in SHREC 2010 as well as SHREC 2011. For 3D face recognition, the rank-1 recognition rate is 61.9% and the equal error rate is 11.8% on the BU-3DFE database. This decreased performance is attributed to the fact that the isometric deformation assumption only holds to a limited extent for facial expressions. This is also demonstrated in this paper. [Copyright &y& Elsevier]

Abstract: This paper presents a framework for the indexing and retrieval of artwork 3D models, allowing global and partial model classification and retrieval. The first part of the paper deals with database classification based on global shape descriptors. A search engine “RETIN-3D”, using a SVM classifier coupled with an active learning strategy allows to retrieve categories of similar objects. In a second part, the classification is improved thanks to a local description of the models. A new framework for 3D surface segmentation is proposed. Shape descriptors are adapted to surface regions and kernels on descriptor bags are used to perform the database classification. Our system is designed for classifying and retrieving in ancient artwork 3D databases, and results from this application domain are presented and commented along the paper. [ABSTRACT FROM AUTHOR]

Abstract: Video retrieval is increasingly based on image content. A number of studies on video retrieval have used low-level pixel content related to statistical moments, shape, colour and texture. However, it is well recognised that such information is not enough for uniquely discriminating across different multimedia content. The use of semantic information, especially which derived from spatio-temporal analysis is of great value in multimedia annotation, archiving and retrieval. In this review paper, we detail how the use of spatiotemporal semantic knowledge is changing the way in which modern research the conducted. In this paper we review a number of studies and concepts related to such analysis, and draw important conclusions on where future research is headed. [Copyright &y& Elsevier]

Abstract: In this paper, a content-based image retrieval system is built based on the image local descriptors and database indexing and searching techniques. The proposed system is designed for a powerful computer system-the ReMIX platform. The ReMIX system manages hundreds of GB memories and a number of power CPUs. Using this ReMIX system, the image retrieval over a large database becomes an easy and efficient task. In this paper, firstly, the image local descriptors, the ReMIX system''s architectures and multi-dimensional database indexing and searching techniques are described. Secondly, the specific database searching strategies based on the ReMIX system are explained. Thirdly, due to the hardware requirement of the ReMIX platform, integer instead of float or double is the best data format for hardware implementation. Therefore, the novel float-to-integer conversion is presented with the step by step procedures. Finally, in the experimental results, the comparisons among using float values and integer values and also using different integer conversion parameters are presented. Several images generated with large image variations are queried into a large database to demonstrate the image retrieval power of the ReMIX system. From the results, it is clear that the ReMIX system is a powerful tool for the content-based image retrieval. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a novel visual similarity measuring technique to retrieve face images in photo album databases for law enforcement. Though much work is being done on face similarity matching techniques, little attention is given to the design of face matching schemes suitable for visual retrieval in single model databases where accuracy, robustness to scale and environmental changes, and computational efficiency are three important issues to be considered. This paper presents a robust face retrieval approach using structural and spatial point correspondence in which the directional corner points (DCPs) are generated for efficient face coding and retrieval. A complete investigation on the proposed method is conducted, which covers face retrieval under controlled/ideal condition, scale variations, environmental changes and subject actions. The system performance is compared with the performance of the eigenface method. It is an attractive finding that the proposed DCP retrieval technique has performed superior to the eigenface method in most of the comparison experiments. This research demonstrates that the proposed DCP approach provides a new way, which is both robust to scale and environmental changes, and efficient in computation, for retrieving human faces in single model databases. [Copyright &y& Elsevier]

In this paper we describe a collaborative filtering system for automatically recommending high-quality information to users with similar interests on arbitrarily narrow information domains. It asks a user to rate a gauge set of items. It then evaluates the user''s rates and suggests a recommendation set of items. We interpret the process of evaluation as an inference mechanism that maps a gauge set to a recommendation set. We accomplish the mapping with fuzzy associative memory. We implemented the suggested system in a Web server and tested its performance in the domain of retrieval of technical papers, especially in the field of information technologies. The experimental results show that it may provide reliable recommendations. [Copyright &y& Elsevier]

Information retrieval in historical documents has long consisted in spotting words. In this paper, we focus on graphical pattern spotting. Contrary to object detection and classification, where models of the object of interest may be trained, pattern spotting does not rely on any prior information regarding the query, nor predefined class of graphical objects. An offline sliding window approach may be suitable, provided that the challenge raised by high computational and storage costs is handled. We propose an unsupervised, segmentation-free approach that takes advantage of recent developments in computer vision to overcome these issues. We also investigate the use of new, compact descriptors for the data, namely the vectors of locally aggregated descriptors (VLAD) and Fisher Vectors, instead of the usual bag-of-visual-words approach. Results obtained on medieval manuscripts from the DocExplore project show that our approach achieves better retrieval results, with a better efficiency in terms of time/memory, compared to standard approaches. Experimentations show that VLAD and Fisher Vectors can be fruitfully used in the future for the description of historical documents. Additionally, we show that our system can be easily turned into a word spotting system with slight adaptation, and that it achieves results comparable to those recently published in ICDAR 2015 keyword spotting challenge. [ABSTRACT FROM AUTHOR]

This paper presents a novel Content-Based Video Retrieval approach in order to cope with the semantic gap challenge by means of latent topics. Firstly, a supervised topic model is proposed to transform the classical retrieval approach into a class discovery problem. Subsequently, a new probabilistic ranking function is deduced from that model to tackle the semantic gap between low-level features and high-level concepts. Finally, a short-term relevance feedback scheme is defined where queries can be initialised with samples from inside or outside the database. Several retrieval simulations have been carried out using three databases and seven different ranking functions to test the performance of the presented approach. Experiments revealed that the proposed ranking function is able to provide a competitive advantage within the content-based retrieval field. [ABSTRACT FROM AUTHOR]

Keyword spotting is an alternative method for retrieving user specified text or query word images, without Optical Character Recognition (OCR), by calculating the similarity between features of word images rather than ASCII content. However, because of unconstrained writing styles with large variations, the retrieving results are always not very satisfactory. In this paper, we propose a novel method, which is based on Heat Kernel Signature (HKS) and triangular mesh structure to achieve handwritten word image matching. HKS can tolerate large variations in handwritten word images and capture local features. On the other hand, the triangular mesh structure is used to present global characteristics. Moreover, our method does not need pre-processing steps. [ABSTRACT FROM AUTHOR]

Identifying individuals in photographs of animals collected over time is a non-invasive approach for ecological monitoring and conservation. This paper describes the design and use of Sloop, the first image retrieval system for individual animal identification incorporating crowd-sourced relevance feedback. Sloop׳s iterative retrieval strategy using hierarchical and aggregated matching and relevance feedback consistently improves deformation and correspondence-based approaches for individual identification across several species. Its crowdsourcing strategy is successful in utilizing relevance feedback on a large scale. Sloop is in operational use. The user experience and results are presented here to facilitate the creation of a community-based individual identification system for conservation planning. [ABSTRACT FROM AUTHOR]

In this paper, an automatic image–text alignment algorithm is developed to achieve more effective indexing and retrieval of large-scale web images by aligning web images with their most relevant auxiliary text terms or phrases. First, a large number of cross-media web pages (which contain web images and their auxiliary texts) are crawled and segmented into a set of image–text pairs (informative web images and their associated text terms or phrases). Second, near-duplicate image clustering is used to group large-scale web images into a set of clusters of near-duplicate images according to their visual similarities. The near-duplicate web images in the same cluster share similar semantics and are simultaneously associated with a same or similar set of auxiliary text terms or phrases which co-occur frequently in the relevant text blocks, thus performing near-duplicate image clustering can significantly reduce the uncertainty on the relatedness between the semantics of web images and their auxiliary text terms or phrases. Finally, random walk is performed over a phrase correlation network to achieve more precise image–text alignment by refining the relevance scores between the web images and their auxiliary text terms or phrases. Our experiments on algorithm evaluation have achieved very positive results on large-scale cross-media web pages. [ABSTRACT FROM AUTHOR]

Abstract: In the database retrieval and nearest neighbor classification tasks, the two basic problems are to represent the query and database objects, and to learn the ranking scores of the database objects to the query. Many studies have been conducted for the representation learning and the ranking score learning problems, however, they are always learned independently from each other. In this paper, we argue that there are some inner relationships between the representation and ranking of database objects, and try to investigate their relationships by learning them in a unified way. To this end, we proposed the Unified framework for Representation and Ranking (UR2) of objects for the database retrieval and nearest neighbor classification tasks. The learning of representation parameter and the ranking scores are modeled within one single unified objective function. The objective function is optimized alternately with regard to representation parameter and the ranking scores. Based on the optimization results, iterative algorithms are developed to learn the representation parameter and the ranking scores on a unified way. Moreover, with two different formulas of representation (feature selection and subspace learning), we give two versions of UR2. The proposed algorithms are tested on two challenging tasks – MRI image based brain tumor retrieval and nearest neighbor classification based protein identification. The experiments show the advantage of the proposed unified framework over the state-of-the-art independent representation and ranking methods. [Copyright &y& Elsevier]

Abstract: The goal of object retrieval is to rank a set of images by the similarity of their contents to those of a query image. However, it is difficult to measure image content similarity due to visual changes caused by varying viewpoint and environment. In this paper, we propose a simple, efficient method to more effectively measure content similarity from image measurements. Our method is based on the ranking information available from existing retrieval systems. We observe that images within the set which, when used as queries, yield similar ranking lists are likely to be relevant to each other and vice versa. In our method, ranking consistency is used as a verification method to efficiently refine an existing ranking list, in much the same fashion that spatial verification is employed. The efficiency of our method is achieved by a list-wise min-Hash scheme, which allows rapid calculation of an approximate similarity ranking. Experimental results demonstrate the effectiveness of the proposed framework and its applications. [Copyright &y& Elsevier]

Abstract: Latent topic model such as Latent Dirichlet Allocation (LDA) has been designed for text processing and has also demonstrated success in the task of audio related processing. The main idea behind LDA assumes that the words of each document arise from a mixture of topics, each of which is a multinomial distribution over the vocabulary. When applying the original LDA to process continuous data, the word-like unit need be first generated by vector quantization (VQ). This data discretization usually results in information loss. To overcome this shortage, this paper introduces a new topic model named Gaussian-LDA for audio retrieval. In the proposed model, we consider continuous emission probability, Gaussian instead of multinomial distribution. This new topic model skips the vector quantization and directly models each topic as a Gaussian distribution over audio features. It avoids discretization by this way and integrates the procedure of clustering. The experiments of audio retrieval demonstrate that Gaussian-LDA achieves better performance than other compared methods. [Copyright &y& Elsevier]

Abstract: With multimedia information retrieval, combining different modalities – text, image, audio or video provides additional information and generally improves the overall system performance. For this purpose, the linear combination method is presented as simple, flexible and effective. However, it requires to choose the weight assigned to each modality. This issue is still an open problem and is addressed in this paper. Our approach, based on Fisher Linear Discriminant Analysis, aims to learn these weights for multimedia documents composed of text and images. Text and images are both represented with the classical bag-of-words model. Our method was tested over the ImageCLEF datasets 2008 and 2009. Results demonstrate that our combination approach not only outperforms the use of the single textual modality but provides a nearly optimal learning of the weights with an efficient computation. Moreover, it is pointed out that the method allows to combine more than two modalities without increasing the complexity and thus the computing time. [Copyright &y& Elsevier]

Abstract: In this paper, a unified framework for multimodal content retrieval is presented. The proposed framework supports retrieval of rich media objects as unified sets of different modalities (image, audio, 3D, video and text) by efficiently combining all monomodal heterogeneous similarities to a global one according to an automatic weighting scheme. Then, a multimodal space is constructed to capture the semantic correlations among multiple modalities. In contrast to existing techniques, the proposed method is also able to handle external multimodal queries, by embedding them to the already constructed multimodal space, following a space mapping procedure of a submanifold analysis. In our experiments with five real multimodal datasets, we show the superiority of the proposed approach against competitive methods. [Copyright &y& Elsevier]

Abstract: Most clustering algorithms organize a collection of objects into a set of disjoint clusters. Although this approach has been successfully applied in unsupervised learning, there are several applications where objects could belong to more than one cluster. Overlapping clustering is an alternative in those contexts like social network analysis, information retrieval and bioinformatics, among other problems where non-disjoint clusters appear. In addition, there are environments where the collection changes frequently and the clustering must be updated; however, most of the existing overlapping clustering algorithms are not able to efficiently update the clustering. In this paper, we introduce a new overlapping clustering algorithm, called DClustR, which is based on the graph theory approach and it introduces a new strategy for building more accurate overlapping clusters than those built by state-of-the-art algorithms. Moreover, our algorithm introduces a new strategy for efficiently updating the clustering when the collection changes. The experimentation conducted over several standard collections shows the good performance of the proposed algorithm, wrt. accuracy and efficiency. [Copyright &y& Elsevier]

Abstract: 2D cartoon plays an important role in many areas, but it requires effective methods to relieve manual labors. In this paper, we propose a heterogeneous cartoon gesture recognition method with applications. Firstly, heterogeneous features with different dimensions are assigned to express cartoon and human-subject images according to their characteristics. Then for recognition, we simultaneously integrate shared structure learning (SSL) and graph-based transductive learning into a joint framework to learn reliable classifiers on heterogeneous features. Provided with the framework, the similarities between cartoon and human-subject gestures can be quantitatively evaluated in a cross-feature manner. Extensive experiments on self-defined datasets have demonstrated the effectiveness of our method. Finally, applications illustrate the usages in various aspects of 2D cartoon industry. [Copyright &y& Elsevier]

Abstract: Recent technological advances have enhanced the computer-based indexing and searching of digitised printed books. The performance now achievable in this domain, however, does not at present extend to handwritten texts which inherently contain more significant letter-based variation within their content. Furthermore, in most studies that address the handwritten text retrieval problem, a large training dataset is required which, very often, influences the context and search lexicon. In this paper a novel method is described to overcome the training data problem using a character-based modelling (termed grapheme spectrum) approach and a word modelling technique (termed synthesised word) enabling the retrieval of keywords that have not explicitly been seen in the training set. When tested on an illustrative historical manuscript the performance of the proposed word retrieval technique shows a clear advantage over existing methods. [Copyright &y& Elsevier]

Abstract: Novelty detection is an important functionality that has found many applications in information retrieval and processing. In this paper we propose a novel framework that deals with novelty detection in multiple-scene image sets. Working with wildlife image data, the framework starts with image segmentation, followed by feature extraction and classification of the image blocks extracted from image segments. The labelled image blocks are then scanned through to generate a co-occurrence matrix of object labels, representing the semantic context within the scene. The semantic co-occurrence matrices then undergo binarization and principal component analysis for dimension reduction, forming the basis for constructing one-class models on scene categories. An algorithm for outliers detection that employs multiple one-class models is proposed. An advantage of our approach is that it can be used for novelty detection and scene classification at the same time. Our experiments show that the proposed approach algorithm gives favourable performance for the task of detecting novel wildlife scenes, and binarization of the semantic co-occurrence matrices helps increase the robustness to variations of scene statistics. [Copyright &y& Elsevier]

Abstract: In this paper, two perceptually motivated morphological strategies (PMMS) are proposed to enhance the retrieval performance of common shape matching methods. Firstly, two human perception customs are introduced, which have important relations to shape retrieval. Secondly, these two customs are properly modeled by morphological operations. Finally, the proposed PMMS is applied to improve the retrieval performances of a popular shape matching method named Inner-Distance Shape Contexts (IDSC), and then the Locally Constrained Diffusion Process (LCDP) method is exploited to further enhance the retrieval performance. This combination achieves a retrieval rate of 98.56% on MPEG-7 dataset. We also conduct the experiments on Swedish Leaf dataset, the ETH-80 dataset and the Natural Silhouette dataset. The experimental results obtained from four datasets demonstrate clearly the effectiveness of the proposed method. [Copyright &y& Elsevier]

Abstract: Canonical correlation analysis (CCA) is a popular and powerful dimensionality reduction method to analyze paired multi-view data. However, when facing semi-paired and semi-supervised multi-view data which widely exist in real-world problems, CCA usually performs poorly due to its requirement of data pairing between different views and un-supervision in nature. Recently, several extensions of CCA have been proposed, however, they just handle the semi-paired scenario by utilizing structure information in each view or just deal with semi-supervised scenario by incorporating the discriminant information. In this paper, we present a general dimensionality reduction framework for semi-paired and semi-supervised multi-view data which naturally generalizes existing related works by using different kinds of prior information. Based on the framework, we develop a novel dimensionality reduction method, termed as semi-paired and semi-supervised generalized correlation analysis (S2GCA). S2GCA exploits a small amount of paired data to perform CCA and at the same time, utilizes both the global structural information captured from the unlabeled data and the local discriminative information captured from the limited labeled data to compensate the limited pairedness. Consequently, S2GCA can find the directions which make not only maximal correlation between the paired data but also maximal separability of the labeled data. Experimental results on artificial and four real-world datasets show its effectiveness compared to the existing related dimensionality reduction methods. [Copyright &y& Elsevier]

Abstract: Based on the gradient flows in Lie group, a partial retrieval approach for CAD models is presented in this paper. First, a representation of the face Attributed Relational Graph (ARG) for a CAD model is created from its B-rep model and thus partial retrieval is converted to a subgraph matching problem. Then, an optimization method is adopted to solve the matching problem, where the optimization variable is the vertex mapping and the objective function is the measurement of compatibility between the mapped vertices and between the mapped edges. Different from most previously proposed methods, a homogeneous transformation matrix is introduced to represent the vertex mapping in subgraph matching, whose translational sub-matrix gives the vertex selection in the larger graph and whose orthogonal sub-matrix presents the vertex permutation for the same-sized mapping from the selected vertices to the smaller graph''s vertices. Finally, a gradient flow method is developed to search for optimal matching matrix in Special Euclidean group SE(n). Here, a penalty approach is used to handle the constraints on the elements of the matching matrix, which leads its orthogonal part to be a permutation matrix and its translational part to have different integer elements. Experimental results show that it is a promising method to support the partial retrieval of CAD models. [Copyright &y& Elsevier]

Abstract: This paper presents a novel ranking framework for content-based multimedia information retrieval (CBMIR). The framework introduces relevance features and a new ranking scheme. Each relevance feature measures the relevance of an instance with respect to a profile of the targeted multimedia database. We show that the task of CBMIR can be done more effectively using the relevance features than the original features. Furthermore, additional performance gain is achieved by incorporating our new ranking scheme which modifies instance rankings based on the weighted average of relevance feature values. Experiments on image and music databases validate the efficacy and efficiency of the proposed framework. [Copyright &y& Elsevier]

Abstract: In content-based image retrieval context, a classic strategy consists in computing off-line a dictionary of visual features. This visual dictionary is then used to provide a new representation of the data which should ease any task of classification or retrieval. This strategy, based on past research works in text retrieval, is suitable for the context of batch learning, when a large training set can be built either by using a strong prior knowledge of data semantics (like for textual data) or with an expensive off-line pre-computation. Such an approach has major drawbacks in the context of interactive retrieval, where the user iteratively builds the training data set in a semi-supervised approach by providing positive and negative annotations to the system in the relevance feedback loop. The training set is thus built for each retrieval session without any prior knowledge about the concepts of interest for this session. We propose a completely different approach to build the dictionary on-line from features extracted in relevant images. We design the corresponding kernel function, which is learnt during the retrieval session. For each new label, the kernel function is updated with a complexity linear with respect to the size of the database. We propose an efficient active learning strategy for the weakly supervised retrieval method developed in this paper. Moreover this framework allows the combination of features of different types. Experiments are carried out on standard databases, and show that a small dictionary can be dynamically extracted from the features with better performances than a global one. [Copyright &y& Elsevier]

Abstract: Visual search reranking involves an optimization process that uses visual content to recover the “genuine” ranking list from the helpful but noisy one generated by textual search. This paper presents an evolutionary approach, called Adaptive Particle Swarm Optimization (APSO), for unsupervised visual search reranking. The proposed approach incorporates the visual consistency regularization and the ranking list distance. In addition, to address the problem that existing list distance fails to capture the genuine disagreement between two ranking lists, we propose a numerical ranking list distance. Furthermore, the parameters in APSO are self-tuned adaptively according to the fitness values of the particles to avoid being trapped in local optima. We conduct extensive experiments on automatic search task over TRECVID 2006–2007 benchmarks and show significant and consistent improvements over state-of-the-art works. [Copyright &y& Elsevier]

Abstract: This paper proposes a method for binary image retrieval, where the black-and-white image is represented by a novel feature named the adaptive hierarchical density histogram, which exploits the distribution of the image points on a two-dimensional area. This adaptive hierarchical decomposition technique employs the estimation of point density histograms of image regions, which are determined by a pyramidal grid that is recursively updated through the calculation of image geometric centroids. The extracted descriptor combines global and local properties and can be used in variant types of binary image databases. The validity of the introduced method, which demonstrates high accuracy, low computational cost and scalability, is both theoretically and experimentally shown, while comparison with several other prevailing approaches demonstrates its performance. [Copyright &y& Elsevier]

Abstract: Image segmentation is an important tool in image processing and can serve as an efficient front end to sophisticated algorithms and thereby simplify subsequent processing. In this paper, we present a color image segmentation using pixel wise support vector machine (SVM) classification. Firstly, the pixel-level color feature and texture feature of the image, which is used as input of SVM model (classifier), are extracted via the local homogeneity model and Gabor filter. Then, the SVM model (classifier) is trained by using FCM with the extracted pixel-level features. Finally, the color image is segmented with the trained SVM model (classifier). This image segmentation not only can fully take advantage of the local information of color image, but also the ability of SVM classifier. Experimental evidence shows that the proposed method has a very effective segmentation results and computational behavior, and decreases the time and increases the quality of color image segmentation in comparison with the state-of-the-art segmentation methods recently proposed in the literature. [Copyright &y& Elsevier]