Search

Poor retrieval performance significantly degenerates users' experience of visual search, especially in mobile search. Ideally, users would like to be alerted when bad queries are present, which helps eliminate latency as well as waste of bandwidth, especially in 3G wireless environment. In this paper, we propose a visual query performance prediction (v-QPP) approach to predict the retrieval effectiveness. We employ latent dirichlet allocation (LDA)to derive latent topics from image database. From the collection statistics, we model the query's specificity based on topics. High specificity helps a retrieval system to derive user's search intent exactly. Moreover, as low discriminative content is difficult to search in terms of distinguishing relevant images from irrelevant one, we propose a topics based inverse concept frequency (t-ICF) model to deal with specific queries but difficult to discriminate in the reference database. Comparison experiments over MPEG CDVS benchmarking datasets have shown our method significantly outperforms existing approaches in document retrieval. [ABSTRACT FROM PUBLISHER]

Coming with the proliferation of mobile devices, mobile visual search emerges. One fundamental issue here is the query transmission latency, especially in a bandwidth constraint wireless link. Towards low bit rate retrieval, recent works have proposed to extract compact visual descriptors directly on the mobile end, where the vocabulary tree based bag-of-words representation has shown superior performance in producing compact descriptors [2][9]. However, the corresponding tree-structure vector quantizer is extremely large against a mobile end implementation. In this paper, we propose two alternatives to prune this tree structure based on the subtree discriminability analysis, where either information gain based or ranking based pruning are investigated. Furthermore, we have unveiled that the tree structure can be even discarded while retaining only the discriminative leaves together with their radii in practice. We evaluate our tree pruning on Android HTC Desire G7, with application to low bit rate mobile landmark search in a 10-million landmark photo collection, where over 10 scale memory reduction with almost identical search accuracy is reported. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a compact yet discriminative local descriptor which tackles the wireless query transmission latency in mobile visual search. The descriptor captures gradient statistics of canonical patches over a log-polar location grid whose parameters are optimized using training samples. We quantize the resulting descriptor using product quantization. The descriptor achieves about 95% bits reduction compared with 128-Byte SIFT and allows adaptation of descriptor lengths to support user required performance. Moreover, accurate matching of descriptors with low complexity is allowed within several table lookup operations. We perform a comprehensive comparison with SIFT, GLOH and CHoG in the context of image retrieval, image matching and object localization. We achieve competing matching and retrieval performance with SIFT, GLOH with much fewer bits. In particular, the descriptor outperforms CHoG at the same bits on eight data sets contributed to MPEG Compact Descriptor for Visual Search(CDVS) Standardization. [ABSTRACT FROM PUBLISHER]

Compressing a query image's signature via vocabulary coding is an effective approach to low bit rate mobile visual search. State-of-the-art methods concentrate on offline learning a codebook from an initial large vocabulary. Over a large heterogeneous reference database, learning a single codebook may not suffice for maximally removing redundant codewords for vocabulary based compact descriptor. In this paper, we propose to learn multiple codebooks (m-Codebooks) for extremely compressing image signatures. A query-specific codebook (q-Codebook) is online generated at both client and server sides by adaptively weighting the off-line learned multiple codebooks. The q-Codebook is subsequently employed to quantize the query image for producing compact, discriminative, and scalable descriptors. As q-Codebook may be simultaneously generated at both sides, without transmitting the entire vocabulary, only small overhead (e.g. codebook ID and codeword 0/1 index) is incurred to reconstruct the query signature at the server end. To fulfill m-Codebooks and q-Codebook, we adopt a Bi-layer Sparse Coding method to learn the sparse relationships of codewords vs. codebooks as well as codebooks vs. query images via l1 regularization. Experiments on benchmarking datasets have demonstrated the extremely small descriptor's supervior performance in image retrieval. [ABSTRACT FROM PUBLISHER]

We introduce a Compact Topical Descriptor to learn a compact yet discriminative image signature from the reference image corpus. This descriptor is deployed over the well used bag-of-words image histogram, with two merits over the traditional topical features: First, we propose to directly control the topical sparsity to achieve the descriptor compactness. Second, we ensure the descriptor discriminability by minimizing the bag-of-words reconstruction errors during the topical histogram encoding. To this end, we have a generative viewpoint of the topical feature extraction, which is estimated as a sparse MAP estimation over the original bag-of-words. We learn such estimation by a bi-convex optimization, iterating between both hierarchical sparse coding from words to topical histograms and dictionary learning of the corresponding word-to-topic transform. Especially, supervised labels such as image ranking list can be also incorporated into our descriptor learning paradigm. We quantize our performance in both Im-ageNet 10K and NUS-WIDE, with comparisons to bag-of-words, LDA, miniBoF, and Aggregated Local Descriptors. In practice, we also implement our descriptor for a low bit rate mobile visual search application, i.e. sending compact descriptors instead of the image to reduce the query delivery latency. Our descriptor has significantly outperformed the state-of-the-art compact descriptors by quantitative evaluations over 10 million reference images. [ABSTRACT FROM PUBLISHER]

We propose a novel locality sensitive vocabulary coding scheme to extract compact descriptors for low bit rate visual search. We employ Latent Dirichlet Allocation (LDA) to learn the topic vocabularies of lower dimension to generate compact descriptors. To deal with diverse datasets, LDA model is introduced to subdivide a dataset into groups of images with a topic model, where the code word distributions in each group produce coherent statistics from generative learning. Moreover, our empirical study has shown that the original Bag-of-Word (BoW) is sparse, and the occurrences of non-zero words is coherent within a topic. Our proposed topic-wise vocabulary learning yields a more compact yet discriminative codebook to search images. Given a query image, multiple topics are determined, which is fed into the topic vocabularies to generate more compact topical descriptor. Comparison experiments show our topic-wise locality sensitive vocabulary coding produces more compact and discriminative descriptors than the state-of-the-arts. [ABSTRACT FROM PUBLISHER]