Search

Objectives Our study is a cross-sectional, qualitative study analyzing pelvic organ prolapse-related content on Pinterest, a social media platform that also functions as a visual search engine. Primary outcomes assessed include quality, understandability, and actionability of content. Secondary outcomes include levels of misinformation and commercial bias. Our study further characterizes sources of prolapse content and treatment modalities discussed. Methods A Pinterest search was conducted using the term "pelvic organ prolapse," analyzing the first 100 relevant results. We examined the publisher of each pin and the writer of linked content. Using validated tools including the Patient Education Materials Assessment Tool and DISCERN criteria, we assessed understandability and actionability of content. We further compared content to professional guidelines for discussing and treating prolapse, evaluating for misinformation. We also evaluated for commercial bias and subjectively rated pin quality. Results Of pins evaluated, low understandability was found in 16, low actionability in 30, and low-to-moderate quality in 69. The most common publisher of pins was health and wellness groups, and the most common writer of content was physical therapists. Pelvic floor muscle training (PFMT) is the treatment modality discussed most frequently on Pinterest. Conclusions Pinterest content discussing pelvic organ prolapse shows good levels of understandability for patients at all levels of health literacy. However, the number of pins showing poor actionability raises concern that content may not encourage readers to take direct steps toward prolapse assessment and management. In all, a majority of prolapse-related content on Pinterest was of low-to-moderate quality.

Chapbooks were short, cheap printed booklets produced in large quantities in Scotland, England, Ireland, North America and much of Europe between roughly the seventeenth and nineteenth centuries. A form of popular literature containing songs, stories, poems, games, riddles, religious writings and other content designed to appeal to a wide readership, they were frequently illustrated, particularly on their title-pages. This paper describes the visual analysis of such chapbook illustrations. We automatically extract all the illustrations contained in the National Library of Scotland Chapbooks Printed in Scotland dataset, and create a visual search engine to search this dataset using full or part-illustrations as queries. We also cluster these illustrations based on their visual content, and provide keyword-based search of the metadata associated with each publication. The visual search; clustering of illustrations based on visual content; and metadata search features enable researchers to forensically analyse the chapbooks dataset and to discover unnoticed relationships between its elements. We release all annotations and software tools described in this paper to enable reproduction of the results presented and to allow extension of the methodology described to datasets of a similar nature.

Vizualne tražilice pronalaze subjekt u slici te kao rezultat daju više informacija o tom subjektu. To se može dizajnirati kao traženje drugih slika koje sadrže taj subjekt. Kako bi se pronašao subjekt u slici, moguće je koristiti model za detekciju predmeta, te pomoću korisnika odabrati subjekt. Potrebno je izvući značajke iz slika među kojima će se pretraživati one najpogodnije. To se može postiću koristeći model za izvlačenje značajki. Dimenzionalnost istih se smanjuje pomoću modela za smanjenje dimenzionalnosti kako bi se uštedilo na memoriji te kako bi se čak poboljšala preciznost. Te značajke se zatim indeksiraju kako bi ih se lakše pronašlo. Upitna slika također prolazi kroz proces izvlačenja značajki i smanjenje dimenzionalnosti. Naposlijetku, algoritam zaslužen za indeksiranje također nalazi najsličnije značajke upitnoj slici te se dotične slike prikazuju korisniku. Visual search engines consist of searching the subject of interest and providing more results about the subject to the user, which can be designed as searching for images that contain that subject. The way to extract the subject of interest in the image is to use an object detection model and choose the subject with the help of the user. From the images from which the retrieval will be done have to be extracted some features. This can be done using a feature extraction model. To save some memory and even have better accuracy, the features dimensionality gets reduced using a dimensionality reduction model. Before saving them, they get indexed for easier retrieval. The query image also goes though feature extraction and dimensionality reduction. Finally, the indexing algorithm retrieves the result features, and the respective images get shown to the user.

In this paper, we present Generic Object Detection (GenOD), one of the largest object detection systems deployed to a web-scale general visual search engine that can detect over 900 categories for all Microsoft Bing Visual Search queries in near real-time. It acts as a fundamental visual query understanding service that provides object-centric information and shows gains in multiple production scenarios, improving upon domain-specific models. We discuss the challenges of collecting data, training, deploying and updating such a large-scale object detection model with multiple dependencies. We discuss a data collection pipeline that reduces per-bounding box labeling cost by 81.5% and latency by 61.2% while improving on annotation quality. We show that GenOD can improve weighted average precision by over 20% compared to multiple domain-specific models. We also improve the model update agility by nearly 2 times with the proposed disjoint detector training compared to joint fine-tuning. Finally we demonstrate how GenOD benefits visual search applications by significantly improving object-level search relevance by 54.9% and user engagement by 59.9%., In Proceedings of the 27th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD) 2021, Virtual Event, Singapore

This paper introduces the large scale visual search algorithm and system infrastructure at Alibaba. The following challenges are discussed under the E-commercial circumstance at Alibaba (a) how to handle heterogeneous image data and bridge the gap between real-shot images from user query and the online images. (b) how to deal with large scale indexing for massive updating data. (c) how to train deep models for effective feature representation without huge human annotations. (d) how to improve the user engagement by considering the quality of the content. We take advantage of large image collection of Alibaba and state-of-the-art deep learning techniques to perform visual search at scale. We present solutions and implementation details to overcome those problems and also share our learnings from building such a large scale commercial visual search engine. Specifically, model and search-based fusion approach is introduced to effectively predict categories. Also, we propose a deep CNN model for joint detection and feature learning by mining user click behavior. The binary index engine is designed to scale up indexing without compromising recall and precision. Finally, we apply all the stages into an end-to-end system architecture, which can simultaneously achieve highly efficient and scalable performance adapting to real-shot images. Extensive experiments demonstrate the advancement of each module in our system. We hope visual search at Alibaba becomes more widely incorporated into today's commercial applications., accepted by KDD 2018

This paper presents details of our frame-based Ad-hoc Video Search system with manually assisted querying that will be used for the Video Browser Showdown 2021 (VBS2021). The main contributions of our new system consist of an improved automatic keywording component, better visual feature vectors which have been fine-tuned for the task of image retrieval, and an improved visual presentation of the search results. Additionally, we use a more powerful joint textual/visual search engine based on Lucene, which can perform a search according to the temporal sequence of textual or visual properties of the video frames.

From the time of prehistoric etchings on the walls of the Lascaux cave to the present day, people have always been creating art. With millions of artistic artifacts filling museums, churches, cultural institutions, and private collections across the globe, connecting to our shared cultural and artistic past is no longer impossible.

Web search engines play a significant role in many applications in our everyday lives. Among them, online shopping is one major technological advancement which has made our life easier and comfortable. Currently, most e-commerce websites support either text-based or voice-based search. The problem with text and voice-based approaches is that they need an appropriate item name or description for the search results to be accurate. Also, with the huge variety of items available online, it is not easy to find the desired object in the top results. Lately, some top e-commerce websites started supporting visual search, where the user can submit an image of an item they'd like to find. However, this domain is still in its infancy. In this work, we propose ViSeR, a visual search engine architecture using deep learning and machine learnging techniques, with a proof-of-concept implementation focusing on the fashion eretail industry. ViSeR first classifies the query image to the right category using image classification. Then all the images in that category are ranked based on their similarity and the top images are retrieved as recommendations. We present in detail the experimental results with different deep and machine learning algorithms and provide additional details with regards to deploying this model to achieve high accuracy and low latency (in terms of training and recommendation time).

We present a visual search engine for graphics such as math, chemical diagrams, and figures. Graphics are represented using Line-of-Sight (LOS) graphs, with symbols connected only when they can ‘see’ each other along an unobstructed line. Symbol identities may be provided (e.g., in PDF) or taken from Optical Character Recognition applied to images. Graphics are indexed by pairs of symbols that ‘see’ each other using their labels, spatial displacement, and size ratio. Retrieval has two layers: the first matches query symbol pairs in an inverted index, while the second aligns candidates with the query and scores the resulting matches using the identity and relative position of symbols. For PDFs, we also introduce a new tool that quickly extracts characters and their locations. We have applied our model to the NTCIR-12 Wikipedia Formula Browsing Task, and found that the method can locate relevant matches without unification of symbols or using a math expression grammar. In the future, one might index LOS graphs for entire pages and search for text and graphics. Our source code has been made publicly available.

We introduce the problem of content-based retrieval for moments in videogames. This new area for artificial intelligence in games exercises automated gameplay and visual understanding while making connections to information retrieval. We propose a number of techniques to discover the interesting moments in a game (crawling), show how to compress moments into an efficiently searchable structure (indexing), and recall those moments most relevant to a user-provided query (retrieving). We combine these ideas in a prototype visual search engine and compare it with commercial visual search engines. Searching within a corpus of moments from Super Nintendo Entertainment System games using query images extracted from YouTube videos, our prototype is able to identify moments that web-oriented search engines rarely see.

To fill a gap in online educational tools, we are working to support search in lecture videos using formulas from lecture notes and vice versa. We use an existing system to convert single-shot lecture videos to keyframe images that capture whiteboard contents along with the times they appear. We train classifiers for handwritten symbols using the CROHME dataset, and for LATEX symbols using generated images. Symbols detected in video keyframes and LATEX formula images are indexed using Line-of-Sight graphs. For search, we lookup pairs of symbols that can 'see' each other, and connected pairs are merged to identify the largest match within each indexed image. We rank matches using symbol class probabilities and angles between symbol pairs. We demonstrate how our method effectively locates formulas between typeset and handwritten images using a set of linear algebra lectures. By combining our search engine Tangent-V) with temporal keyframe metadata, we are able to navigate to where a query formula in LATEX is first handwritten in a lecture video. Our system is available as open-source. For other domains, only the OCR modules require updating.

Introduction In Chapter 5 we saw that clustering search engines such as Carrot2 were able to display results in a non-linear form, by the use of folders and grouping like results together. There are various advantages to this approach, such as preventing searchers from thinking that the first result is in some way ‘the best’. A logical next step on from this category is to be able to see the websites themselves directly in the search results, rather than having to click on a link to go to them. The tile approach is also becoming more popular, based on the increased use of mobile devices. This chapter will look at search engines that use this approach to display results. However, that's only one side of the visual search coin – rather than seeing search results portrayed in a visual format you might actually just want to search for images themselves (which of course has a very visual component in the way that results are displayed) and this chapter will also look at some image search options in detail. A visual search engine will display results by showing searchers the actual web pages themselves that are returned from the search that's been run. Some engines (particularly free-text) may include a thumbnail of a screenshot, but that's more to simply help identify a site, rather than offering any practical use. However, those in the visual search engines category (and it should be admitted that it's a very small category) do rather more than that – the results are the web pages themselves. Searchers can f lick from one screen to the next and get a very good indication of what's on the web pages from the SERPs, without going directly to the sites in question. There are a number of advantages to this approach; users can view a page safe in the knowledge that no one at the website will know that they are looking at it, which is helpful when it comes to competitive intelligence.

The ‘bag of visual features’ image representation was applied to create generic microstructural signatures that can be used to automatically find relationships in large and diverse microstructural image data sets. Using this representation, a support vector machine (SVM) was trained to classify microstructures into one of seven groups with greater than 80% accuracy over 5-fold cross validation. In addition, the bag of visual features was implemented as the basis for a visual search engine that determines the best matches for a query image in a database of microstructures. These novel applications demonstrate the potential and the limitations of computer vision concepts in microstructural science.

Universal adversarial perturbations (UAPs), a.k.a. input-agnostic perturbations, has been proved to exist and be able to fool cutting-edge deep learning models on most of the data samples. Existing UAP methods mainly focus on attacking image classification models. Nevertheless, little attention has been paid to attacking image retrieval systems. In this paper, we make the first attempt in attacking image retrieval systems. Concretely, image retrieval attack is to make the retrieval system return irrelevant images to the query at the top ranking list. It plays an important role to corrupt the neighbourhood relationships among features in image retrieval attack. To this end, we propose a novel method to generate retrieval-against UAP to break the neighbourhood relationships of image features via degrading the corresponding ranking metric. To expand the attack method to scenarios with varying input sizes or untouchable network parameters, a multi-scale random resizing scheme and a ranking distillation strategy are proposed. We evaluate the proposed method on four widely-used image retrieval datasets, and report a significant performance drop in terms of different metrics, such as mAP and mP@10. Finally, we test our attack methods on the real-world visual search engine, i.e., Google Images, which demonstrates the practical potentials of our methods., Comment: ICCV 2019

Due to the rapid growth of medical images, user-specific ROI and object classification are the significant factors in the region-based segmentation instead of a pixel-based segmentation. Manual image annotation, classification, and filtering are not only a difficult task, but also high memory and time usage. In the visual search system, an unknown query image was given as input, relevant visual images with different diagnoses features are retrieved and then used as clinical decisions. The main goal of the visual search engine is to efficiently retrieve user-specific images that are visually identical to a selected ROI query. In this paper, a survey on traditional visual search methods is analyzed in terms of visual features and accuracy are concerned. Based on the survey performed by different visual search systems, the diagnostic efficiency is increased from 30 to 60% for clinical decision.

The objective of this work is a scalable, real-time visual search engine for medical images. In contrast to existing systems that retrieve images that are globally similar to a query image, we enable the user to select a query Region Of Interest (ROI) and automatically detect the corresponding regions within all returned images. This allows the returned images to be ranked on the content of the ROI, rather than the entire image. Our contribution is two-fold: (i) immediate retrieval - the data is appropriately pre-processed so that the search engine returns results in real-time for any query image and ROI; (ii) structured output - returning ROIs with a choice of ranking functions. The retrieval performance is assessed on a number of annotated queries for images from the IRMA X-ray dataset and compared to a baseline.

Nowadays, traditional search engines such as Google, Yahoo and Bing facilitate the retrieval of information in the format of images, but the results are not always useful for the users. This is mainly due to two problems: (1) the semantic keywords are not taken into consideration and (2) it is not always possible to establish a query using the image features. This issue has been covered in different domains in order to develop content-based image retrieval (CBIR) systems. The expert community has focussed their attention on the healthcare domain, where a lot of visual information for medical analysis is available. This paper provides a solution called iPixel Visual Search Engine, which involves semantics and content issues in order to search for digitized mammograms. iPixel offers the possibility of retrieving mammogram features using collective intelligence and implementing a CBIR algorithm. Our proposal compares not only features with similar semantic meaning, but also visual features. In this sense, the comparisons are made in different ways: by the number of regions per image, by maximum and minimum size of regions per image and by average intensity level of each region. iPixel Visual Search Engine supports the medical community in differential diagnoses related to the diseases of the breast. The iPixel Visual Search Engine has been validated by experts in the healthcare domain, such as radiologists, in addition to experts in digital image analysis.

We demonstrate that, with the availability of distributed computation platforms such as Amazon Web Services and open-source tools, it is possible for a small engineering team to build, launch and maintain a cost-effective, large-scale visual search system with widely available tools. We also demonstrate, through a comprehensive set of live experiments at Pinterest, that content recommendation powered by visual search improve user engagement. By sharing our implementation details and the experiences learned from launching a commercial visual search engines from scratch, we hope visual search are more widely incorporated into today's commercial applications., in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge and Discovery and Data Mining, 2015

International audience; Speeding up the collection and integration of raw botanical observation data is a crucial step towards a sustainable development of agriculture and the conservation of biodiversity. Initiated in the context of a citizen sciences project, the main contribution of this paper is an innovative collaborative workflow focused on image-based plants identification as a mean to enlist new contributors and facilitate access to botanical data. Since 2010, hundreds of thousands of geo-tagged and dated plantphotographs were collected and revised by hundreds of novice, amateur and expert botanists of a specialized social network. An image-based identification tool - available as both a web and a mobile application - is synchronized with that growing data and allows any user to query or enrich the system with new observations. An important originality is that it works with up to five different organs contrarily to previous approaches that mainly relied on the leaf. This allows querying the system at any period of the year and with complementary images composing a plant observation. Extensive experiments of the visual search engine as well as system-oriented and user-oriented evaluations of the application show that it is already very helpful to determine a plant among hundreds or thousands of species. At the time of writing, the whole framework covers about half of the plant species living in France (2200 species), which already makes it the widest existing automated identification tool(with its imperfections).

Museums, libraries, national archives and art galleries deal with visual objects that must be made accessible to a wide variety of experts or non-experts like researchers, art lovers or interested people. The ability to identify objects sharing some aspect of visual similarity can be useful when trying to trace historical influences or when looking for further examples of paintings, sculptures or other cultural objects appealing to their taste. In this direction, we use our recent detector [1] for an image retrieval task on a subset of Europeana's1 content. The detector produces distinctive features by grouping sampled image edges according to proper shape stability measures. We evaluate the detector by integrating it into the Visual Search Engine for Europeana (Vieu)2 tool.

The objective of this work is a scalable, real-time, visual search engine for 3-D medical images, where a user is able to select a query Region Of Interest (ROI) and automatically detect the corresponding regions within all returned images. We make three contributions: (i) we show that with appropriate off-line processing, images can be retrieved and ROIs registered in real time; (ii) we propose and evaluate a number of scalable exemplar-based image registration schemes; (iii) we propose a discriminative method for learning to rank the returned images based on the content of the ROI. The retrieval system is demonstrated on MRI data from the ADNI dataset, and it is shown that the learnt ranking function outperforms the baseline.

International audience; We develop a system for 3D object retrieval based on sketched feature lines as input. For objective evaluation, we collect a large number of query sketches from human users that are related to an existing data base of objects. The sketches turn out to be generally quite abstract with large local and global deviations from the original shape. Based on this observation, we decide to use a bag-of-features approach over computer generated line drawings of the objects. We develop a targeted feature transform based on Gabor filters for this system. We can show objectively that this transform is better suited than other approaches from the literature developed for similar tasks. Moreover, we demonstrate how to optimize the parameters of our, as well as other approaches, based on the gathered sketches. In the resulting comparison, our approach is significantly better than any other system described so far.

This work presents a browser that supports two strategies for video browsing: the navigation through visual hierarchies and the retrieval of similar images. The input videos are firstly processed by a keyframe extractor to reduce the temporal redundancy and decrease the number of elements to consider. These generated keyframes are hierarchically clustered with the Hierachical Cellular Tree (HCT) algorithm, an indexing technique that also allows the creation of data structures suitable for browsing. Different clustering criteria are available, in the current implementation, based on four MPEG-7 visual descriptors computed at the global scale. The navigation can directly drive the user to find the video timestamps that best match the query or to a keyframe which is globally similar in visual terms to the query. In the latter case, a visual search engine is also available to find other similar keyframes, based as well on MPEG-7 visual descriptors.

Improving the relevancy of Web search results has been of increasing interest in recent years. The nature of the Web implies heterogeneity, large volumes, and varied structures. Hence, finding results that best suit the needs of every individual is a very challenging problem. Accordingly, interactive graphical and visualization techniques are suggested to increase the ability of the display to handle large numbers of results while simultaneously presenting several attributes for each Web page. In addition, query reformulation and reconstruction is usually controlled by the search engine. Consequently, the results suffer from redundancy and/or irrelevancy. Integrating the user in the process of query reformulation-by visualizing the process itself-may benefit the overall search relevance. This paper presents an interactive Visual Search Engine (the VSE) in which both processes of query reformulation and results presentation are visualized. In the user study, the effectiveness of the VSE was demonstrated when compared to Google.

In this paper, we evaluate a visual search engine that is designed to solve the real problem of browsing and searching for documents in a vast repository of colleges/courses located at Western Kentucky University (WKU). The architectural design of this interface combines Formal Concept Analysis (FCA) with Semantic Factoring to decompose complex, vast concepts into their primitives in order to develop knowledge representation for the HyperManyMedia plat-form. The main objective of this study is to test: (a) the efficiency of ranking the documents using precision, and (b) the usability of the visual search engine. This approach has been implemented and used by online students at WKU.

This report is an evaluation of if it is good or bad to use a metaphor in order to display the results of an academic search engine in a web interface. In order to evaluate this we are describing our work with developing two different web interfaces for an academic search engine by the name Silverfish. This project has been a co-operation between Indian Institute of Information Technol- ogy in Bangalore, India and Malmˆ University K3, Sweden. To start our report we describe how we see our context we are to work within. We define our stakeholders as being academics worldwide and also define that we are working within a web 2.0 context. To strengthen our choices regarding the design process of the two different interfaces as well as in order to give more validity to our discussion surrounding metaphors we continue with presenting different studies and facts that give more weight to the above mentioned parts. To make it possible to create the interfaces we have made use of several methods. We give a short definition of how these methods are to be used and later describe in the design process how we have made use of them. To describe how we have made use of the methods as well as to describe how we have developed our prototypes we continue our report with describing the design process, regarding which deci- sions we have made and why we have made them. To summarize our report we come to a con- clusion regarding our thesis question; communicating related key phrases through web interface metaphors; good or bad? Regarding our question we have found that the orientational metaphor we are using does not work as it is supposed to. We believe that further studies are required in order to get a deeper un- derstanding of how the user understands the orientational metaphor we are using. This informa- tion could help us come to an understanding of how we could make better use of our orienta- tional metaphor, or help us find out of a metaphor that would be better to use than our orienta- tional metaphor.

We demonstrate ThemExplorer1:, a mash-up for geographic image retrieval. The application combines: Geonames, a geographic thesaurus; TagMaps, a tool for visualizing tags on a map; PIRIA - a visual search engine, and pictures collected from Flickr and Google images. The user can query ThemExplorer using both keywords and example images.

Nowadays there are many product comparison web sites. But most of them only use text information. This paper introduces a novel visual search engine for product images, which provides a brand-new way of visually locating products through Content-based Image Retrieval (CBIR) technology. We discusses the unique technical challenges, solutions, and experimental results in the design and implementation of this system.

In this paper we show how online images can be automatically exploited for scene visualization and reconstruction starting from a mere visual query provided by the user. A visual query is used to retrieve images of a landmark place using a visual search engine. These images are used to reconstruct robust 3nD features and camera poses in projective space. Novel views are then rendered corresponding to a virtual camera ying smoothly through the projective space by triangulation of the projected points in the output view. We introduce a method to fuse the rendered novel views from all input images at each virtual view point by computing their intrinsic image and illuminations. This approach allows us to remove the occlusions and maintain consistent and controlled illumination throughout the rendered sequence. We demonstrate the performance of our prototype system on two landmark structures.

We propose a Visual Search Engine (ViSE) as a semi-automatic component in a surveillance system using networked cameras. The ViSE aims to assist the monitoring operation of huge amounts of captured video streams, which tracks and finds people in the video based on their primitive features with the interaction of a human operator. We address the issues of object detection and tracking, shadow suppression and color-based recognition for the proposed system. The experimental results on a set of video data with ten subjects showed that ViSE retrieves correct candidates with 83% recall at 83% precision.

The Web provides a large repository of multimedia data, text, images, etc. Most current search engines focus on textural retrieval. In this paper, we focus on using an integrated textural and visual search engine for Web documents. We support query refinement which proves useful and enables cross-media browsing in addition to regular search.

This paper reports about an implementation of a search engine for visual information content, which has been developed in the context of the forthcoming MPEG-7 standard. The system supports similarity-based retrieval of visual (image and video) data along feature axes like color, texture, shape and geometry. The descriptors for these features have been developed in a way such that invariance against common transformations of visual material, e.g. filtering, contrast/color manipulation, resizing etc. is achieved, and that they are fitted to human perception properties. Furthermore, descriptors have been designed that allow a fast, hierarchical search procedure, where the inherent search mechanisms of database systems can be employed. This is important for client-server applications, where pre-selection should be performed at the database side. Database interfaces have been implemented in a platform-independent way based on SQL. The results show that efficient search and retrieval in distributed visual database systems is possible based on a normative feature description such as MPEG-7.

Online communities are helpful with many advantages for the deaf who are not able to hear or hear hard as it is for hearing people. Deaf people use sign language which is a visual form of communication including the combination of hand shapes, orientation and movement of the hands and facial expressions instead of voice which is used in spoken languages for communication. Furthermore, it is shown that the reading skills of the majority of the deaf society are poor. It is not hard to believe that the deaf will face problems while surfing web pages and also other Internet facilities like chat systems, voice mails and Internet telephony. In this paper, an online community is proposed in which specific facilities and characteristics are considered via a design especially for the deaf.

We study the query-based attack against image retrieval to evaluate its robustness against adversarial examples under the black-box setting, where the adversary only has query access to the top-k ranked unlabeled images from the database. Compared with query attacks in image classification, which produce adversaries according to the returned labels or confidence score, the challenge becomes even more prominent due to the difficulty in quantifying the attack effectiveness on the partial retrieved list. In this paper, we make the first attempt in Query-based Attack against Image Retrieval (QAIR), to completely subvert the top-k retrieval results. Specifically, a new relevance-based loss is designed to quantify the attack effects by measuring the set similarity on the top-k retrieval results before and after attacks and guide the gradient optimization. To further boost the attack efficiency, a recursive model stealing method is proposed to acquire transferable priors on the target model and generate the prior-guided gradients. Comprehensive experiments show that the proposed attack achieves a high attack success rate with few queries against the image retrieval systems under the black-box setting. The attack evaluations on the real-world visual search engine show that it successfully deceives a commercial system such as Bing Visual Search with 98% attack success rate by only 33 queries on average.

Increasingly, the ubiquity of satellite imagery has made the data analysis and machine learning of large geographical datasets one of the building blocks of visuospatial intelligence. It is the key to discover current (and predict future) cultural, social, financial and political realities. How can we, as designers and researchers, empower citizens to understand and participate in the design of our cities amid this technological shift? As an initial step towards this broader ambition, a series of creative web applications, in the form of visual search engines, has been developed and implemented to data mine large datasets. Using open sourced deep learning and computer vision libraries, these applications facilitate the searching, detecting and curating of urban objects. In turn, the paper proposes and formulates a framework to design truly citizen-centric creative visual search engines – a contribution to citizen science and citizen journalism in spatial terms.