Search

SOMHunter represents a modern light-weight framework for known-item search in datasets of visual data like images or videos. The framework combines an effective W2VV++ text-to-image search approach, a traditional Bayesian like model for maintenance of relevance scores influenced by positive examples, and several types of exploration and exploitation displays. With this initial setting in 2020, already the first prototype of the system turned out to be highly competitive in comparison with other state-of-the-art systems at Video Browser Showdown and Lifelog Search Challenge competitions. In this paper, we present a new version of the system further extending the list of visual data search capabilities. The new version combines localized text queries with collage queries tested at VBS 2021 in two separate systems by our team. Furthermore, the new version of SOMHunter will integrate also the new CLIP text search model recently released by OpenAI. We believe that all the extensions will improve chances to effectively initialize the search that can continue with already supported browsing capabilities.

In our information-driven societies, full-text search is ubiquitous. Search is memory-intensive. Quickly searching massive corpora requires building indices, which consumes big volatile heaps. Search is storage I/O-intensive. Limited main memory necessitates writing large partial indices on non-volatile storage, where they finally live in merged form. These indices reside in memory, in full or in part, during query evaluation. Memory and I/O intensity make it hard to index and search content rapidly and efficiently. On the hardware side, the recently introduced Intel Optane DC persistent memory (PM) offers byte-addressability, high capacity, and non-volatility. This paper evaluates and exploits Optane PM for text indexing and search on multicore platforms. We identify essential structures in inverted indices (hash table, merge tree, and key-value store), where they reside (memory or storage), and key operations over them (sort, flush, and merge). We allocate index structures in DRAM, Optane PM, and block storage by modifying an existing search engine. We then evaluate a myriad of hybrid memory and storage configurations. Our findings include: (1) careful placement of index structures across DRAM, Optane PM, and SSD, speeds up indexing with a single core compared to a high-performance baseline, but does not scale to many cores, (2) crash-consistent indexing with Optane PM is feasible without incurring a high overhead, and (3) the tail latency of the longest multi-term conjunctive queries is lower with a PM-backed index than an SSD-backed one. This paper opens up persistent memory to a practical role in full-text search.

SQLite with JSON (JavaScript Object Notation) format is widely adopted for local data storage in mobile applications such as Twitter and Instagram. With more data are generated and stored, it becomes vitally important to efficiently index and search JSON records in SQLite. However, current methods in SQLite either require full text search (that incurs big memory usage and long query latency) or indexing based on expression (that needs to be manually created by specifying search keys). On the other hand, existing JSON automatic indexing techniques, mainly focusing on big data and cloud environments, depend on a colossal tree structure that cannot be applied in memory-constrained mobile devices. In this paper, we propose a novel schema-agnostic indexing technique called LiteIndex that can automatically index JSON records by extracting keywords from long text and maintaining user-preferred items within a given memory constraint. This is achieved by memory-efficient index organization with light-weight keyword extraction from long text and user-preference-aware reinforcement-learning-based index pruning mechanism. LiteIndex has been implemented in a Android smartphone platform and evaluated with a dataset from Tweet. Experimental results show that LiteIndex can significantly reduce the query latency by up to 18x with less memory usage compared with SQLite with FTS3/FTS4 extensions.

The growing number of undergraduated thesis produced at the end of undergraduated courses in Brazil are sometimes available only in physical libraries or have limited digital access, making it difficult for researchers and the general population to access this information. This paper presents the development of a system that uses data analysis and information retrieval techniques capable of providing full-text search, as well as the recommendation of works based on their similarity using the Latent Dirichlet Allocation (LDA) technique. The goal of this solution is to offer higher education institutions in Brazil a platform capable of providing digital access to the documents they produced, enabling the dissemination of scientific production. The tool was validated using data from a real educational institution, and the search and recommendation mechanisms proved to be efficient for their purpose.

Keeping up with the rapid growth of Deep Learning (DL) research is a daunting task. While existing scientific literature search systems provide text search capabilities and can identify similar papers, gaining an in-depth understanding of a new approach or an application is much more complicated. Many publications leverage multiple modalities to convey their findings and spread their ideas - they include pseudocode, tables, images and diagrams in addition to text, and often make publicly accessible their implementations. It is important to be able to represent and query them as well. We utilize RDF Knowledge graphs (KGs) to represent multimodal information and enable expressive querying over modalities. In our demo we present an approach for extracting KGs from different modalities, namely text, architecture images and source code. We show how graph queries can be used to get insights into different facets (modalities) of a paper, and its associated code implementation. Our innovation lies in the multimodal nature of the KG we create. While our work is of direct interest to DL researchers and practitioners, our approaches can also be leveraged in other scientific domains.

As reported by respected evaluation campaigns focusing both on automated and interactive video search approaches, deep learning started to dominate the video retrieval area. However, the results are still not satisfactory for many types of search tasks focusing on high recall. To report on this challenging problem, we present two orthogonal task-based performance studies centered around the state-of-the-art W2VV++ query representation learning model for video retrieval. First, an ablation study is presented to investigate which components of the model are effective in two types of benchmark tasks focusing on high recall. Second, interactive search scenarios from the Video Browser Showdown are analyzed for two winning prototype systems implementing a selected variant of the model and providing additional querying and visualization components. The analysis of collected logs demonstrates that even with the state-of-the-art text search video retrieval model, it is still auspicious to integrate users into the search process for task types, where high recall is essential.

In the last decade, the Video Browser Showdown (VBS) became a comparative platform for various interactive video search tools competing in selected video retrieval tasks. However, the participation of new teams with an own, novel tool is prohibitively time-demanding because of the large number and complexity of components required for constructing a video search system from scratch. To partially alleviate this difficulty, we provide an open-source version of the lightweight known-item search system SOMHunter that competed successfully at VBS 2020. The system combines several features for text-based search initialization and browsing of large result sets; in particular a variant of W2VV++ model for text search, temporal queries for targeting sequences of frames, several types of displays including the eponymous self-organizing map view, and a feedback-based approach for maintaining the relevance scores inspired by PICHunter. The minimalistic, easily extensible implementation of SOMHunter should serve as a solid basis for constructing new search systems, thus facilitating easier exploration of new video retrieval ideas.

Traditional search through collections of audio recordings compares a text-based query to text metadata associated with each audio file and does not address the actual content of the audio. Text descriptions do not describe all aspects of the audio content in detail. Query by vocal imitation (QBV) is a kind of query by example that lets users imitate the content of the audio they seek, providing an alternative search method to traditional text search. Prior work proposed several neural networks, such as TL-IMINET, for QBV, however, previous systems have not been deployed in an actual search engine nor evaluated by real users. We have developed a state-of-the-art QBV system (Vroom!) and a baseline query-by-text search engine (TextSearch). We deployed both systems in an experimental framework to perform user experiments with Amazon Mechanical Turk (AMT) workers. Results showed that Vroom! received significantly higher search satisfaction ratings than TextSearch did for sound categories that were difficult for subjects to describe by text. Results also showed a better overall ease-of-use rating for Vroom! than TextSearch on the sound library used in our experiments. These findings suggest that QBV, as a complimentary search approach to existing text-based search, can improve both search results and user experience.

Inverted index serves as a fundamental data structure for efficient search across various applications such as full-text search engine, document analytics and other information retrieval systems. The storage requirement and query load for these structures have been growing at a rapid rate. Thus, an ideal indexing system should maintain a small index size with a low query processing time. Previous works have mainly focused on using CPUs and GPUs to exploit query parallelism while utilizing state-of-the-art compression schemes to fit the index in memory. However, scaling parallelism to maximally utilize memory bandwidth on these architectures is still challenging. In this work, we present IIU, a novel inverted index processing unit, to optimize the query performance while maintaining a low memory overhead for index storage. To this end, we co-design the indexing scheme and hardware accelerator so that the accelerator can process highly compressed inverted index at a high throughput. In addition, IIU provides flexible interconnects between modules to take advantage of both intra- and inter-query parallelism. Our evaluation using a cycle-level simulator demonstrates that IIU provides an average of 13.8\times× query latency reduction and 5.4\times× throughput improvement across different query types, while reducing the average energy consumption by 18.6\times×, compared to Apache Lucene, a production-grade full-text search framework.

Many modern recommender systems use collaborative filtering or historical engagement data to serve the best recommendations for each item. However, the context of each recommendation instance can be very different. Some users may be casually browsing, while others are searching with high intent. At Pinterest, we realized that building our system solely on aggregated historical data or pin-board collaborative filtering [1] would not be able to capture these differences. Incorporating contextual signals helps us serve better recommendations for every instance. Pinterest Related Pins is an item-to-item recommender system that accounts for 40 percent of engagement on Pinterest. [2] On Pinterest, Related Pins appears as a feed of content relevant to the Pin a user has clicked on. Users arrive at Related Pins feeds from a variety of surfaces, such as their Home Feed, Search results, or Boards. As expected, these users often have different intents. Users coming from Search have already executed a specific text query and clicked on one of the Pins in the Search results. This context tells us that the user has high intent and is interested in something related to both the Search query as well as the clicked Pin. This context is very different from a user who is casually scrolling through their Home Feed and clicks on a Pin that happens to catch their eye. The Related Pins recommendations for each of these clicked Pins should therefore also differ accordingly. Related Pins are generally relevant to the clicked Pin. The recommendations for a women's dress shoe Pin will be other shoes of similar style, some of which may be paired with matching outfits. However, if the user searched in particular for "red ballet flats with sequins," the Related Pins may not be specific enough to be useful to the user. In order to address this, we developed a hybrid search that takes both the text search query and the clicked pin image and metadata as inputs, and outputs a set of results tailored to both. We found that this improved user engagement for Related Pins from Search by 20% on top of the previous production recommendation system [2]. Following this exciting launch, we are planning to further incorporate contextual signals by adding them as features in our model.

There are various approaches to storing numeric IDs as extra channels within CG rendered images. Using these channels, individual objects can be selected and separately modified. To associate an object with a text string a table or manifest is required mapping numeric IDs to text strings. This allows readable identification of ID-based selections, as well as the ability to make a selection using a text search. A scheme for storage of this ID Manifest is proposed which is independent of the approach used to store the IDs within the image. The total size of the raw strings within an ID Manifest may be very large but often contains much repeated information. A novel compression scheme is therefore employed which significantly reduces the size of the manifest.

Aiming at the ElasticSearch search engine used in the research and demonstration platform of local chronicles visualization technology, the content searched only by the keyword in the local chronicles' full text search process is not accurate enough, a Lucene optimization algorithm combines the feature of the terms' sequence relationship is proposed after one of the core algorithm----Similarity Score Algorithm is reflected thoroughly by analyzing Lucene which is in the bottom of ElasticSearch. The experiment result shows that the algorithm proposed enhances the mean average precision for searching by 6 percentage points approximately compared with Luene default search algorithm, which helps to achieve accurate searching of the local chronicles' text.

The pervasive availability of voice assistants may support children in finding answers to informational queries by removing the literacy requirements of text search (e.g., typing, spelling). However, most such systems are not designed for the specific needs and preferences of children and may struggle with understanding the intent of their questions. In our investigation, we observed 87 children and 27 adults interacting with three Wizard-of-Oz speech interfaces to arrive at answers to questions that required reformulation. We found that many children and some adults required help to reach an effective question reformulation. We report the common types of reformulations (both effective and ineffective ones). We also compared three versions of speech interfaces with different approaches to referring to itself (personification) and to the participant (naming personalization). We found that children preferred personified interfaces, but naming personalization did not affect preference. We connect our findings to implications for design of speech systems for families.

This paper presents eKichabi, a tool for retrieving contact information for agriculture-related enterprises in Tanzania. eKichabi is an Unstructured Supplementary Service Data (USSD) application which users can access through basic mobile phones. We describe our focus groups, a design iteration, deployment in four villages, and follow up interviews by phone. This work demonstrates the feasibility of USSD for information access applications that have the potential for deployment on a large scale in the developing world. From user interviews, we identified strong evidence of eKichabi fulfilling an unmet need for business related information, both in identifying business contacts in other villages, as well locating specific service providers. One of our key findings demonstrates that users access information through multiple modes, including text search, in addition to menu navigation organized by both business sector category and geographic area.

Analysis of ancient Khmer documents can be quite challenging due to the elaborated shape of Khmer handwritten characters combined with the complex structure of how words are formed from those characters. Palm leaf manuscripts, one of the most well-known old Khmer documents, have been being digitized and centralized; therefore, document analysis functions such as text search capabilities are necessary but still remain unavailable for this type of documents. In order to contribute to the progress of relevant researches, we introduce in this paper a new dataset called SleukRith set comprising of 657 pages of Khmer palm leaf manuscripts randomly selected from various collections whose quality and digitization method are variable. The dataset contains three types of data: isolated characters, words, and lines. Each type of data is annotated with the ground truth information which is very useful for evaluating and serving as a training set for common document analysis tasks such as character/text recognition, word/line segmentation, and word spotting. In order to serve as a base line, the result of an evaluation study of Khmer isolated character recognition that we have conducted on SleukRith Set using Convolutional Neural Network is also presented.

We consider the problem of indexing the complete OpenStreetMap planet data set (> 500GB of raw data) to support complex queries involving both text search as well as context-aware spatial relations. This requires (a) formalization of spatial relations like 'north of', 'between', 'near' depending on the context, and (b) the development of suitable data representations to integrate textual and spatial information for efficient query performance.

We present QLever, a query engine for efficient combined search on a knowledge base and a text corpus, in which named entities from the knowledge base have been identified (that is, recognized and disambiguated). The query language is SPARQL extended by two QLever-specific predicates ql:contains-entity and ql:contains-word, which can express the occurrence of an entity or word (the object of the predicate) in a text record (the subject of the predicate). We evaluate QLever on two large datasets, including FACC (the ClueWeb12 corpus linked to Freebase). We compare against three state-of-the-art query engines for knowledge bases with varying support for text search: RDF-3X, Virtuoso, Broccoli. Query times are competitive and often faster on the pure SPARQL queries, and several orders of magnitude faster on the SPARQL+Text queries. Index size is larger for pure SPARQL queries, but smaller for SPARQL+Text queries.

To me, an awareness of history is a fundamental requirement for progress; and I believe that we in the field of information retrieval are currently ill-served in this domain, or at least not as aware as we should be. While it is true that a researcher in IR is expected to acquire some knowledge of what has gone before, this knowledge is typically fairly narrow in scope. there are some IR researchers who regard IR as a branch of computer science, for example, despite the fact that the field has a long and venerable history entirely outside the domain of computers, as well as a considerable current presence in academic departments well away from CSDs. Outside of our immediate community, there is a widespread belief that web search engines arose out of nothing, a totally new invention for the totally new world of the web. This is unfortunate. It is true that there were huge changes in the information retrieval world in the second half of the twentiethcentury. The ideas, methods and systems that were around in 2000 (particularly the early web search engines, up to and including Google) seem at first glance to be completely different from the ideas, methods and systems that were around in 1950. But these developments involved in large measure the usual process of evolution rather than revolution, and the usual mix of steps large or small, forwards or backwards or sideways. Some of these steps were taken in the world of IR research, and some in the domain of practical commercial systems.

In this study, we introduce a simple and high-speed search engine for large-scale system logs, called Hayabusa. Hayabusa uses SQLite, standard lightweight database software with GNU Parallel and general Linux commands, such that it can run efficiently without complex components. Network administrators can use Hayabusa to accumulate and store log information at high speeds and to search the logs quickly.In our experiments, Hayabusa required only 8 seconds to convert 1.2 M log messages into a database file. Moreover, Hayabusa required only 5 seconds to search a keyword from 1.7 billion records. Hayabusa achieved high-performance search speed in a stand-alone environment without a complex distributed environment. Compared with the distributed environment, Spark, the proposed stand-alone Hayabusa was approximately 27 times faster.

A fundamental concern in search engines is to determine which documents have the best content for satisfying the user, based on analysis of the user's query and the text of documents. For this query-content match, many learning to rank systems make use of IR features developed in the 1990s in the TREC framework. Such features are still important in a variety of search tasks, and particularly in the long tail where clicks, links and social media signals become sparse. I will present our current progress, in particular three different neural models, with the goal of surpassing the 1990s models in full text search. This will include evidence, using proprietary Bing datasets, that large-scale training data can be useful. I will also argue that for the field to make progress on query-content relevance modeling, it may be valuable to set up a shared blind evaluation similar to 1990s TREC, possibly with large-scale training data.

Technological advances in Head-Up Displays (HUDs) have renewed vehicle manufacturer interest. Recent research links time with eyes off of the road to increased chance of accidents, a problem that could be diminished when using HUDs. Before HUDs can be safely integrated, there must be proven methods of assessing these displays. NHTSA established guidelines for assessing in-vehicle tasks; however, these have not been examined with HUDs, differ from traditional Head-Down Displays (HDDs). This study followed most of the NHTSA guidelines for the Eye Glance Test for a visual text search task while tracking driving and secondary task performance measures. The HUD performed worse on the NHTSA eye glance test than the HDD did; however, the driving performance measures were superior when driving with the HUD. There were no significant differences in the secondary task performance between the two displays. Therefore, the NHTSA standard may not adequately assess HUDs in vehicles.

Recent advances in the technology of the Graphics Processing Unit (GPU) has led to a surge of interest in using the GPU for general purpose applications. We can utilize the GPU in computation as a massive parallel co-processor because the GPU consists of multiple cores. The GPU is also an affordable, attractive, and user-programmable commodity. Although the inverted index is a useful data structure that can be used for full text search or document retrieval, the large number of documents will require tremendous time to create the index. The performance of document inversion can be improved by multicore GPU. Our approach is to implement a linear-time, hash-based, single program multiple data (SPMD), document inversion algorithm on the NVIDIA GPU/CUDA programming platform utilizing the huge computational power of the GPU, to develop high performance solutions for document indexing.

The growing popularity of mobile search and the advancement in voice recognition technologies have opened the door for web search users to speak their queries, rather than type them. While this kind of voice search is still in its infancy, it is gradually becoming more widespread. In this paper, we examine the logs of a commercial search engine's mobile interface, and compare the spoken queries to the typed-in queries. We place special emphasis on the semantic and syntactic characteristics of the two types of queries. %Our analysis suggests that voice queries focus more on audio-visual content and question answering, and less on social networking and adult domains. We also conduct an empirical evaluation showing that the language of voice queries is closer to natural language than typed queries. Our analysis reveals further differences between voice and text search, which have implications for the design of future voice-enabled search tools.

This paper describes a new test collection for passage retrieval from multilingual, informal text. The task being modeled is that of a monolingual English-speaking user who wishes to search discussion forum text in a foreign language. The system retrieves relevant short passages of text and presents them to the user, translated into English. The test collection contains more than 2 billion words of discussion thread text, 250 queries representing complex informational search needs, and manual relevance judgments of forum post passages, pooled from real systems. This information retrieval test collection is the first to combine multilingual search, passage retrieval, and informal online genre text.

Enriched by natural language texts, Stack Overflow code snippets are an invaluable code-centric knowledge base of small units of source code. Besides being useful for software developers, these annotated snippets can potentially serve as the basis for automated tools that provide working code solutions to specific natural language queries. With the goal of developing automated tools with the Stack Overflow snippets and surrounding text, this paper investigates the following questions: (1) How usable are the Stack Overflow code snippets? and (2) When using text search engines for matching on the natural language questions and answers around the snippets, what percentage of the top results contain usable code snippets? A total of 3M code snippets are analyzed across four languages: C\#, Java, JavaScript, and Python. Python and JavaScript proved to be the languages for which the most code snippets are usable. Conversely, Java and C\# proved to be the languages with the lowest usability rate. Further qualitative analysis on usable Python snippets shows the characteristics of the answers that solve the original question. Finally, we use Google search to investigate the alignment of usability and the natural language annotations around code snippets, and explore how to make snippets in Stack Overflow an adequate base for future automatic program generation., 13th IEEE/ACM International Conference on Mining Software Repositories, 11 pages

Time-travel text search over a temporally evolving document collection is useful in various applications. Supporting a wide range of query classes demanded by these applications require different index layouts optimized for their respective query access patterns. The problem we tackle is how to efficiently handle different query classes using the same index layout. Our approach is to use list intersections on single-attribute indexes of keywords and temporal attributes. Although joint predicate evaluation on single-attribute indexes is inefficient in general, we show that partitioning the index based on version lifespans coupled with exploiting the transaction-time ordering of record-identifiers, can significantly reduce the cost of list intersections. We empirically evaluate different index partitioning alternatives on top of open-source Lucene, and show that our approach is the only technique that can simultaneously support a wide range of query classes efficiently, have high indexing throughput in a real-time ingestion setting, and also have negligible extra storage costs.

We investigate the use of text search to retrieve information visualizations. This is important as the body of available visualizations on the web is growing. And it the meantime, they are difficult to find as they don't have an immediate textual description. Our approach is first to create a taxonomy of textual terms to describe visualizations, both in a general and in a specific way using economics visualization as application domain. Then we designed and implemented a search engine to query this vocabulary on a real website, The Atlas of Economic Complexity, using economics data. Results from an exploratory study informed us on the types of tasks to support and on the visual design of the search box, such as showing recommendations similar to Google results. Those early and promising results pave the way for a more diverse and complex vocabulary that exploits the full wealth of information that graphical elements can contain. Such a search feature has wide ranging applicability, from making visualizations more accessible for less technical users, to using other input modalities such as voice.

In genomic sequence analysis, approximate searching is necessary to tolerate errors involved from genetic or mechanic reasons. Short-read mapping method is one of the methods that can search genomic objects allowing an acceptable level of error rates. In this model, a large set of fragments is given, and it is asked to find the similar regions on a long sequence, where the fragments are matched as many as possible. In this paper, we exploit this model to search textual objects treating them in a genomic view. And we also propose a new method for fragment ordering that considers a time limit in the search process. The proposed method tends to select fragments having less frequency, which allows the maximum number of distinct fragments to be processed within a short time period. Empirical tests showed that our method is much faster than the previous method.

With the explosive growth and widespread accessibility of image content in social media, many users are eagerly searching for most recent and relevant images on topics of their interests. However, most current microblog platforms merely make use of textual information, specifically, keywords, for image search which cannot achieve satisfactory results since in most cases image content is inconsistent with textual content. In this paper we tackle this problem under the application of searching for celebrity image. The proposed method is based on the idea of refining the initial text-based search results by utilizing multimedia plus social information. Given a text search query, we first obtain an initial text-based result. Next, we extract a seed tweet set whose images contain faces recognized as celebrities and texts contain the expanded keywords. Third, we extend the seed set based on visual and user information. Lastly, we employ a multi-modal graph based learning method to properly rank the obtained tweets by integrating social and visual information. Extensive experiments on data collected from Tencent Weibo demonstrate that our proposed method could approximately achieve 3-fold improvement in results as compared to the text baseline, typically used in microblog search service.

With the development of academic research, the number of scientific papers has risen sharply, there is an urgent need to assist researchers in locating the candidate cited papers they are looking for. Classical relation-based and text-based approaches ignore the chronological nature of the citation recommendation task. In this study, we propose an innovative dynamic topic/citation influence model (DTCIM), which assumes user initial information need could shift while they are looking for the papers in different time slices. Specifically, we integrate text search and citation link navigation in a chronological dynamic topic model environment. Unlike previous studies, the new model characterizes user topical information need shifting and intrinsic citation time-decay. We apply this model for chronological citation recommendation, which can recommend time-series ranking lists based on users' initial textual information needs. Our experiment on the ACM sequential corpus shows that DTCIM is an effective model for chronological citation recommendation, comparing with classic models and algorithms.

Web page segmentation is a crucial step for many applications like information retrieval, text classification, noise removal, full text search and automatic page adaptation can benefit from this structure. In literature, many methods have been proposed on HTML Web page segmentation because of its more availability, whereas XML and Flash Web pages are omitted because of their less availability. But, in recent days we can see many XML/Flash Web pages on Web. In this article, we propose a segmentation method based on fusion and block merging concepts to improve the existing (Connected Component) Flash Web page segmentation method [10]. Finally, we perform experimental analysis based on ground truth analysis (actual blocks in Web page as per human perception) and obtained the better accuracy level when compared with the existing systems.

In the current paper, we present the web-based finder front-end WiTTFind, together with the Wittgenstein Advanced Search Tools (WAST), which offer new possibilities for exploring and researching Ludwig Wittgenstein's philosophy and work. Unlike the search capabilities of Google books and the Open Library project, our tools are rule-based together with local grammar search technology and TEI-P5 XML version of Wittgenstein's Nachlass, in combination with an electronic lexicon and various computational tools to enable lemmatized, semantic and syntactic queries to the texts. The query results are displayed in a web browser as HTML transformations of the transcribed texts, together with a facsimile of the matching segment in the original.

In this paper we present limitations of conventional faceted search in the way data, facets, and queries are modelled. We discuss how these limitations can be addressed with Semantic Web technologies such as RDF, OWL 2, and SPARQL 1.1. We also present a system, SemFacet, that is a proof-of-concept prototype of our approach implemented on top of Yago knowledge base, powered by the OWL 2 RL triple store RDFox, and the full text search engine Lucene.

Currently, popular search engines retrieve documents on the basis of text information. However, integrating the visual information with the text-based search for video and image retrieval is still a hot research topic. In this paper, we propose and evaluate a video search framework based on using visual information to enrich the classic text-based search for video retrieval. The framework extends conventional text-based search by fusing together text and visual scores, obtained from video subtitles (or automatic speech recognition) and visual concept detectors respectively. We attempt to overcome the so called problem of semantic gap by automatically mapping query text to semantic concepts. With the proposed framework, we endeavor to show experimentally, on a set of real world scenarios, that visual cues can effectively contribute to the quality improvement of video retrieval. Experimental results show that mapping text-based queries to visual concepts improves the performance of the search system. Moreover, when appropriately selecting the relevant visual concepts for a query, a very significant improvement of the system's performance is achieved.

The organizers of this workshop have issued a call for help, stating that one challenge in using knowledge and semantic annotations in search is that "standard text search excels at shallow information needs expressed by short keyword queries, and so semantic annotation contributes very little". I suggest that the answer is right there in the question, meaning that those who work with deeper knowledge representations should also try to aid more complex information needs. I will talk about search user interfaces, current and future, that have and hopefully will successfully improve user experience by adding in more understanding of content.

Text search engines are a fundamental tool nowadays. Their efficiency relies on a popular and simple data structure: the inverted indexes. Currently, inverted indexes can be represented very efficiently using index compression schemes. Recent investigations also study how an optimized document ordering can be used to assign document identifiers (docIDs) to the document database. This yields important improvements in index compression and query processing time. In this paper we follow this line of research, yet from a different perspective. We propose a docID reassignment method that allows one to focus on a given subset of inverted lists to improve their performance. We then use run-length encoding to compress these lists (as many consecutive 1s are generated). We show that by using this approach, not only the performance of the particular subset of inverted lists is improved, but also that of the whole inverted index. Our experimental results indicate a reduction of about 10% in the space usage of the whole index docID reassignment was focused. Also, decompression speed is up to 1.22 times faster if the runs must be explicitly decompressed and up to 4.58 times faster if implicit decompression of runs is allowed. Finally, we also improve the Document-at-a-Time query processing time of AND queries (by up to 12%), WAND queries (by up to 23%) and full (non-ranked) OR queries (by up to 86%).

With the increasing popularity of cloud computing, huge amount of documents are outsourced to the cloud for reduced management cost and ease of access. Although encryption helps protecting user data confidentiality, it leaves the well-functioning yet practically-efficient secure search functions over encrypted data a challenging problem. In this paper, we present a privacy-preserving multi-keyword text search (MTS) scheme with similarity-based ranking to address this problem. To support multi-keyword search and search result ranking, we propose to build the search index based on term frequency and the vector space model with cosine similarity measure to achieve higher search result accuracy. To improve the search efficiency, we propose a tree-based index structure and various adaption methods for multi-dimensional (MD) algorithm so that the practical search efficiency is much better than that of linear search. To further enhance the search privacy, we propose two secure index schemes to meet the stringent privacy requirements under strong threat models, i.e., known ciphertext model and known background model. Finally, we demonstrate the effectiveness and efficiency of the proposed schemes through extensive experimental evaluation.

There is an increasing amount of structure on the Web as a result of modern Web languages, user tagging and annotation, emerging robust NLP tools, and an ever growing volume of linked data. These meaningful, semantic, annotations hold the promise to significantly enhance information access, by enhancing the depth of analysis of today's systems. Currently, we have only started exploring the possibilities and only begin to understand how these valuable semantic cues can be put to fruitful use. To complicate matters, standard text search excels at shallow information needs expressed by short keyword queries, and here semantic annotation contributes very little, if anything. The main questions for the workshop are how to leverage the rich context currently available, especially in a mobile search scenario, giving powerful new handles to exploit semantic annotations. And how can we fruitfully combine information retrieval and semantic web approaches, and for the first time work actively toward a unified view on exploiting semantic annotations.

There has been widespread interest in both academia and industry around techniques to help in fault localization. Much of this work leverages static or dynamic code analysis and hence is constrained by the programming language used or presence of test cases. In order to provide more generically applicable techniques, recent work has focused on devising text search based approaches that recommend source files which a developer can modify to fix a bug. Text search may be used for fault localization in either of the following ways. We can search a repository of past bugs with the bug description to find similar bugs and recommend the source files that were modified to fix those bugs. Alternately, we can directly search the code repository to find source files that share words with the bug report text. Few interesting questions come to mind when we consider applying these text-based search techniques in real projects. For example, would searching on past fixed bugs yield better results than searching on code? What is the accuracy one can expect? Would giving preference to code words in the bug report better the search results? In this paper, we apply variants of text-search on four open source projects and compare the impact of different design considerations on search efficacy.

The Burrows Wheeler transform (BWT) has become popular in text compression, full-text search, XML representation, and DNA sequence matching. It is very efficient to perform a full-text search on BWT encoded text using backward search. This paper aims to study different approaches for applying BWT on multi-byte encoded (e.g. UTF-16) text documents. While previous work has studied BWT on word-based models, and BWT can be applied directly on multi-byte encodings (by treating the document as single-byte coded), there has been no extensive study on how to utilize BWT on multi-byte encoded documents for efficient full-text search. Therefore, in this paper, we propose several ways to efficiently backward search multi-byte text documents. We demonstrate our findings using Chinese text documents. Our experiment results show that our extensions to the standard BWT method offer faster search performance and use less runtime memory.

People browsing the web or reading a document may see text passages that describe a topic of interest, and want to know more about it by searching. Manually formulating a query from that text can be difficult, however, and an effective search is not guaranteed. In this paper, to address this scenario, we propose a learning-based approach which generates effective queries from the content of an arbitrary user-selected text passage. Specifically, the approach extracts and selects representative chunks (noun phrases or named entities) from the content (a text passage) using a rich set of features. We carry out experiments showing that the selected chunks can be effectively used to generate queries both in a TREC environment, where weights and query structure can be directly incorporated, and with a "black-box" web search engine, where query structure is more limited.

We discuss the advantages and shortcomings of full-text search on the one hand and search in ontologies/triple stores on the other hand. We argue that both techniques have an important quality missing from the other. We advocate a deep integration of the two, and describe the associated requirements and challenges.

Time-travel text search enriches standard text search by temporal predicates, so that users of web archives can easily retrieve document versions that are considered relevant to a given keyword query and existed during a given time interval. Different index structures have been proposed to efficiently support time-travel text search. None of them, however, can easily be updated as the Web evolves and new document versions are added to the web archive.In this work, we describe a novel index structure that efficiently supports time-travel text search and can be maintained incrementally as new document versions are added to the web archive. Our solution uses a sharded index organization, bounds the number of spuriously read index entries per shard, and can be maintained using small in-memory buffers and append-only operations. We present experiments on two large-scale real-world datasets demonstrating that maintaining our novel index structure is an order of magnitude more efficient than periodically rebuilding one of the existing index structures, while query-processing performance is not adversely affected.

With rapid intensification of existing multimedia documents and mounting demand for information indexing and retrieval, much endeavor has been done on extracting the text from images and videos. Text extraction in video documents, as a momentous research division of content-based information retrieval and indexing, continues to be a topic of much interest to researchers. Text extracting is demanding owing to a range of setbacks like complex background, varying font size, different style, lower resolution and blurring, position, viewing angle and so on. In this paper we propose a hybrid method where the two most well-liked text extraction methods explicitly region based method and connected component (CC) based method comes together. The former method is used to obtain the text prevailing confidence where as the latter is used for text extraction and grouping. The video splitting and key frame detection is followed by the preprocessing to designate the text region indicator. The extracted features are scrutinized using artificial neural network as the classifier and lastly grouped into words/lines based on the bounding box distance. We evaluated the performance of the proposed approach on various videos and obtained considerable results when weighed against the existing methods.

Attaching a large external display can help a mobile device user view more content at once. This paper reports on a study investigating how different configurations of input and output across displays affect performance, subjective workload and preferences in map, text and photo search tasks. Experimental results show that a hybrid configuration where visual output is distributed across displays is worst or equivalent to worst in all tasks. A mobile device-controlled large display configuration performs best in the map search task and equal to best in text and photo search tasks (tied with a mobile-only configuration). After conducting a detailed analysis of the performance differences across different UI configurations, we give recommendations for the design of distributed user interfaces. Postprint

Emails are important tools for communication and cooperation, they contain large amount of information and connections to knowledge and data sources. Because of this, it is very important to improve the efficiency of their processing. This paper describes an email search system which integrates full-text search with social search while processing also the attached and linked resources. The project described in this paper is still in progress. Due to this fact, some proposed parts of the system are not implemented and also not proven yet. The proposed equation for determining the social importance of an email has also to be tuned during the last phases of the development and the evaluation phase. The already implemented part of the system includes content extraction from the email messages, attached and linked resources and also the textual search and social relation extraction is implemented. The next phase of the development includes tuning of the social evaluation and it's integration with textual search.

The growing sizes of text repositories on the world wide web has created a need for efficient indexing and retrieval methods for text collections. Almost all of the text retrieval and indexing methods have been designed for the case of simple keyword search, in which a few keywords are specified, and the text is retrieved on the basis of matches to these keywords. However, in many applications there is a need for a greater specificity during the search, such as the use of phrases, sentences, text fragments, or even documents for the retrieval process. An even more general case is one in which a collection of documents is available as a query to the search process. In such cases, it is desirable to return sets of all pairwise similar documents. Such queries are referred to as corpus to corpus queries, and are computationally intensive because of the very large number of document pairs which need to be compared. Such cases cannot be efficiently processed by the available indexing and searching methods. Most of the currently available techniques can index the text based on only a small number of keywords or representative phrases. In this paper, we design a compressed finger print index which can support the following more general queries: (a) The method can process very efficient document-to-corpus search because of their efficient bit-wise operations for the search process. (b) We further extend the method to work for corpus-to-corpus queries, in which it is desirable to determine the most similar pairs of documents in two collections. We design an efficient search technique which is able to reduce the search time for large collections. The key technique used to enable this is an efficient fingerprint representation, which can be used effectively for the search process. To the best of our knowledge, this is the first work on corpus-based search in massive document collections.

Text-based search and retrieval is used by developers in the context of many SE tasks, such as, concept location, traceability link retrieval, reuse, impact analysis, etc. Solutions for software text search range from regular expression matching to complex techniques using text retrieval. In all cases, the results of a search depend on the query formulated by the developer. A developer needs to run a query and look at the results before realizing that it needs reformulating. Our aim is to automatically assess the performance of a query before it is executed. We introduce an automatic query performance assessment approach for software artifact retrieval, which uses 21 measures from the field of text retrieval. We evaluate the approach in the context of concept location in source code. The evaluation shows that our approach is able to predict the performance of queries with 79% accuracy, using very little training data.

This paper reports a system developed for video browsing based on multimodal analysis. Our multimodal approach performs audio transcription for shot categorization (sports, weather, politics and economy) combining audio and visual information for theme categorization. Its main features include static and dynamic summaries, segmentation using face detection, classification into Indoor/Outdoor scenes based on Support Vector Machine (SVM) and audio transcription for theme keyword search. Keywords are selected to represent the subjects, followed by a simple text search. We conduct a set of experiments for evaluating the effectiveness of the shot subject categorization using audio transcription information.

In this paper, we propose a novel image search scheme, contextual image search. Different from conventional image search schemes that present a separate interface (e.g., text input box) to allow users to submit a query, the new search scheme enables users to search images by only masking a few words when they are reading through Web pages or other documents. Rather than merely making use of the explicit query input that is often not sufficient to express user's search intent, our approach explores the context information to better understand the search intent with two key steps: query augmenting and search results reranking using context, and expects to obtain better search results. Beyond contextual Web search, the context in our case is much richer and includes images besides texts. In addition to this type of search scheme, called contextual image search with text input, we also present another type of scheme, called contextual image search with image input, to allow users to select an image as the search query from Web pages or other documents they are reading. The key idea is to use the search-to-annotation technique and the contextual textual query mining scheme to determine the corresponding textual query, to finally get semantically similar search results. Experiments show that the proposed schemes make image search more convenient and the search results are more relevant to user intention.