Search

• Featured graph models inter-word semantics of texts. • Stacked co-attention learns fine-grained correlations. • 19% increase on MAP. Cross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality. The main challenge is to learn the semantic correlations between different modalities and measure the distance across modalities. For text-image retrieval, existing work mostly uses off-the-shelf Convolutional Neural Network (CNN) for image feature extraction. For texts, word-level features such as bag-of-words or word2vec are employed to build deep learning models to represent texts. Besides word-level semantics, the semantic relations between words are also informative but less explored. In this paper, we explore the inter-word semantics by modelling texts by graphs using similarity measure based on word2vec. Besides feature presentations, we further study the problem of information imbalance between different modalities when describing the same semantics. For example textual descriptions often contain more background information that cannot be conveyed by images and vice versa. We propose a stacked co-attention network to progressively learn the mutually attended features of different modalities and enhance their fine-grained correlations. A dual-path neural network is proposed for cross-modal information retrieval. The model is trained by a pairwise similarity loss function to maximize the similarity of relevant text-image pairs and minimize the similarity of irrelevant pairs. Experimental results show that the proposed model outperforms the state-of-the-art methods significantly, with 19% improvement on accuracy for the best case. [ABSTRACT FROM AUTHOR]

Abstract This paper presents a method for automatic constructing a conceptual semantic network (Task ontology) based on texts of documents. Ontology is considered as metagraph that is reflecting immanent and situational relations. Verbose constructs, which corresponds to concepts and relationships, are extracted from texts by analyzing the linguistic characteristics of words and the templates of typical constructions. Linguistic constructions of situational links are identified by relations taxonomy. [ABSTRACT FROM AUTHOR]

Up to this point, research on written scholarly communication has focused primarily on syntactic, rather than semantic, analyses. Consequently, we have yet to understand semantic change as it applies to disciplinary discourse. The objective of this study is to illustrate word semantic change in biomedical literature. To that end, we identify a set of representative words in biomedical literature based on word frequency and word-topic probability distributions. A word2vec language model is then applied to the identified words in order to measure word- and topic-level semantic changes. We find that for the selected words in PubMed, overall, meanings are becoming more stable in the 2000s than they were in the 1980s and 1990s. At the topic level, the global distance of most topics (19 out of 20 tested) is declining, suggesting that the words used to discuss these topics are stabilizing semantically. Similarly, the local distance of most topics (19 out of 20) is also declining, showing that the meanings of words from these topics are becoming more consistent with those of their semantic neighbors. At the word level, this paper identifies two different trends in word semantics, as measured by the aforementioned distance metrics: on the one hand, words can form clusters with their semantic neighbors, and these words, as a cluster, coevolve semantically; on the other hand, words can drift apart from their semantic neighbors while nonetheless stabilizing in the global context. In relating our work to language laws on semantic change, we find no overwhelming evidence to support either the law of parallel change or the law of conformity. [ABSTRACT FROM AUTHOR]

Understanding topic evolution in a scientific domain is essential for capturing key domain developments and facilitating knowledge transfer within and across domains. Using a data set on information retrieval (IR) publications, this paper examines how research topics evolve by analyzing the topic trends, evolving dynamics, and semantic word shifts in the IR domain. Knowledge transfer between topics and the developing status of the major topics have been recognized, which are represented by the merging and splitting of local topics in different time periods. Results show that the evolution of a major topic usually follows a pattern from adjusting status to mature status, and sometimes with re-adjusting status in between the evolving process. Knowledge transfer happens both within a topic and among topics. Word migration via topic channels has been defined, and three migration types (non-migration, dual-migration, and multi-migration) are distinguished to facilitate better understanding of the topic evolution. [ABSTRACT FROM AUTHOR]

• Existing Visual Semantic Embedding (VSE) networks are trained by a hard negatives loss function that learns an objective margin between the similarity of relevant and irrelevant image–description embedding pairs and ignores the semantic differences between the irrelevant pairs. • We propose a novel Semantically Enhanced Hard negatives Loss function (LSEH) for Cross-modal Information Retrieval that considers the semantic differences between irrelevant training pairs. • The proposed LSEH function dynamically adjusts the learning objectives of VSE networks to make their learning flexible and efficient. • Experiments with various benchmark datasets and VSE networks revealed that the proposed LSEH function reduces their training epochs by approximately 50% and also improves their retrieval performance. Visual Semantic Embedding (VSE) networks aim to extract the semantics of images and their descriptions and embed them into the same latent space for cross-modal information retrieval. Most existing VSE networks are trained by adopting a hard negatives loss function which learns an objective margin between the similarity of relevant and irrelevant image–description embedding pairs. However, the objective margin in the hard negatives loss function is set as a fixed hyperparameter that ignores the semantic differences of the irrelevant image–description pairs. To address the challenge of measuring the optimal similarities between image–description pairs before obtaining the trained VSE networks, this paper presents a novel approach that comprises two main parts: (1) finds the underlying semantics of image descriptions; and (2) proposes a novel semantically-enhanced hard negatives loss function, where the learning objective is dynamically determined based on the optimal similarity scores between irrelevant image–description pairs. Extensive experiments were carried out by integrating the proposed methods into five state-of-the-art VSE networks that were applied to three benchmark datasets for cross-modal information retrieval tasks. The results revealed that the proposed methods achieved the best performance and can also be adopted by existing and future VSE networks. [ABSTRACT FROM AUTHOR]

Semantic relatedness assessment between concepts is a critical issue in many domains such as artificial intelligence, information retrieval, psychology, biology, linguistics and cognitive science. Therefore, several methods assess relatedness by exploiting knowledge bases to express the semantics of concepts. However, there are some limitations such as high-dimensional space, high-computational complexity, fitting non-dynamic domains. Considering that Wikipedia, a domain-independent encyclopedic repository, which provides very large coverage, has been exploited by many methods as a huge semantic resource. In this paper, we propose a novel graph-based relatedness assessment method using Wikipedia features to avoid some of the limitations and drawbacks mentioned above. Firstly, for each term in a word pair, the top k most relevant Wikipedia concepts are returned by the Naive-ESA algorithm to reduce the dimensional space of Explicit Semantic Analysis (ESA) method. Secondly, for each different candidate concept in two relevant concept sets, we collect its categories set from the Wikipedia Category Graph (WCG). Based on the categories in WCG network, the relatedness between concepts at the correspondence position of the two sorted concept sets is computed as the association coefficient. Thirdly, based on this parameter, a novel relatedness assessment metric is presented. The evaluation is performed on some datasets well-recognized as benchmarks, using several widely used metrics and a new metric designed by ourselves. The result demonstrates that our method has a better correlation with the intuitions of human judgments than other related works. [ABSTRACT FROM AUTHOR]

Text document classification is a well known theme in the field of the information retrieval and text mining. Selection of most desired features in the text document plays a vital role in classification problem. This research article addresses the problem of text classification by considering Sentence–Vector Space Model (S-VSM) and Unigram representation models for the text document. An enhanced S-VSM model will be considered for the constructive representation of text documents. A neural network based representation for text documents is proposed for effective capturing of semantic information of the text data. Two different classifiers are designed based on the two different representation models of the text documents. Score level fusion is applied on two proposed models to find out the overall accuracy of the proposed model. Key contributions of the paper are an enhanced S-VSM model, an interval valued representation model for the proposed S-VSM approach. A word level representation model for semantic information preserving of the text document and score level fusion approach. [ABSTRACT FROM AUTHOR]

In the era of Web 2.0, the knowledge is the de-facto social currency in the global network environment. Knowledge is not an accumulation of data, but a relation-based representation of the information content, which needs to be distilled and arranged in a semantic infrastructure to guarantee interoperability and sharable understanding. In the light of this scenario, the paper introduces a semantically enhanced document retrieval system that describes each retrieved document with an ontological multi-grained network of the extracted conceptualization. The system is based on two well-known latent models: Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA): LSA provides a spatial distribution of the input documents, facilitating their retrieval, thanks to an ontological representation of their relationship network. LDA works instead at deeper level: it drives the ontological structuring of the knowledge inside the individual retrieved documents in terms of words, concepts and topics. The novelty of this approach is a multi-level granulation of the knowledge: from a document matching the query (coarse granularity), to the topics that join documents, until to the words describing a concept into a topic (fine granularity). The final result is a SKOS-based ontology, ad-hoc created for a document corpus; graphically supported for the navigation, it enables the exploration of the concepts at different granularity levels. [ABSTRACT FROM AUTHOR]

Word Sense Disambiguation represents a crucial task in many natural language processing applications such as information extraction, information retrieval and text summarization. It intends to identify the correct sense of an ambiguous term (target word) according to its context. In this paper, we present a biomedical knowledge-based disambiguation method for determining the adequate domain or sub-domain (sense) of an ambiguous biomedical term. This method uses the MeSH thesaurus as a knowledge resource and an intrinsic information content-based semantic similarity to measure the likeness between their different senses of the ambiguous term MeSH and its context. We define this latter as the set of unambiguous MeSH terms characterizing a document. We aim on one hand to minimize the time and computational complexity and on the other hand to increase the accuracy of the disambiguation by using such context. To evaluate this method, we involved it into a document indexing and retrieval system. The results of experiments, carried out on a sub-set of the OHSUMED collection, in terms of Mean Average Precision show that our method performs well. [ABSTRACT FROM AUTHOR]

Hashing based approximate nearest neighbors (ANN) search has drawn considerable attraction owing to its low-memory storage and hardware-level logical computing which is doomed to be greatly applicable to quantities of large-scale and practical scenarios, such as information retrieval, computer vision and natural language processing. However, most existing hashing methods concentrate either on images only or on pairwise image-texts (labels, short documents) and rarely utilize more common sentences. In this paper, we propose D iversity R egularized L atent S emantic M atch for H ashing (DRLSMH), a new multimodal hashing method that projects images and sentences into a shared latent semantic space with label-supervised semantic constraints to proceed on multimodal retrieval. Notably, soft orthogonality is induced as a novel regularizer to preserve diverse hashing functions for compact and accurate representations; what's more, this kind of regularization also benefits the derivations of closed-form solutions with some proper relaxations under iterative optimization framework. Extensive experiments on two public datasets demonstrate the advantages of our method over some state-of-the-art baselines under cross-modal retrieval both on image-query-image, image-query-text and text-query-image tasks. [ABSTRACT FROM AUTHOR]

The ubiquitous presence of surveillance systems generates massive amounts of video data. Storage and analysis of this data in real-time is a substantial challenge. There is huge potential in representing data in machine-readable and machine-interpretable format due to the presence of hidden semantics in images and videos. However, such representation requires ontology, which calls for expert domain knowledge. In this paper, a novel NLP-guided approach to generate an ontology for multimedia representation and information retrieval is proposed. A semi-automatic NLP-guided framework, which extracts all possible relations among objects is presented. This framework leverages the textual data of the domain to generate possible descriptions and actions within the domain. Relations among objects get embedded as object properties, whereas the category of an object as a class. Features and attributes of objects encode the data properties of the ontology. The proposed ontology is compared with existing multimedia ontologies and evaluated with regard to its capability to represent relations occurring in benchmark datasets, demonstrating the completeness and thorough coverage of the domain concepts. Spatial reasoning rules are established using Semantic Web Rule Language (SWRL) rules, and information retrieval is demonstrated using Description Logic (DL) and SPARQL queries. The proposed NLP-guided ontology generation approach is general enough to help in the development of ontologies for other domains as well, by providing video and textual data of the domain of interest, with limited human involvement. • Object properties extraction with textbase and NLP-guided approach. • Frequency-based semantic refinement to enhance domain-specific relations. • A novel data-driven video surveillance ontology "VizITS" for the ITS domain. • Semantic information retrieval as events, metadata, and question-answering. [ABSTRACT FROM AUTHOR]

Minimal-interval semantics [8] associates with each query over a document a set of intervals, called witnesses , that are incomparable with respect to inclusion (i.e., they form an antichain): witnesses define the minimal regions of the document satisfying the query. Minimal-interval semantics makes it easy to define and compute several sophisticated proximity operators, provides snippets for user presentation, and can be used to rank documents. In this paper we provide algorithms for computing conjunction and disjunction that are linear in the number of intervals and logarithmic in the number of operands; for additional operators, such as ordered conjunction and Brouwerian difference, we provide linear algorithms. In all cases, space is linear in the number of operands. More importantly, we define a formal property of optimal laziness , and either prove it, or prove its impossibility, for each algorithm. We cast our results in a general framework of finite antichains of intervals on total orders, making our algorithms directly applicable to other domains. [ABSTRACT FROM AUTHOR]

Many types of heterogeneous CAD models created by previous product development activities are stored in enterprise databases. Searching reusable CAD models for engineer design requirements is still very difficult. In this paper, we propose an ontology-based semantic retrieval approach for heterogeneous 3D CAD models by using semantic web theory and information retrieval theory. By exploiting layered feature ontology and ontology mapping, a uniform description for heterogeneous CAD models is generated as semantic descriptor. Meanwhile, to improve the representative ability, several heuristic rules are constructed with the aid of SWRL language and ontology reasoning is acted on the semantic descriptors, which makes our retrieval system achieve better performance. Furthermore, multi-mode similarity measurement is carried out to meet engineers’ diversified requirements. We demonstrate the feasibility and effectiveness of our approach through experiments on heterogeneous 3D CAD model library. [ABSTRACT FROM AUTHOR]

A common approach to software architecture documentation in industry projects is the use of file-based documents. This approach offers a single-dimensional arrangement of the architectural knowledge. Knowledge retrieval from file-based architecture documentation is efficient if the organisation of knowledge supports the needs of the readers; otherwise it can be difficult. In this paper, we compare the organisation and retrieval of architectural knowledge in a file-based documentation approach and an ontology-based documentation approach. The ontology-based approach offers a multi-dimensional organisation of architectural knowledge by means of a software ontology and semantic wiki, whereas file-based documentation typically uses hierarchical organisation by directory structure and table of content. We conducted case studies in two companies to study the efficiency and effectiveness of retrieving architectural knowledge from the different organisations of knowledge. We found that the use of better knowledge organisation correlates with the efficiency and effectiveness of AK retrieval. Professionals who used the knowledge organisation found this beneficial. [ABSTRACT FROM AUTHOR]

Explicit Semantic Analysis (ESA) is a knowledge-based method which builds the semantic representation of the words depending on the textual description of the concepts in the certain knowledge source. Due to its simplicity and success, ESA has received wide attention from researchers in the computational linguistics and information retrieval. However, the representation vectors formed by ESA method are generally very excessive, high dimensional, and may contain many redundant concepts. In this paper, we introduce a reduced semantic representation method that constructs the semantic interpretation of the words as the vectors over the latent topics from the original ESA representation vectors. For modeling the latent topics, the Latent Dirichlet Allocation (LDA) is adapted to the ESA vectors for extracting the topics as the probability distributions over the concepts rather than the words in the traditional model. The proposed method is applied to the wide knowledge sources used in the computational semantic analysis: WordNet and Wikipedia. For evaluation, we use the proposed method in two natural language processing tasks: measuring the semantic relatedness between words/texts and text clustering. The experimental results indicate that the proposed method overcomes the limitations of the representation of the ESA method. [ABSTRACT FROM AUTHOR]

In media retrieval system for intelligent transportation, media data variety and heterogeneity have been one of the most critical features. Documents with different formats may express similar semantic information, thus, searching documents reflecting users׳ intention has been a crucial and important task. For solving this problem, this paper proposes a novel semantic-based heterogeneous transportation media retrieval (TMR) approach to improve the performance. TMR supports the function of retrieving various media types such as image, video, audio and text by using a single media type. Firstly, semantic fields are extracted from the user annotating and automatic learning to express the users׳ intention. Secondly, ontology is used to represent the semantic fields of a media, and the ontology represented semantic information is saved together with the media document data. Thirdly, the semantic field adjustment process is described. Finally, fuzzy matching is employed to measure the similarity between the users׳ intention and media documents. For the returned results, we carry out the performance evaluation models in comparison with the existing approaches. Experimental result indicates the superiority of TMR in term of precision rate, computing speed, storage cost and user experience. [ABSTRACT FROM AUTHOR]

This paper presents a novel Content-Based Video Retrieval approach in order to cope with the semantic gap challenge by means of latent topics. Firstly, a supervised topic model is proposed to transform the classical retrieval approach into a class discovery problem. Subsequently, a new probabilistic ranking function is deduced from that model to tackle the semantic gap between low-level features and high-level concepts. Finally, a short-term relevance feedback scheme is defined where queries can be initialised with samples from inside or outside the database. Several retrieval simulations have been carried out using three databases and seven different ranking functions to test the performance of the presented approach. Experiments revealed that the proposed ranking function is able to provide a competitive advantage within the content-based retrieval field. [ABSTRACT FROM AUTHOR]

In this paper, a novel method based on tagging technology is proposed for assessing learners which is named TagAssessment. In this method, in order to assess learners, the semantic relationship between educational contents and learner's tags is computed using WordNet ontology. An automatic keyword extraction component is also proposed to facilitate using this method for each educational content. To evaluate this method, an e-learning system is developed. The obtained results show that learner's assessment with the proposed method was too close to the multiple choice question method. Also, the results of the automatic keyword extraction component were very similar to standard tags which were gained by experts. [ABSTRACT FROM AUTHOR]

Objective: Case-based reasoning (CBR) is a problem-solving paradigm that uses past knowledge to interpret or solve new problems. It is suitable for experience-based and theory-less problems. Building a semantically intelligent CBR that mimic the expert thinking can solve many problems especially medical ones.Methods: Knowledge-intensive CBR using formal ontologies is an evolvement of this paradigm. Ontologies can be used for case representation and storage, and it can be used as a background knowledge. Using standard medical ontologies, such as SNOMED CT, enhances the interoperability and integration with the health care systems. Moreover, utilizing vague or imprecise knowledge further improves the CBR semantic effectiveness. This paper proposes a fuzzy ontology-based CBR framework. It proposes a fuzzy case-base OWL2 ontology, and a fuzzy semantic retrieval algorithm that handles many feature types.Material: This framework is implemented and tested on the diabetes diagnosis problem. The fuzzy ontology is populated with 60 real diabetic cases. The effectiveness of the proposed approach is illustrated with a set of experiments and case studies.Results: The resulting system can answer complex medical queries related to semantic understanding of medical concepts and handling of vague terms. The resulting fuzzy case-base ontology has 63 concepts, 54 (fuzzy) object properties, 138 (fuzzy) datatype properties, 105 fuzzy datatypes, and 2640 instances. The system achieves an accuracy of 97.67%. We compare our framework with existing CBR systems and a set of five machine-learning classifiers; our system outperforms all of these systems.Conclusion: Building an integrated CBR system can improve its performance. Representing CBR knowledge using the fuzzy ontology and building a case retrieval algorithm that treats different features differently improves the accuracy of the resulting systems. [ABSTRACT FROM AUTHOR]

Human-machine interaction mixed initiatives require a pragmatic coordination between different systems. Context understanding is established from the content, analysis, and guidance from query-based coordination between users and machines. Inspired by Level 5 Information Fusion ‘user refinement’, a live-video computing (LVC) structure is presented for user-based query access of a data-base management of information. Information access includes multimedia fusion of query-based text, images, and exploited tracks which can be utilized for context assessment, content-based information retrieval (CBIR), and situation awareness. In this paper, we explore new developments in dynamic data-driven application systems (DDDAS) of context analysis for user support. Using a common image processing data set, a system-level time savings is demonstrated using a query-based approach in a context, control, and semantic-aware information fusion design. [ABSTRACT FROM AUTHOR]

The Semantic Web has emerged as an extension of the current Web, in which Web content has well-defined meaning through the addition of logic-based metadata. However, current mechanisms for information retrieval from semantic knowledge bases restrict their use to only experienced users. To address this gap, the natural language processing (NLP) is deemed to be very intuitive from a use point of view, due to it hides the formality of a knowledge base as well as the executable query language. This paper presents a novel ontology-based information retrieval system for DBpedia called ONLI (Ontology-based Natural Language Interface). ONLI proposes the use of an ontology model in order to represent both the syntactic question’s structure and the question’s context. This model allows inferring the answer type expected by the user through an established question’s classification. These features allow reducing the search space thus increasing the probability of providing the correct answer. From this perspective, ONLI was evaluated in terms of their ability to find the correct answer into DBpedia’s content, achieving promising results and proving to be very useful to non-experienced users. [ABSTRACT FROM AUTHOR]