Search

With the rapid growth of multimedia data (e.g., text, image, video, audio and 3D model, etc) in the web, there are a large number of media objects with different modalities in the multimedia documents such as webpages, which exhibit latent semantic correlation. As a new type of multimedia retrieval method, cross-media retrieval is becoming increasingly attractive, through which users can get the results with various media types with the same semantic information by submitting a retrieval of any media type. The explosive increasing of the number of media objects, however, makes it difficult for the traditional local standalone mode to process efficiently. So the powerful parallel processing capability of cloud computing is accommodated to facilitate the efficient large-scale cross-media retrieval. In this paper, based on a Multi-Layer-Cross-Reference-Graph(MLCRG) model, we propose an efficient parallel cross-media retrieval (PCMR) method in which two enabling techniques (i.e., 1) the adaptive cross-media data allocation algorithm and 2) the PCIndex scheme) are accommodated to effectively speedup the retrieval performance. To the best of our knowledge, there is little research on the parallel retrieval processing of the large-scale cross-media databases in the mobile cloud network. Extensive experiments are conducted to testify that our proposed PCIndex method outperform the three competitors (e.g., the PFAR (Mao et al, 22), the MBSR (Retrieval 4(2):153-164, 42) and the SPECH (Knowl Based Syst 251(5):1-13, 40)) in terms of the effectiveness and efficiency, respectively. [ABSTRACT FROM AUTHOR]

Web video event mining based on cross-media fusion has become a research hotspot. However, each video is only described by a dozen noisy words, resulting in extremely unstable textual features. Moreover, different people might describe the same video with completely different words. Thus, the semantic association between textual and visual information would be much sparse, which brings great challenges to web video event mining based on cross-media associations. To address this issue, this paper proposes a novel framework to enrich the associations between near duplicate keyframes (NDK) and terms based on multiple semantic-paths embedding. After data preprocessing, we build a heterogeneous information network to establish associations among NDKs, terms and videos. Then, semantic-path walk strategy is designed to generate meaningful semantic-node sequences for embedding. Next, an embedding fusion method is proposed to predict the distribution characteristics of each term in NDKs. Finally, multiple correspondence analysis is used to mine web video events. Experiments on web videos from YouTube show that our proposed method performs better than several state-of-the-art baseline models, with an average F1 score improvement of 19–50%. [ABSTRACT FROM AUTHOR]

This paper addresses mining and extraction in the Arctic with examples fetched from the Identified Geographic Areas in the Euro-Asian Arctic region. The overall Arctic region is undergoing significant changes in land and sea use, fishing, forestry, transportation, freshwater diversion, urbanisation, and more. The Arctic, covering 6 percent of the earth’s surface, is a nursery of the planet food chains and migratory wildlife important to Earth and ecosystem survival. It houses 25 percent of the world’s Large Marine Ecosystems, substantial amounts of energy resources including oil, gas, and critically required materials. The region, with a mere 0.1 percent of the global human population, has generated considerable material wealth for the eight Arctic states surrounding it and the global north. The region is facing unprecedented changes and opportunities. The Arctic has warmed up four times faster than the rest of the planet and an intensive exploitation of its mineral wealth, fish stocks and strategic military location has resulted in 3,000 hazardous hotspots areas to remedy. Key challenges and opportunities for sustainability of the Arctic resources relate to the mining and extractive sector and associated infrastructure. The issues include on-shore and off-shore exploitation such as intentions to mine sea- and riverbeds in sensitive eco-systems. Action-wise, in the regulatory and incentives spheres, the European Union and the Nordics countries are moving forward to embrace the world leading initiative on a Green Deal. Similarly, Canada, Russia and the US have amended or are adjusting their regulatory framework to tackle the rapidly changing arena. For the wellbeing of Arctic, organisations such as the Arctic Council, its permanent participants of Indigenous peoples, and observers, including countries like China, Japan, India, Germany, the Netherlands, have engaged constructively and aspiring to continue doing so to access a fair share of the Arctic Wealth. [ABSTRACT FROM AUTHOR]

We empirically examine the cross-media effects of personalized and mass media on consumers' purchase incidence in a multichannel shopping environment. We capture the cross-media effect as the combined impact of two distinct marketing communications on consumers' purchase behavior. Our data consists of individual-level transaction data and information on consumers' exposure to multiple marketing media consisting of personalized (catalog and email) and mass (television and radio) media. We find that personalized (mass) media are more influential in driving consumers' online (offline) purchases in a multichannel shopping environment. Our analysis of cross-media effects reveals synergistic (attenuating) effects between media components across (within) personalized and mass media. Furthermore, our examination of media elasticities demonstrates that discounting such cross-media effects between personalized and mass media components can bias a firm's assessment of the effectiveness of media components in a multichannel-multimedia marketing environment. Results from our model can help marketing managers in the optimal planning of integrated marketing communication in a multichannel-multimedia shopping environment. [ABSTRACT FROM AUTHOR]

In this work, we introduce Video Question Answering in the temporal domain to infer the past, describe the present and predict the future. We present an encoder-decoder approach using Recurrent Neural Networks to learn the temporal structures of videos and introduce a dual-channel ranking loss to answer multiple-choice questions. We explore approaches for finer understanding of video content using the question form of 'fill-in-the-blank', and collect our Video Context QA dataset consisting of 109,895 video clips with a total duration of more than 1000 h from existing TACoS, MPII-MD and MEDTest 14 datasets. In addition, 390,744 corresponding questions are generated from annotations. Extensive experiments demonstrate that our approach significantly outperforms the compared baselines. [ABSTRACT FROM AUTHOR]

The growing popularity of social media provides a huge volume of social data including Tweets. These collections of social data can be potentially useful, but the extent of meaningful data in these collections has not been sufficiently researched, especially in South Korea Twitter data. In general, the South Korea Twitter data has been researched as a source of political media. Nonetheless, previous research on South Korea Twitter data has not adequately covered what kind of trend Twitter represents in terms of major topic categories such as politics, economics, or sports. In this paper, we present a cross-media approach to define the nature of South Korea Tweets by inferring the topic category distribution through short-text categorization. We select newspapers as cross-media, examine the categorization of news articles from major newspapers, and then train our classifier based on the features from each topic category. In addition, for grafting news topics onto South Korea Tweets, we propose a word clustering and filtering approach to exclude those words that do not provide semantic content for the topic categories. Based on the proposed procedures, we analyze the South Korea Tweets to determine the primary topic category focus of Twitter users. We observe the special behaviors of the South Korea Twitter users based on various parameters such as date, time slot, and day of the week. Because our research includes a macroscopic analysis of Twitter data using a cross-media strategy, our research can provide useful resources for other social media analysis as well. [ABSTRACT FROM AUTHOR]

With the explosion of multimedia data, it is usual that different multimedia data often coexist in web repositories. Accordingly, it is more and more important to explore underlying intricate cross-media correlation instead of single-modality distance measure so as to improve multimedia semantics understanding. Cross-media distance metric learning focuses on correlation measure between multimedia data of different modalities. However, the existence of content heterogeneity and semantic gap makes it very challenging to measure cross-media distance. In this paper, we propose a novel cross-media distance metric learning framework based on sparse feature selection and multi-view matching. First, we employ sparse feature selection to select a subset of relevant features and remove redundant features for high-dimensional image features and audio features. Secondly, we maximize the canonical coefficient during image-audio feature dimension reduction for cross-media correlation mining. Thirdly, we further construct a Multi-modal Semantic Graph to find embedded manifold cross-media correlation. Moreover, we fuse the canonical correlation and the manifold information into multi-view matching which harmonizes different correlations with an iteration process and build Cross-media Semantic Space for cross-media distance measure. The experiments are conducted on image-audio dataset for cross-media retrieval. Experiment results are encouraging and show that the performance of our approach is effective. [ABSTRACT FROM AUTHOR]

Cross-media semantics understanding, which focuses on multimedia data of different modalities, is a rising hot topic in social media analysis. One of the most challenging issues for cross-media semantics understanding is how to represent multimedia data of different modalities. Most traditional multimedia semantics analysis works are based on single modality data sources, such as Flickr images or YouTube videos, leaving efficient cross-media data representation wide open. In this paper, we propose a novel nonnegative cross-media recoding approach, which learns co-occurrences of cross-media feature spaces by explicitly learning a common subset of basis vectors. Moreover, we impose the nonnegativity constraint on the decomposed matrices so that the basis vectors represent important and locally meaningful features of the cross-media data. We take two kinds of typical multimedia data, that is, image and audio, as experimental data. Our approach can be applied to a wide range of multimedia applications. The experiments are conducted on image-audio dataset for applications of cross-media retrieval and data clustering. Experiment results are encouraging and show that the performance of our approach is effective. [ABSTRACT FROM AUTHOR]

Cross-media retrieval aims to automatically perform the content-based search procedure among various media types (e.g., image, video and text), in which media representation plays an important role for providing the heterogeneous similarity measure. In this work, a novel semantic representation of cross-media, called accumulated reconstruction error vector (AREV), is proposed, which includes category-specific dictionary learning, media sample reconstruction, and accumulative reconstruction error concatenation. Instead of directly learning the correlation relationship among heterogeneous items in the same semantic groups, the AREV projects individually their original feature descriptions into a shared semantic space, in which each component is semantic consistent for various media types due to the consistency in category information. Experiments on the commonly used datasets, i.e. Wikipedia dataset and NUS-Wide dataset, show the good performance in terms of effectiveness and efficiency. [ABSTRACT FROM AUTHOR]

Recently, we designed a new experimental system MSearch, which is a cross-media meta-search system built on the database of the WikipediaMM task of ImageCLEF 2008. For a meta-search engine, the kernel problem is how to merge the results from multiple member search engines and provide a more effective rank list. This paper deals with a novel fusion model employing supervised learning. Our fusion model employs ranking SVM in training the fusion weight for each member search engine. We assume the fusion weight of each member search engine as a feature of a result document returned by the meta-search engine. For a returned result document, we first build a feature vector to represent the document, and set the value of each feature as the document's score returned by the corresponding member search engine. Then we construct a training set from the documents returned from the meta-search engine to learn the fusion parameter. Finally, we use the linear fusion model based on the overlap set to merge the results set. Experimental results show that our approach significantly improves the performance of the cross-media meta-search (MSearch) and outperforms many of the existing fusion methods. [ABSTRACT FROM AUTHOR]

A cross-media platform was designed for a community of young teenagers oriented towards societal change. The platform involved an interactive web-forum featuring creative and communicative collaborative tools in a 3D avatar environment, and a weekly show in national public-service television. Informal assessment of the work indicated that (1) an integrated spiral of production and consumption across the two media channels involved is a viable design concept to support community building, (2) off-the-shelf avatar technology and consumer-grade Internet connectivity can form a feasible infrastructure for collaborative storytelling tools, and (3) a participatory design process wherein participants transition into the role of mentors and norm carriers upon deployment can be a feasible way to support subcultural community building towards “difficult” topics, even though it entails considerable resource demands. All of these findings are potentially transferable to other design domains and audiences. [ABSTRACT FROM AUTHOR]