Search

Negative emotions may induce dangerous driving behaviors leading to extremely serious traffic accidents. Therefore, it is necessary to establish a system that can automatically recognize driver emotions so that some actions can be taken to avoid traffic accidents. Existing studies on driver emotion recognition have mainly used facial data and physiological data. However, there are fewer studies on multimodal data with contextual characteristics of driving. In addition, fully fusing multimodal data in the feature fusion layer to improve the performance of emotion recognition is still a challenge. To this end, we propose to recognize driver emotion using a novel multimodal fusion framework based on convolutional long-short term memory network (ConvLSTM), and hybrid attention mechanism to fuse non-invasive multimodal data of eye, vehicle, and environment. In order to verify the effectiveness of the proposed method, extensive experiments have been carried out on a dataset collected using an advanced driving simulator. The experimental results demonstrate the effectiveness of the proposed method. Finally, a preliminary exploration on the correlation between driver emotion and stress is performed. [ABSTRACT FROM AUTHOR]

A growing proportion of the global population is becoming overweight or obese, leading to various diseases (e.g., diabetes, ischemic heart disease and even cancer) due to unhealthy eating patterns, such as increased intake of food with high energy and high fat. Food recommendation is of paramount importance to alleviate this problem. Unfortunately, modern multimedia research has enhanced the performance and experience of multimedia recommendation in many fields such as movies and POI, yet largely lags in the food domain. This article proposes a unified framework for food recommendation, and identifies main issues affecting food recommendation including incorporating various context and domain knowledge, building the personal model, and analyzing unique food characteristics. We then review existing solutions for these issues, and finally elaborate research challenges and future directions in this field. To our knowledge, this is the first survey that targets the study of food recommendation in the multimedia field and offers a collection of research studies and technologies to benefit researchers in this field. [ABSTRACT FROM AUTHOR]

Health and well-being are shaped by how lifestyle and the environment interact with biological machines. A navigational paradigm can help users reach a specific health goal by using constantly captured measurements to estimate how their health is continuously changing and provide actionable guidance. [ABSTRACT FROM AUTHOR]

The goal of the MMHealth 2017 Workshop on Multimedia for Personal Health and Health Care was to explore the relevance and contribution of multimedia in healthcare and personal health. After a successful debut in Amsterdam in 2016, the second workshop at ACM Multimedia 2017 in Mountain View, California, again attracted more than 30 participants, indicating the relevance that this topic has in the multimedia research community. [ABSTRACT FROM AUTHOR]

Social image tag refinement, which aims to improve tag quality by automatically completing the missing tags and rectifying the noise-corrupted ones, is an essential component for social image search. Conventional approaches mainly focus on exploring the visual and tag information, without considering the user information, which often reveals important hints on the (in)correct tags of social images. Towards this end, we propose a novel tri-clustered tensor completion framework to collaboratively explore these three kinds of information to improve the performance of social image tag refinement. Specifically, the inter-relations among users, images and tags are modeled by a tensor, and the intra-relations between users, images and tags are explored by three regularizations respectively. To address the challenges of the super-sparse and large-scale tensor factorization that demands expensive computing and memory cost, we propose a novel tri-clustering method to divide the tensor into a certain number of sub-tensors by simultaneously clustering users, images and tags into a bunch of tri-clusters. And then we investigate two strategies to complete these sub-tensors by considering (in)dependence between the sub-tensors. Experimental results on a real-world social image database demonstrate the superiority of the proposed method compared with the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Heterogeneous data fusion from disparate geospatial sensors has drawn increasing attention in multimedia. Unfortunately, environmental sensors are usually sparsely and preferentially located, which restricts situation recognition of geographical regions and results in uncertainty in derived inferences. Spatial interpolation is an effective way to solve the problem of data sparsity, which demands the availability of related data sources. However, these data sources are usually in different resolutions, distributions, scales, and densities, which poses a major challenge in data integration. To address this problem, we present a novel spatial interpolation framework to incorporate diverse data sources and model the spatial processes explicitly at multiple resolutions. Spectral analysis is deployed to generate features at multiple spatial resolutions and to improve the interpolation accuracy at unobserved locations. A statistical operator based on the spatial Gaussian process is implemented and integrated into a geospatial situation recognition system, which can analyze heterogeneous spatio-temporal data streams derived from sensors. To verify the effectiveness and efficiency of the proposed framework, this framework is applied to the PM2.5 air pollution application. Experiments conducted in California, USA, demonstrate that the proposed method outperforms state-of-the-art approaches. [ABSTRACT FROM PUBLISHER]

The authors propose the social-sensed multimedia computing paradigm and advocate for the need to organically integrate social network and social media data with multimedia computing tasks. More researchers in the multimedia community should be focusing on the user dimension to quickly advance this line of research. [ABSTRACT FROM AUTHOR]

Food-related photos have become increasingly popular , due to social networks, food recommendations, and dietary assessment systems. Reliable annotation is essential in those systems, but unconstrained automatic food recognition is still not accurate enough. Most works focus on exploiting only the visual content while ignoring the context. To address this limitation, in this paper we explore leveraging geolocation and external information about restaurants to simplify the classification problem. We propose a framework incorporating discriminative classification in geolocalized settings and introduce the concept of geolocalized models, which, in our scenario, are trained locally at each restaurant location. In particular, we propose two strategies to implement this framework: geolocalized voting and combinations of bundled classifiers. Both models show promising performance, and the later is particularly efficient and scalable. We collected a restaurant-oriented food dataset with food images, dish tags, and restaurant-level information, such as the menu and geolocation. Experiments on this dataset show that exploiting geolocation improves around 30% the recognition performance, and geolocalized models contribute with an additional 3–8% absolute gain, while they can be trained up to five times faster. [ABSTRACT FROM PUBLISHER]

This special issue touches on many significant aspects of multimedia retrieval, including content analysis and understanding, content- and context-based indexing, search and retrieval, HCI technologies, and image and video summarization and visualization. It converges on the nexus of social multimedia and storytelling around real-world experiences, events, and places. Aside from the challenging research problems in this emerging area, its topics are linked to a host of important commercial and creative applications in sectors such as media, entertainment, arts and culture, sports, and music. [ABSTRACT FROM PUBLISHER]

A major disruption is taking place in terms of how photographs are captured and the role they play in modern society, offering the tantalizing possibility of creating visual connections and conversations beyond anything we've yet seen or imagined. The author reviews the evolution of visual documentation and considers where we're headed, introducing the Visual Web. This department is part of a special issue on social multimedia and storytelling. [ABSTRACT FROM PUBLISHER]

Humans have always been interested in understanding themselves and their environment. Understanding their relationship with the environment is important to survival as well as thriving in the present situation and planning for the future. With advances in technology, the 21st century has witnessed significant advances in storage, processing, sensing, and communication technologies. All these have resulted in the popularization of strong data-dependent approaches, leading to the rise in the popularity of scientism in almost all disciplines where data can be collected. As the availability of data has become widespread, the desire to understand the physical reality at different levels in different applications has also become possible and desirable. [ABSTRACT FROM PUBLISHER]