Search

Urban anomalies may result in loss of life or property if not handled properly. Automatically alerting anomalies in their early stage or even predicting anomalies before happening are of great value for populations. Recently, data-driven urban anomaly analysis frameworks have been forming, which utilize urban big data and machine learning algorithms to detect and predict urban anomalies automatically. In this survey, we make a comprehensive review of the state-of-the-art research on urban anomaly analytics. We first give an overview of four main types of urban anomalies, traffic anomaly, unexpected crowds, environment anomaly, and individual anomaly. Next, we summarize various types of urban datasets obtained from diverse devices, i.e., trajectory, trip records, CDRs, urban sensors, event records, environment data, social media and surveillance cameras. Subsequently, a comprehensive survey of issues on detecting and predicting techniques for urban anomalies is presented. Finally, research challenges and open problems as discussed., Comment: Accepted by IEEE Transactions on Big Data

With the prevalence of smartphones, people have left abundant behavior records in cyberspace. Discovering and understanding individuals' cyber activities can provide useful implications for policymakers, service providers, and app developers. In this paper, we propose a framework to discover daily cyber activity patterns across people's mobile app usage. We first segment app usage traces into small time windows and then design a probabilistic topic model to infer users' cyber activities of each window. By exploring the coherence of users' activity sequences, the daily patterns of individuals are identified. Next, we recognize the common patterns across diverse groups of individuals using a hierarchical clustering algorithm. We then apply our framework on a large-scale and real-world dataset, consisting of 653,092 users with 971,818,946 usage records of 2,000 popular mobile apps. Our analysis shows that people usually obey yesterday's activity patterns, but the patterns tend to deviate as the time-lapse increases. We also discover five common daily cyber activity patterns, including afternoon reading, nightly entertainment, pervasive socializing, commuting, and nightly socializing. Our findings have profound implications on identifying the demographics of users and their lifestyles, habits, service requirements, and further detecting other disrupting trends such as working overtime and addiction to the game and social media.

Crowd flow prediction is an essential task benefiting a wide range of applications for the transportation system and public safety. However, it is a challenging problem due to the complex spatio-temporal dependence and the complicated impact of urban structure on the crowd flow patterns. In this article, we propose a novel framework, 3- D imensional G raph C onvolution N etwork (3DGCN), to predict citywide crowd flow. We first model it as a dynamic spatio-temporal graph prediction problem, where each node represents a region with time-varying flows, and each edge represents the origin–destination (OD) flow between its corresponding regions. As such, OD flows among regions are treated as a proxy for the spatial interactions among regions. To tackle the complex spatio-temporal dependence, our proposed 3DGCN can model the correlation among graph spatial and temporal neighbors simultaneously. To learn and incorporate urban structures in crowd flow prediction, we design the GCN aggregator to be learned from both crowd flow prediction and region function inference at the same time. Extensive experiments with real-world datasets in two cities demonstrate that our model outperforms state-of-the-art baselines by 9.6%∼19.5% for the next-time-interval prediction.

A rapid-spreading epidemic of COVID-19 hit China at the end of 2019, resulting in unignorable social and economic damage in the epicenter, Wuhan. POIs capture the microscopic behavior of citizens, providing valuable information to understand city reactions toward the epidemic. Leveraging large-scale check-in records, we analyze the POI visit trends over the epidemic period and normal times. We demonstrate that COVID-19 greatly influences the society, where most POIs demonstrate more than 60% of visit drops during the city lockdown period. Among them, Tourist Attractions received greatest impact with a 78.8% drop. Entertainment, Food, Medical and Shopping are sensible to the disease before lockdown, and we identify these "early birds" to investigate the public reaction in the early stage of the epidemic. We further analyze the revival trends, generating four different revival patterns that correlated with the necessity of POI functions. Finally, we analyze the perseverance during the COVID-19, finding no large-scale closures compared with the tremendous visit drop. The strong resilience in Wuhan supports the rapid recovery of society. These findings are important for researchers, industries, and governments to understand the city respondence under severe epidemic, proposing better regulations to respond, control, and prevent public emergencies.

Publisher Copyright: © 2021 ACM. Copy-and-paste operations are the most popular features on computing devices such as desktop computers, smartphones and tablets. However, the copy-and-paste operations are not sufficiently addressed on the Augmented Reality (AR) smartglasses designated for real-time interaction with texts in physical environments. This paper proposes two system solutions, namely Granularity Scrolling (GS) and Two Ends (TE), for the copy-and-paste operations on AR smartglasses. By leveraging a thumb-size button on a touch-sensitive and pressure-sensitive surface, both the multi-step solutions can capture the target texts through indirect manipulation and subsequently enables the copy-and-paste operations. Based on the system solutions, we implemented an experimental prototype named Press-n-Paste (PnP). After the eight-session evaluation capturing 1,296 copy-and-paste operations, 18 participants with GS and TE achieve the peak performance of 17,574 ms and 13,951 ms per copy-and-paste operation, with 93.21% and 98.15% accuracy rates respectively, which are as good as the commercial solutions using direct manipulation on touchscreen devices. The user footprints also show that PnP has a distinctive feature of miniaturized interaction area within 12.65 mm∗14.48 mm. PnP not only proves the feasibility of copy-and-paste operations with the flexibility of various granularities on AR smartglasses, but also gives significant implications to the design space of pressure widgets as well as the input design on smart wearables.

In this paper, we investigate the feasibility of using mobility patterns and demographic data to predict hospital visits. We collect mobility traces from two thousand users for around two months. We extract 16 mobility features from these passively collected mobility traces and train an XGBoost model to predict users' hospital visits. We demonstrate that the designed mobility features can significantly improve prediction accuracy (p < 0.01, AUC = 0.79). We further analyze how these mobility features affect the prediction results and measure their importance by using Shapley additive explanation values. We discover that users with less mobility activity, less visit diversity, and few sports facilities, bountiful entertainment around their visited locations are more likely to visit hospitals. Moreover, we conduct predictions on the populations with different demographic features, which achieves meaningful and insightful results, i.e. maintaining a high mobility activity is crucial for older people's health, while fast food store more substantially affects younger people's health; visit patterns can indicate females' health, while the neighborhood environment is more indicative of males, etc. These results shed light on how to use and understand large scale mobility data in health monitoring and other health-related applications in practice.

Air pollution introduces a major challenge for societies, where it leads to the premature deaths of millions of people each year globally. Massive deployment of air quality sensing devices and data analysis for the resultant data will pave the way for the development of real-time intelligent applications and services, e.g., minimization of exposure to poor air quality either on an individual or city scale. 5G and edge computing supports dense deployments of sensors at high resolution with ubiquitous connectivity, high bandwidth, high-speed gigabit connections, and ultralow latency analysis. This article conceptualizes AI-powered scalable air quality monitoring and presents two systems of calibrating low-cost air quality sensors and the image processing of pictures captured by hyperspectral cameras to better detect air quality. We develop and deploy different AI algorithms in these two systems on a 5G edge testbed and perform a detailed analytics regarding to 1) the performance of AI algorithms and 2) the required communication and computation resources.

Due to explosive growth of mobile data over the overburdened cellular network, opportunistic offloading is proposed to transmit the traffic originally transmitted through infrastructure network to delay-tolerant device-to-device networks at the edge of mobile networks. Existing offloading schemes mainly focus on selecting a subset of mobile nodes to work as offloadees, who first download the contents through the infrastructure, and then transmit them to subscriber nodes. However, when there is large amount of contents to be offloaded, and if all contents are disseminated through only one fixed set of offloadees every time, it may result in the risk of extreme cases that only extremely few nodes received the content, even if the expectation is large. This is like putting all of your eggs in one basket. To deal with this problem, we introduce an economic concept, conditional value at risk (CVaR) to measure the risk in offloadee set selection. CVaR is a widely used risk assessment measure for investments, which provides a unified and comprehensive risk evaluation framework for complex investment portfolio. We propose an algorithm to compute a portfolio over multiple offloadee sets with a guarantee on CVaR. Moreover, we prove the feasibility of our methodology from the mathematical point of view. Extensive evaluations with real-world traces show that our proposed offloading scheme can obtain a portfolio with significantly better CVaR, i.e., 91.53 percent performance gain at most, compared with the state-of-the-art solutions.

Artificial intelligence shows promise for solving many practical societal problems in areas such as healthcare and transportation. However, the current mechanisms for AI model diffusion such as Github code repositories, academic project webpages, and commercial AI marketplaces have some limitations; for example, a lack of monetization methods, model traceability, and model auditabilty. In this work, we sketch guidelines for a new AI diffusion method based on a decentralized online marketplace. We consider the technical, economic, and regulatory aspects of such a marketplace including a discussion of solutions for problems in these areas. Finally, we include a comparative analysis of several current AI marketplaces that are already available or in development. We find that most of these marketplaces are centralized commercial marketplaces with relatively few models.

User authentication is key in user authorization on smart and personal devices. Over the years, several authentication mechanisms have been proposed: these also include behavioral-based biometrics. However, behavioral-based biometrics suffer from two issues: they are prone to degradation in performance (accuracy) over time (e.g., due to data distribution changes arising from user behavior) and the need to learn the machine learning model from scratch, when adding new users. In this paper, we propose ContAuth, a system that can enhance the robustness of behavioral-based authentication. ContAuth continuously adapts to new incoming data (data incremental learning) and is able to add new users without retraining (class incremental learning). Specifically, ContAuth combines deep learning models with online learning models to achieve learning on the fly, thereby preventing a severe drop in the accuracy between sessions (over time). To add new users, ContAuth employs class incremental learning methods. We evaluate ContAuth on multiple behavior-based user authentication modalities: breathing, gait. and EMG. Our results show that our framework can help True Positive Rate (TPR) to remain high (>85 %) compared to other methods for all the modalities except EMG (>70%) across the sessions while keeping False Positive Rates (FPR) at a minimum (0-10%). It can achieve up to 35% improvement in TPR over a traditional deep learning model. Additionally, iCaRL (an incremental learning method) enables ContAuth to allow the addition of new users by alleviating catastrophic forgetting, to a large extent. Finally, we also show that ContAuth can be deployed efficiently and effectively on device, further providing data privacy.

Steady State Motion Visual Evoked Potential (SSMVEP)-based Brain Computer Interface (BCI) is widely studied and has been used to varies of occasions on account of its good performance, mild stimulation, and free of additional training. We design a trolley control system based on SSMVEP signals and observe a phenomenon named “BCI Illiterate,” in which case some subjects present unsatisfactory performance with low classification accuracies. In order to cope with this challenging problem in real-world contexts, we introduce a deep learning (DL) method. The method allows improving the accuracies for both EEG literate and EEG illiterate. In particular, we firstly conduct SSMVEP experiments to obtain EEG signals from 10 subjects, including 5 EEG literates and 5 EEG illiterates. Then we construct a convolutional neural network with long short-term memory (CNN-LSTM) framework, which allows extracting the spectral, spatial, and temporal features of EEG signals, to realize the high classification accuracies of SSMVEP signals. The results show that DL method can reach 96.83% and 91.86% for EEG literate and EEG illiterate respectively, which are 12.68% and 31.08% higher than the results of traditional methods. These results indicate that DL method is not only suitable for EEG literate, but more importantly, can greatly improve the performance for EEG illiterate, which finally can enhance the robustness and universality of the SSMVEP-based BCI.

Influence map (IM) is often used as a decision supporting technology in game artificial intelligence (AI). However, the traditional influence map does not describe dynamic information. Some improved IM models can describe dynamic information, but not accurately enough. When an object moves, it would produce large influence in its moving direction than other directions. Therefore, the influence produce by the object to a location depends on the relation between the location and the object’s moving direction. This paper proposed a dynamic influence map model based on distance adjustment, DADIM. This model produces different influence values in different direction by adjusting the “distance” between two locations. This method can encode dynamic information into the influence map easily. Experiments show this model avoids the weakness of dynamic influence map with location prediction. Compared with other influence maps, this model could improve the performance of the game AI with time complexity being unchanged.

Geo-textual data is ubiquitous nowadays, where each object has a location and is associated with some keywords. Many types of queries based on geo-textual data, termed as spatial keyword queries, have been proposed, and are to find optimal object(s) in terms of both its (their) location(s) and keywords. In this paper, we propose a new type of query called nearby-fit spatial keyword query (NSKQ), where an optimal object is defined based not only on the location and the keywords of the object itself, but also on those of the objects nearby. For example, in an application of finding a hotel, not only the location of a hotel but also the objects near the hotel (e.g., shopping malls, restaurants, and bus stops nearby) might need to be taken into consideration. The query is proved to be NP-hard, and in order to perform the query efficiently, we developed two approximate algorithms with small constant approximation factors equal to 1.155 and 1.79. We conducted extensive experiments based on both real and synthetic datasets, which verified our algorithms. Nanyang Technological University Accepted version The research of Cheng Long is partially supported by NTU SUG M4082302.020.

The diversity of personal interest and preference of mobile users results in a wide spectrum of mobile app usage, and it is important to predict such app preference in order to provide personalized services. However, currently available individual app usage data is very limited, which does not cover a large user base. In this paper, we demonstrate that it is possible to make personalized app usage estimation by learning user's app preference from the social media, i.e., public accessible tweets, which can also reflect user's interest and make up for the sparsity of app usage data. By proposing a novel generative model named IMCF+ to transfer user interest from rich tweet information to sparse app usage, we achieve personalized app recommendations via learning the interest's correlation between apps and tweets. Based on a real-world app usage and tweet dataset over a large population, we evaluate the performance of IMCF+ with a variety of scenarios and parameters. With only 10 percent training data, our IMCF+ approach achieves 82.5 percent hitrate in predicting the top ten apps. Moreover, IMCF+ outperforms the other six state-of-the-art algorithms by 4.7 percent and 10 percent in high sparsity case and user cold-start scenario, indicating the effectiveness of our method. All these results demonstrate that our technique can reliably learn user's interest from tweets to help solve the personalized app recommendation problem. Our study is the first step forward for transferring user's interest learned from social media to app preference, which paves the way for providing higher-quality personalized app recommendation and services for mobile users.

Online social networks (OSNs) have become a commodity in our daily life. As an important concept in sociology and viral marketing, the study of social influence has received a lot of attentions in academia. Most of the existing proposals work well on dominant OSNs, such as Twitter, since these sites are mature and many users have generated a large amount of data for the calculation of social influence. Unfortunately, cold-start users on emerging OSNs generate much less activity data, which makes it challenging to identify potential influential users among them. In this work, we propose a practical solution to predict whether a cold-start user will become an influential user on an emerging OSN, by opportunistically leveraging the user’s information on dominant OSNs. A supervised machine learning-based approach is adopted, transferring the knowledge of both the descriptive information and dynamic activities on dominant OSNs. Descriptive features are extracted from the public data on a user’s homepage. In particular, to extract useful information from the fine-grained dynamic activities that cannot be represented by the statistical indices, we use deep learning technologies to deal with the sequential activity data. Using the real data of millions of users collected from Twitter (a dominant OSN) and Medium (an emerging OSN), we evaluate the performance of our proposed framework to predict prospective influential users. Our system achieves a high prediction performance based on different social influence definitions.

Many schemes have been recently advanced for storing data on multiple clouds. Distributing data over different cloud storage providers (CSPs) automatically provides users with a certain degree of information leakage control, for no single point of attack can leak all the information. However, unplanned distribution of data chunks can lead to high information disclosure even while using multiple clouds. In this paper, we study an important information leakage problem caused by unplanned data distribution in multicloud storage services. Then, we present StoreSim , an information leakage aware storage system in multicloud. StoreSim aims to store syntactically similar data on the same cloud, thus minimizing the user's information leakage across multiple clouds. We design an approximate algorithm to efficiently generate similarity-preserving signatures for data chunks based on MinHash and Bloom filter, and also design a function to compute the information leakage based on these signatures. Next, we present an effective storage plan generation algorithm based on clustering for distributing data chunks with minimal information leakage across multiple clouds. Finally, we evaluate our scheme using two real datasets from Wikipedia and GitHub . We show that our scheme can reduce the information leakage by up to 60 percent compared to unplanned placement. Furthermore, our analysis on system attackability demonstrates that our scheme makes attacks on information more complex.

Vehicular communication applications require an efficient communication architecture for timely information delivery. Centralized, cloud-based infrastructures present latencies too high to satisfy the requirements of emergency information processing and transmission, while Vehicle-to-Vehicle communication is too variable for reliable in-time information transmission. In this paper, we present EAVVE, a novel Vehicle-to-Everything system, consisting of vehicles with and without comprehensive data processing capabilities, facilitated by edge servers co-located with roadside units. Adding computation capabilities at the edge of the network allows reducing the overall latency compared to vehicle-to-cloud and makes up for scenarios in which in-vehicle computational power is not sufficient to satisfy the service demand. To improve the offloading efficiency, we propose a decentralized algorithm for real-time task scheduling and a client/server algorithm for information filtering. We demonstrate the practical applications of EAVVE with a bandwidth-hungry, latency constrained real-life prototype system that connects vehicular vision through Augmented Reality vision. We evaluate this prototype system with real-life road tests. We complement this practical evaluation with extensive simulations based on real-world base station and vehicular traffic data to demonstrate the scalability of EAVVE and its performance in citywide scenarios. EAVVE decreases the latency by 42.6% and 78.7% compared to local and remote cloud solutions while relaxing congestion at the bottleneck by 99% with reasonable infrastructure expenditure.

Being a leading online service providing both local search and social networking functions, Foursquare has attracted tens of millions of users all over the world. Understanding the user behavior of Foursquare is helpful to gain insights for location-based social networks (LBSNs). Most of the existing studies focus on a biased subset of users, which cannot give a representative view of the global user base. Meanwhile, although the user-generated content (UGC) is very important to reflect user behavior, most of the existing UGC studies of Foursquare are based on the check-ins. There is a lack of a thorough study on tips, the primary type of UGC on Foursquare. In this article, by crawling and analyzing the global social graph and all published tips, we conduct the first comprehensive user behavior study of all 60+ million Foursquare users around the world. We have made the following three main contributions. First, we have found several unique and undiscovered features of the Foursquare social graph on a global scale, including a moderate level of reciprocity, a small average clustering coefficient, a giant strongly connected component, and a significant community structure. Besides the singletons, most of the Foursquare users are weakly connected with each other. Second, we undertake a thorough investigation according to all published tips on Foursquare. We start from counting the numbers of tips published by different users and then look into the tip contents from the perspectives of tip venues, temporal patterns, and sentiment. Our results provide an informative picture of the tip publishing patterns of Foursquare users. Last but not least, as a practical scenario to help third-party application providers, we propose a supervised machine learning-based approach to predict whether a user is an influential by referring to the profile and UGC, instead of relying on the social connectivity information. Our data-driven evaluation demonstrates that our approach can reach a good prediction performance with an F1-score of 0.87 and an AUC value of 0.88. Our findings provide a systematic view of the behavior of Foursquare users and are constructive for different relevant entities, including LBSN service providers, Internet service providers, and third-party application providers.

Service liability interconnections among globally networked IT- and IoT-driven service organizations create potential channels for cascading service disruptions worth billions of dollars, due to modern cyber-crimes such as DDoS, APT, and ransomware attacks. A natural question that arises in this context is: What is the likelihood of a cyber-blackout? , where the latter term is defined as the probability that all (or a major subset of) organizations in a service chain become dysfunctional in a certain manner due to a cyber-attack at some or all points in the chain. The answer to this question has major implications to risk management businesses such as cyber-insurance when it comes to designing policies by risk-averse insurers for providing coverage to clients in the aftermath of such catastrophic network events. In this article, we investigate this question in general as a function of service chain networks and different cyber-loss distribution types. We show somewhat surprisingly (and discuss the potential practical implications) that, following a cyber-attack, the effect of (a) a network interconnection topology and (b) a wide range of loss distributions on the probability of a cyber-blackout and the increase in total service-related monetary losses across all organizations are mostly very small. The primary rationale behind these results are attributed to degrees of heterogeneity in the revenue base among organizations and the Increasing Failure Rate property of popular (i.i.d/non-i.i.d) loss distributions, i.e., log-concave cyber-loss distributions. The result will enable risk-averse cyber-risk managers to safely infer the impact of cyber-attacks in a worst-case network and distribution oblivious setting.

Correction IEEE ACCESS Volume9 Pages14179-14180 Article Numbere abe4038 DOI10.1109/ACCESS.2021.3051825 Published 2021 In the modern era of the mobile apps (the era of surveillance capitalism - as termed by Shoshana Zuboff) huge quantities of surveillance data about consumers and their activities offer a wave of opportunities for economic and societal value creation. ln-app advertising - a multi-billion dollar industry, is an essential part of the current digital ecosystem driven by free mobile applications, where the ecosystem entities usually comprise consumer apps, their clients (consumers), ad-networks, and advertisers. Sensitive consumer information is often being sold downstream in this ecosystem without the knowledge of consumers, and in many cases to their annoyance. While this practice, in cases, may result in long-term benefits for the consumers, it can result in serious information privacy breaches of very significant impact (e.g., breach of genetic data) in the short term. The question we raise through this paper is: Is it economically feasible to trade consumer personal information with their formal consent (permission) and in return provide them incentives (monetary or otherwise)?. In view of (a) the behavioral assumption that humans are 'compromising' beings and have privacy preferences, (b) privacy as a good not having strict boundaries, and (c) the practical inevitability of inappropriate data leakage by data holders downstream in the data-release supply-chain, we propose a design of regulated efficient/bounded inefficient economic mechanisms for oligopoly data trading markets using a novel preference function bidding approach on a simplified sellers-broker market. Our methodology preserves the heterogeneous privacy preservation constraints (at a grouped consumer, i.e., app, level) upto certain compromise levels, and at the same time satisfies information demand (via the broker) of agencies (e.g., advertising organizations) that collect client data for the purpose of targeted behavioral advertising.

In the traditional cloud architecture, data needs to be uploaded to the cloud for processing, bringing delays in transmission and response. Edge network emerges as the times require. Data processing on the edge nodes can reduce the delay of data transmission and improve the response speed. In recent years, the need for artificial intelligence of edge network has been proposed. However, the data of a single, individual edge node is limited and does not satisfy the conditions of machine learning. Therefore, performing edge network machine learning under the premise of data confidentiality became a research hotspot. This paper proposes a Privacy-Preserving Asynchronous Federated Learning Mechanism for Edge Network Computing (PAFLM), which can allow multiple edge nodes to achieve more efficient federated learning without sharing their private data. Compared with the traditional distributed learning, the proposed method compresses the communications between nodes and parameter server during the training process without affecting the accuracy. Moreover, it allows the node to join or quit in any process of learning, which can be suitable to the scene with highly mobile edge devices.

Computational Politics is the study of computational methods to analyze and moderate users' behaviors related to political activities such as election campaign persuasion, political affiliation, and opinion mining. With the rapid development and ease of access to the Internet, Information Communication Technologies (ICT) have given rise to massive numbers of users joining online communities and the digitization of political practices such as debates. These communities and digitized data contain both explicit and latent information about users and their behaviors related to politics and social movements. For researchers, it is essential to utilize data from these sources to develop and design systems that not only provide solutions to computational politics but also help other businesses, such as marketers, to increase users' participation and interactions. In this survey, we attempt to categorize main areas in computational politics and summarize the prominent studies in one place to better understand computational politics across different and multidimensional platforms. e.g., online social networks, online forums, and political debates. We then conclude this study by highlighting future research directions, opportunities, and challenges.

Mobile crowd sensing is a new paradigm that enables smart mobile devices to collect and share various types of sensing data in urban environments. However, new challenges arise: one is how to evaluate the quality of data each mobile user potentially is capable of providing; another is how to allocate a satisfactory yet profitable amount of reward to mobile users in order to keep them participating in crowd sensing tasks. In this paper, we first introduce a mathematical model for characterizing quality of sensing data to be contributed by mobile users. Then, we present a utility function and formulate an optimization problem for the platform, who recruits participants to contribute sensing data, to maximize the amount of high quality sensing data under a limited task budget. We next present an effective and quality-aware incentive mechanism to solve this problem for online scenarios where participants may arrive or leave at any random time. Moreover, the proposed incentive mechanism allows the platform to provide selected participants with an extra bonus according to task completion level and their previous performance to motivate them further. We formally show the proposed mechanism has the desirable properties of truthfulness, individual rationality, budgetary feasibility, and computational efficiency. We compare the proposed scheme with existing methods via simulation using a real dataset. Extensive simulation results well justify the effectiveness and robustness of the proposed approach, e.g, compared with another online method “OMG”, the gap to the optimum for our proposed Online-QIM approach is reduce by 33.3 percent when budget $B = 1000$B=1000.

Emotion is a complex mental state that guides people's thoughts and behaviors. The ability to recognize one's own emotions and to understand others' emotions helps people manage their personal lives and social relations more successfully. Meanwhile, mobile and wearable devices have become indispensable tools for most of the population. They provide great opportunities for building emotional intelligent applications. In this article, we introduce how mobile and wearable devices work as "emotion sensors" by leveraging their sensing, computing, and communication capabilities, which help monitor people's mental health, facilitate social interactions, and improve user experience. We first describe a general emotion sensing framework that consists of sensing, inferring, and responding steps. After reviewing the state-of-the-art systems on this topic, we summarize and discuss various sensing modalities enabled by mobile and wearable devices. We then describe the widely used inferring procedures and methods. Additionally, we present three solutions of facial expression recognition on smartphones as a case of emotion inference. Finally, we introduce emerging emotion sensing applications, and discuss challenges and opportunities.

Edge intelligence refers to a set of connected systems and devices for data collection, caching, processing, and analysis proximity to where data are captured based on artificial intelligence. Edge intelligence aims at enhancing data processing and protects the privacy and security of the data and users. Although recently emerged, spanning the period from 2011 to now, this field of research has shown explosive growth over the past five years. In this article, we present a thorough and comprehensive survey of the literature surrounding edge intelligence. We first identify four fundamental components of edge intelligence, i.e., edge caching, edge training, edge inference, and edge offloading based on theoretical and practical results pertaining to proposed and deployed systems. We then aim for a systematic classification of the state of the solutions by examining research results and observations for each of the four components and present a taxonomy that includes practical problems, adopted techniques, and application goals. For each category, we elaborate, compare, and analyze the literature from the perspectives of adopted techniques, objectives, performance, advantages and drawbacks, and so on. This article provides a comprehensive survey of edge intelligence and its application areas. In addition, we summarize the development of the emerging research fields and the current state of the art and discuss the important open issues and possible theoretical and technical directions.

The ubiquity of smart devices, combined with a lack of information about data garnered by them, make privacy a significant challenge for adopting smart devices. Ensuring users can safeguard their privacy without compromising the devices’ functionality requires effective yet intuitive ways to manage personal privacy preferences. Current solutions for privacy management are severely lacking as they are ineffective in making users aware of potential privacy risks or how to mitigate them and as they offer limited support for interaction. As our first contribution, we develop a novel AR privacy management interface (PARA) that uses AR visualization to contextualize data disclosure and improve user’s perceptions of privacy threats. Besides offering support for enhancing user’s privacy perceptions, our interface supports privacy control on compatible devices through privacy-enhancing technologies. As our second contribution, we systematically study factors affecting privacy perceptions and privacy control for two device classes (smart camera and smart speaker) through a user study with N = 32 participants. Our results show that PARA’s contextualization and visualization of privacy disclosure strongly affect the participants’ privacy perceptions. For privacy control, we demonstrate that our prototype improves the participant’s capability to identify risks and provides an effective and easy-to-use mechanism for controlling privacy disclosure, in contrast to existing state-of-the-art privacy management interfaces.

Research on smart device privacy has consistently highlighted how privacy is an important concern for users, but they fail to act on their concerns. While this discrepancy between user perceptions and actions has been consistently reported, currently there is a limited understanding of why this is the case or how the situation can be ameliorated. This paper systematically studies how visualizations in privacy assistants can improve the situation, reporting on two studies that explore the users' privacy perceptions in smart device ecosystems. The first study shows that displaying device location and data type reduces the users' privacy perceptions. Participants also weigh the use of media such as online news as a source to inform users about the possible inferences. The second study analyzes participants' preferences to visualize smart device information and privacy policies using augmented reality. Through these two studies, we derive insights and guidelines on how to design effective privacy assistants and to improve user's knowledge of risks associated with data disclosure in smart home scenarios.

Augmented Reality (AR) offers new capabilities for blurring the boundaries between physical reality and digital media. However, the capabilities of integrating web contents and AR remain underexplored. This paper presents an AR web browser with an integrated context-aware AR-to-Web content recommendation service named as A2W browser, to provide continuously user-centric web browsing experiences driven by AR headsets. We implement the A2W browser on an AR headset as our demonstration application, demonstrating the features and performance of A2W framework. The A2W browser visualizes the AR-driven web contents to the user, which is suggested by the content-based filtering model in our recommendation system. In our experiments, 20 participants with the adaptive UIs and recommendation system in A2W browser achieve up to 30.69% time saving compared to smartphone conditions. Accordingly, A2W-supported web browsing on workstations facilitates the recommended information leading to 41.67% faster reaches to the target information than typical web browsing.

Many factors affect speech intelligibility in face-to-face conversations. These factors lead conversation participants to speak louder and more distinctively, exposing the content to potential eavesdroppers. To address these issues, we introduce Theophany, a privacy-preserving framework for augmenting speech. Theophany establishes ad-hoc social networks between conversation participants to exchange contextual information, improving speech intelligibility in real-time. At the core of Theophany, we develop the first privacy perception model that assesses the privacy risk of a face-to-face conversation based on its topic, location, and participants. This framework allows to develop any privacy-preserving application for face-to-face conversation. We implement the framework within a prototype system that augments the speaker's speech with real-life subtitles to overcome the loss of contextual cues brought by mask-wearing and social distancing during the COVID-19 pandemic. We evaluate Theophany through a user survey and a user study on 53 and 17 participants, respectively. Theophany's privacy predictions match the participants' privacy preferences with an accuracy of 71.26%. Users considered Theophany to be useful to protect their privacy (3.88/5), easy to use (4.71/5), and enjoyable to use (4.24/5). We also raise the question of demographic and individual differences in the design of privacy-preserving solutions.

The current explosion of video traffic compels service providers to deploy caches at edge networks. Nowadays, most caching systems store data with a high programming voltage corresponding to the largest possible ‘expiry date’, typically on the order of years, which maximizes the cache damage. However, popular videos rarely exhibit lifecycles longer than a couple of months. Consequently, the programming voltage can instead be adapted to fit the lifecycle and mitigate the cache damage accordingly. In this paper, we propose LiA-cache, a Lifecycle-Aware caching policy for online videos. LiA-cache finds both near-optimal caching retention times and cache eviction policies by optimizing traffic delivery cost and cache damage cost conjointly. We first investigate temporal patterns of video access from a real-world dataset covering 10 million online videos collected by one of the largest mobile network operators in the world. We next cluster the videos based on their access lifecycles and integrate the clustering into a general model of the caching system. Specifically, LiA-cache analyzes videos and caches them depending on their cluster label. Compared to other popular policies in real-world scenarios, LiA-cache can reduce cache damage up to 90%, while keeping a cache hit ratio close to a policy purely relying on video popularity.

Digital tools for collocated synchronous communication, like backchannels and question management systems, are increasingly used in events and education to support interaction between performer(s) and audience. Discussions in the communication tool are prone to various unexpected interruptions which violate against the communication tools' original purpose, i.e. violating performers' expected norms. We study such norm violations and related performers' coping approaches in two 15-week-long field experiments in two elementary schools. Our results demonstrate how the collocated synchronous tools form dual-channel communication environment where both physical and digital channel are in use at the same time. We show how norm violations 'spill over' from their onset in one channel to continue in the other. However, we observed performers coping approaches were primarily focused in the physical channel. This resulted in an asymmetry of presence and delayed the performers' ability to act on violations. Our findings show that the channels in dual-channel communication are tightly interwoven and must be considered as an integrated communication space instead of two co-occurring but separate channels. We provide a framework to study norm violations and discuss implications of this framework on the design of these systems.

Incremental and quantitative improvements of two-way interactions with e x tended realities (XR) are contributing toward a qualitative leap into a state of XR ecosystems being efficient, user-friendly, and widely adopted. However, there are multiple barriers on the way toward the omnipresence of XR; among them are the following: computational and power limitations of portable hardware, social acceptance of novel interaction protocols, and usability and efficiency of interfaces. In this article, we overview and analyse novel natural user interfaces based on sensing electrical bio-signals that can be leveraged to tackle the challenges of XR input interactions. Electroencephalography-based brain-machine interfaces that enable thought-only hands-free interaction, myoelectric input methods that track body gestures employing electromyography, and gaze-tracking electrooculography input interfaces are the examples of electrical bio-signal sensing technologies united under a collective concept of ExG. ExG signal acquisition modalities provide a way to interact with computing systems using natural intuitive actions enriching interactions with XR. This survey will provide a bottom-up overview starting from (i) underlying biological aspects and signal acquisition techniques, (ii) ExG hardware solutions, (iii) ExG-enabled applications, (iv) discussion on social acceptance of such applications and technologies, as well as (v) research challenges, application directions, and open problems; evidencing the benefits that ExG-based Natural User Interfaces inputs can introduce to the area of XR.

Attention-based techniques have been successfully used for rating image quality, and have been widely employed for set-based face recognition. Nevertheless, for video face recognition, where the base convolutional neural network (CNN) trained on large-scale data already provides discriminative features, fusing features with only predicted quality scores to generate representation are likely to cause duplicate sample dominant problem, and degrade performance correspondingly. To resolve the problem mentioned above, we propose a redundancy removing aggregation network (RRAN) for video face recognition. Compared with other quality-aware aggregation schemes, RRAN can take advantage of similarity information to tackle the noise introduced by redundant video frames. By leveraging metric learning, RRAN introduces a distance calibration scheme to align distance distributions of negative pairs of different video representations, which improves the accuracy under a uniform threshold. A series of experiments is conductedon multiple realistic data sets to evaluate the performance of RRAN, including YouTube Faces, IJB-A, and IJB-C. In comprehensive experiments, we demonstrate that our method can diminish the overall influence of poor quality components with large proportion in the video and further improve the overall recognition performance with individual difference. Specifically, RRAN achieves a 96.84% accuracy on YouTube Face, outperforming all existing aggregation schemes.

Although many children are born with congenital limb malformation, contemporary functional artificial hands are costly and are not meant to be adapted to growing hand. In this work, we develop a low cost, adaptable and personalizable system of an artificial prosthetic hand accompanied with hardware and software modules. Our solution consists of (i) a consumer grade electromyography (EMG) recording hardware, (ii) a mobile companion device empowered by deep learning classification algorithms, (iii) an cloud component for offloading computations, and (iv) mechanical 3D printed arm operated by the embedded hardware. We focus on the flexibility of the designed system making it more affordable than the alternatives. We use 3D printed materials and open-source software thus enabling the community to contribute and improve the system. In this paper, we describe the proposed system and its components and present the experiments we conducted in order to show the feasibility and applicability of our approach. Extended experimentation shows that our proposal is energy efficient and has high accuracy.

As Location-Based Social Networks (LBSNs) have become widely used by users, understanding user engagement and predicting user churn are essential to the maintainability of the services. In this work, we conduct a quantitative analysis to understand user engagement patterns exhibited both offline and online in LBSNs. We employ two large-scale datasets which consist of 1.3 million and 62 million users with 5.3 million reviews and 19 million tips in Yelp and Foursquare, respectively. We discover that users keep traveling to diverse locations where they have not reviewed before, which is in contrast to "human life" analogy in real life, an initial exploration followed by exploitation of existing preferences. Interestingly, we find users who eventually leave the community show distinct engagement patterns even with their first ten reviews in various facets, e.g., geographical, venue-specific, linguistic, and social aspects. Based on these observations, we construct predictive models to detect potential churners. We then demonstrate the effectiveness of our proposed features in the churn prediction. Our findings of geographical exploration and online interactions of users enhance our understanding of human mobility based on reviews, and provide important implications for venue recommendations and churn prediction.

In some applicable scenarios, such as community patrolling, mobile nodes are restricted to move only in their own communities. Exploiting the meetings of the nodes within the same community and the nodes within the neighboring communities, a delay tolerant network (DTN) can provide communication between any two nodes. In this paper, two analytical models based on stochastic reward nets (SRNs) are proposed to evaluate the performance of the epidemic content retrieval in such multi-community DTNs. Performance measures computed by the proposed models are the average retrieval delay and the average number of transmissions. The monolithic SRN model proposed in the first step is not scalable, in terms of the number of communities and nodes, due to the state space explosion in the underlying Markov chain. In order to solve the scalability problem of the monolithic model, an approximate model based on the folding technique is presented which allows us to evaluate the performance of large-scale DTNs. In order to cross-validate the results obtained from the proposed models, we extend the ONE simulator to support our network model. The analytic-numeric results indicate that both models have good accuracy, and the folded model reduces the state space highly, achieving good scalability without any significant loss of accuracy.

In essence, information retrieval is a cognitive process, which is the matching process between the reader's or user's cognition and the system's pre-set cognition. At present, cognitive view has become a research hotspot in the field of information retrieval. The research of information retrieval cognitive model has laid a theoretical foundation for the development of information retrieval system. Based on scarcity theory, this paper analyses the dynamic change of user's cognitive ability and constructs a cognitive model of information retrieval based on scarcity theory. After analysis, this paper evaluates the effect of information retrieval cognitive model based on scarcity theory, and clarifies the specific problems that the model can solve.

In developing countries, reliable electricity access is often undermined by the absence of supply from the national power grid and/or load shedding. To alleviate this problem, smart micro-grid (SMG) networks that are small scale distributed electricity provision networks composed of individual electricity providers and consumers, are being increasingly deployed. To ensure the reliable operation of SMGs, monitoring is necessary for data collection and state estimation processes. However, highly calibrated and trustworthy smart meters that are ideally suited to perform such monitoring tasks are often costly and non-ideally suited to SMGs which operate under unreliable communication network infrastructures. As a result, SMGs are an easy target to an adversary who can very easily gain access to private information by monitoring transmission between nodes in the SMG network, and launch inference-based privacy attacks. These attacks lead to electricity theft and grid instability problems in the SMG. The widely popular differential privacy (DP) technique (a rigorous technique in the family of privacy-preserving data publishing (PPDP) techniques to mathematically guarantee the preservation of data privacy) does not address multi-attribute correlations, that are inherently exploited by an adversary in inference attacks. In this paper, we propose HIDE, an oblivious computationally efficient, and rigorous information-theoretic privacy engineering framework for datasets/databases arising in the SMG environments that robustly accounts for multi-attribute correlations while preserving data privacy in a provably optimal fashion. A salient and powerful advantage of HIDE is its ability to generate optimal utility-privacy tradeoffs (computationally efficiently) when the privacy preserving entity in the worst case might have no prior statistical information that links a user’s private data with his public data.

Execution speed seriously bothers application developers and users for wearable devices such as Google Glass. Intensive applications like 3D games suffer from significant delays when CPU is busy. Energy is another concern when the devices are in low battery level, but users need them for urgency use. To ease such pains, one approach is to expand the computational power by cloud offloading. This paradigm works well when the available Internet access has enough bandwidth. Another way is to leverage nearby devices for computation-offloading, which is known as device-to-device (D2D) offloading. In this paper, we present Dandelion, a unified code offloading system for wearable computing. Such applications can leverage both the nearby devices and cloud for performance acceleration and energy efficiency. Dandelion is a novel generic code offloading system for wearable computing with a reference implementation on Google Glass. Dandelion includes a programmer-friendly framework based on Java annotation, a lightweight offloading service, and a runtime task scheduler to make offloading decisions. We design some wearable applications and several parallel execution benchmark methods for Dandelion performance evaluation. Extensive experiments on a testbed of Google Glass and Android phones demonstrate that Dandelion generally achieves over 5X execution speedup for local execution and can quickly recover from errors caused by network disruption.

D2D interactions are promoted both in the context of traffic and computation offloading. Applications with functionalities that are executed in parallel in more than one mobile device have been proposed while the idea of using another mobile device as a relay is going to be part of 5G. In this work, we first present the main types of D2D applications, then we discuss the characteristics of task offloading and the quality of experience in D2D ecosystems. After that, we list five popular questions regarding the ability of other users to be helpful in terms of resources, connectivity, availability, incentives, and security. To answer the listed questions, we use two mobile big data datasets, one experimental study, and arguments from our experience and the literature.

In this paper, we design, implement, and evaluate , which is to our knowledge the first consumer-class system that streams panoramic videos far beyond the ultra high-definition resolution (up to 16K) to mobile devices, offering truly immersive experiences. Such an immense resolution makes streaming video-on-demand (VoD) content extremely resource-demanding. To tackle this challenge, introduces a novel framework that leverages an edge server to perform efficient, intelligent, and quality-guaranteed content transcoding, by extracting from panoramic frames the viewport stream that will be delivered to the client. To support real-time transcoding of 16K content, employs several key mechanisms such as dual-GPU acceleration, lossless viewport extraction, deep viewport prediction, and a two-layer streaming design. Our extensive evaluations using real users’ viewport movement data indicate that outperforms existing solutions, and can smoothly stream 16K panoramic videos to mobile devices over diverse wireless networks including WiFi, LTE, and mmWave 5G.

https://doi.org/10.1049/blc2.12001 Abstract Peer-to-peer (P2P) networks utilize centralized entities (trackers) to assist peers in finding and exchanging information. Although modern P2P protocols are now trackerless and their function relies on distributed hash tables (DHTs), centralized entities are still needed to build file indices (indexing) and assist users in joining DHT swarms (bootstrapping). Although the functionality of these centralized entities are limited, every peer in the network is expected to trust them to function as expected (e.g. to correctly index new files). In this work, a new approach for designing and building decentralized online applications is proposed by introducing DIBDApp. The approach combines blockchain, smart contracts and BitTorrent for building up a combined technology that permits to create decentralized applications that do not require any assistance from centralized entities. DIBDApp is a software library composed of Ethereum smart contracts and an API to the BitTorrent protocol that fully decentralizes indexing, bootstrapping and file storing. DIBDApp enables any peer to seamlessly connect to the designed smart contracts via the Web3J protocol. Extensive experimentation on the Rinkeby Ethereum testnet shows that applications built using the DIBDApp library can perform the same operations as in traditional back-end architectures with a gas cost of a few USD cents.

This work has been partially supported by the UK EPSRC under grant number EP/N007565/1, “Science of Sensor Systems Software”, and by Academy of Finland projects, grant number 325774, 3196669, 319670, 326305, and 325570. The advances of sensing and computing technologies pave the way to develop novel applications and services for wearable devices. For example, wearable devices measure heart rate, which accurately reflects the intensity of physical exercise. Therefore, heart rate prediction from wearable devices benefits users with optimization of the training process. Conventionally, Cloud collects user data from wearable devices and conducts inference. However, this paradigm introduces significant privacy concerns. Federated learning is an emerging paradigm that enhances user privacy by remaining the majority of personal data on users’ devices. In this paper, we propose a statistically sound, Bayesian inference federated learning for heart rate prediction with autoregression with exogenous variable (ARX) model. The proposed privacy-preserving method achieves accurate and robust heart rate prediction. To validate our method, we conduct extensive experiments with real-world outdoor running exercise data collected from wearable devices. Postprint

In recent years, the number of space exploration missions has multiplied. Such an increase raises the question of effective communication between the multitude of human-made objects spread across our solar system. An efficient and scalable communication architecture presents multiple challenges, including the distance between planetary entities, their motion and potential obstruction, the limited available payload on satellites, and the high mission cost. This paper brings together recent relevant specifications, standards, mission demonstrations, and the most recent proposals to develop a unified architecture for deep-space internetworked communication. After characterizing the transmission medium and its unique challenges, we explore the available communication technologies and frameworks to establish a reliable communication architecture across the solar system. We then draw an evolutive roadmap for establishing a scalable communication architecture. This roadmap builds upon the mission-centric communication architectures in the upcoming years towards a fully interconnected network or InterPlanetary Internet (IPN). We finally discuss the tools available to develop such an architecture in the short, medium, and long terms. The resulting architecture cross-supports space agencies on the solar system-scale while significantly decreasing space communication costs. Through this analysis, we derive the critical research questions remaining for creating the IPN regarding the considerable challenges of space communication.

Question-and-answer (Q&A) websites are one of the latest evolutions in crowdsourced knowledge aggregation. Q&A websites provide more diverse opinions, as they involve the entire community. Quora made its reputation out of enhancing the traditional Q&A model with popular aspects of social media and incites its users to provide their names, locations, and references. This model allows higher quality control � including anonymous content, but more importantly, it leads users to form communities based on other criteria (e.g. profession, city) than similar interests. In this paper, we study the interactions among Quorans to unveil how such communities emerge. We perform both quantitative and qualitative analysis on the user-generated content and relate this content to social and demographic features. We show that being anonymous significantly affects the answers' length and subjectivity. On the other hand, most of the user interactions relate to their geographic locations.

Smartphones are nowadays the dominant end-user device. As a result, they have become gateways to all users' communications, including sensitive personal data. In this paper, we present Aquilis, a privacy-preserving system for mobile platforms following the principles of contextual integrity to define the appropriateness of an information flow. Aquilis takes the form of a keyboard that reminds users of potential privacy leakages through a simple three-colour code. Aquilis considers the instantaneous privacy risk related to posting information (Local Sensitivity), the risk induced by repeating information over time (Longitudinal Sensitivity) and on different platforms (Cross-platform Sensitivity). Considering 50% of Aquilis warnings decreases the proportion of inappropriate information by up to 30%. Repeating information over time or in a broader exposure context increases the risk by 340% in a one-to-one context. We develop our own labeled privacy dataset of over 1000 input texts to evaluate Aquilis' accuracy. Aquilis significantly outperforms other state-of-the-art methods (F-1-0.76). Finally, we perform a user study with 35 highly privacy-aware participants. Aquilis privacy metric is close to users' privacy preferences (average divergence of 1.28/5). Users found Aquilis useful (4.41/5), easy to use (4.4/5), and agreed that Aquilis improves their online privacy awareness (4.04/5).

Wearable devices collect user information about their activities and provide insights to improve their daily lifestyles. Smart health applications have achieved great success by training Machine Learning (ML) models on a large quantity of user data from wearables. However, user privacy and scalability are becoming critical challenges for training ML models in a centralized way. Federated learning (FL) is a novel ML paradigm with the goal of training high quality models while distributing training data over a large number of devices. In this demo, we present FL4W, a FL system with wearable devices enabling training a human activity recognition classifier. We also perform preliminary analytics to investigate the model performance with increasing computation of clients.

We propose Agora, the first blockchain-based data marketplace that enables multiple privacy-concerned parties to get compensated for contributing and exchanging data, without relying on a trusted third party during the exchange. Agora achieves data privacy, output verifiability, and atomicity of payments by leveraging cryptographic techniques, and is designed as a decentralized application via smart contracts. Particularly, data generators provide encrypted data to data brokers who use a functional secret key to learn nothing but the output of a specific, agreed upon, function over the raw data. Data consumers can purchase decrypted outputs from the brokers, accompanied by corresponding proofs of correctness. We implement a working prototype of Agora on Ethereum and experimentally evaluate its performance and deployment costs. As a core building block of Agora, we propose a new functional encryption scheme with additional public parameters that operate as a trust anchor for verifying decrypted results.

Smartwatches and other wearables are characterized by small-scale touchscreens that complicate the interaction with content. In this paper, we present Force9, the first optimized miniature keyboard leveraging force-sensitive touchscreens on wrist-worn computers. Force9 enables character selection in an ambiguous layout by analyzing the trade-off between interaction space and the easiness of force-assisted interaction. We argue that dividing the screen’s pressure range into three contiguous force levels is sufficient to differentiate characters for fast and accurate text input. Our pilot study captures and calibrates the ability of users to perform force-assisted touches on miniature-sized keys on touchscreen devices. We then optimize the keyboard layout considering the goodness of character pairs (with regards to the selected English corpus) under the force-based configuration and the users? familiarity with the QWERTY layout. We finally evaluate the performance of the trimetric optimized Force9 layout, and achieve an average of 10.18 WPM by the end of the final session. Compared to the other state-of-the-art approaches, Force9 allows for single-gesture character selection without addendum sensors.

In the past decade, the heterogeneous information network (HIN) has become an important methodology for modern recommender systems. To fully leverage its power, manually designed network templates, i.e., meta-structures, are introduced to filter out semantic-aware information. The hand-crafted meta-structure rely on intense expert knowledge, which is both laborious and data-dependent. On the other hand, the number of meta-structures grows exponentially with its size and the number of node types, which prohibits brute-force search. To address these challenges, we propose Genetic Meta-Structure Search (GEMS) to automatically optimize meta-structure designs for recommendation on HINs. Specifically, GEMS adopts a parallel genetic algorithm to search meaningful meta-structures for recommendation, and designs dedicated rules and a meta-structure predictor to efficiently explore the search space. Finally, we propose an attention based multi-view graph convolutional network module to dynamically fuse information from different meta-structures. Extensive experiments on three real-world datasets suggest the effectiveness of GEMS, which consistently outperforms all baseline methods in HIN recommendation. Compared with simplified GEMS which utilizes hand-crafted meta-paths, GEMS achieves over $6\%$ performance gain on most evaluation metrics. More importantly, we conduct an in-depth analysis on the identified meta-structures, which sheds light on the HIN based recommender system design., Published in Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM '20)