Search

Nowadays, many cities are suffering from air pollution problems, which endangered the health of the young and elderly for breathing problems. For supporting the government's policy-making and people's decision making, it is important to predict future fine-grained air quality. In this paper, we predict the air quality of the next 48 hours for each monitoring station and the daily average air quality of the next 7 days for a city, considering air quality data, meteorology data, and weather forecast data. Based on the domain knowledge about air pollution, we propose a deep neural network based approach, entitled DeepAir. Our approach consists of a deep distributed fusion network for station-level short-term prediction and a deep cascaded fusion network for the city-level long-term forecast. With the data transformation preprocessing, the former network adopts a neural distributed architecture to fuse heterogeneous urban data for simultaneously capturing the direct and indirect factors affecting air quality. The latter network takes a neural cascaded architecture to learn the dynamic influences from previously existing data and future predicted data on future air quality. We have deployed a real-time system on the cloud, providing fine-grained air quality forecasts for 300 $_+$ Chinese cities every hour. Our system mainly consists of three components: data crawler, task scheduler, and prediction model, which are implemented with a multi-task architecture to improve the system's efficiency and stability. Based on the datasets from three-year nine Chinese cities, experimental results demonstrate the advantages of our proposed method.

Accurate and updated road network data is vital in many urban applications, such as car-sharing, and logistics. The traditional approach to identifying the road network, ie, field survey, requires a significant amount of time and effort. With the wide usage of GPS embedded devices, a huge amount of trajectory data has been generated by different types of mobile objects, which provides a new opportunity to extract the underlying road network. However, the existing trajectory-based map recovery approaches require many empirical parameters and do not utilize the prior knowledge in existing maps, which over-simplifies or over-complicates the reconstructed road network. To this end, we propose a deep learning-based map generation framework, ie, DeepMG, which learns the structure of the existing road network to overcome the noisy GPS positions. More specifically, DeepMG extracts features from trajectories in both spatial view and transition view and uses a convolutional deep neural network T2RNet to infer road centerlines. After that, a trajectory-based post-processing algorithm is proposed to refine the topological connectivity of the recovered map. Extensive experiments on two real-world trajectory datasets confirm that DeepMG significantly outperforms the state-of-the-art methods. Ministry of Education (MOE) Nanyang Technological University Accepted version The research of Cheng Long was supported by the NTU Start-Up Grant and Singapore MOE Tier 1 Grant RG20/19 (S).

A path query aims to find trajectories passing a given sequence of connected road segments within a time period. It is very useful in many urban applications: 1) traffic modeling, 2) frequent path mining, 3) intersection coordination, and 4) traffic anomaly detection. Existing solutions for path query processing are implemented based on single machines, which are not efficient for the following tasks: 1) indexing large-scale historical data; 2) handling real-time trajectory updates; and 3) processing concurrent path queries from urban data mining applications. In this paper, we design and implement a cloud-based path query processing framework based on Microsoft Azure. We modify existing suffix tree structure to index trajectories using Azure Table. The proposed system consists of two main parts: 1) back-end processing , which performs pre-processing (i.e., parsing and map-matching) and index building tasks with a distributed computing platform (i.e., Storm) used to efficiently handle massive real-time trajectory updates; and 2) query processing , which answers path queries using Azure Storm to improve efficiency and overcome I/O bottleneck. Extensive experiments are performed based on the real-time taxi trajectories from Guiyang City, the capital of Guizhou Province, China to confirm the system efficiency. We also demonstrate a real deployed traffic analysis system based on our query processing framework.

The prevalence of positioning technologies has fostered massive trajectory data. Stay points from trajectories indicate the visiting of moving objects to locations, which provide an opportunity to understand the locations comprehensively. Many existing works rely on stay points to analyze locations. However, they are ad-hoc solutions to tackle specific problems, and it is time-consuming and tedious to develop each application. In this paper, we propose a universal StAy point-based LOcation aNalysis platform, i.e., SALON, with the characteristics of universality, efficiency and flexibility. It can retrieve stay points using flexible conditions, associate stay points with locations, extract comprehensive location profiles and visualize the analysis results to users. Based on the combination of these functions, we demonstrate three different location analysis scenarios, i.e., illegal location discovery, popular location ranking, location temporal analysis to show its characteristics.

With the rapid development of the Internet of Things (IoT), massive trajectories have been generated. Trajectory data is beneficial for many urban applications. This demo presents a holistic trajectory data management system based on distributed platforms, such as Spark and HBase, namely JUST-Traj. It provides a variety of indexes to efficiently support spatio-temporal queries and analyses on massive trajectories. Additionally, it provides a convenient SQL engine to execute all operations (storage, queries, analyses) through a SQL-like statement. Finally, we design a web portal for developers and demonstrate different operations in the portal.

Chemical materials are useful but sometimes hazardous, which requires strict regulation from the government. However, due to the potential economic benefits, many illegal hazardous chemical facilities are running underground, which poses a significant public safety threat. However, the traditional solutions, e.g., on-field screening and the anonymous tip-offs, involve a lot of human efforts. In this paper, we propose a ubiquitous approach called ICFinder to detecting illegal chemical facilities with chemical transportation trajectories. We first generate candidate locations by clustering stay points extracted from trajectories, and filter out known locations. Then, we rank those locations in suspicion order by modeling whether it has the loading/unloading events. ICFinder is evaluated over the real-world dataset from Nantong in China, and the deployed system identified 20 illegal chemical facilities in 3 months.

With the prevalence of positioning techniques, a prodigious number of spatio-temporal data is generated con-stantly. To effectively support sophisticated urban applications, e.g., location-based services, based on spatio-temporal data, it is desirable for an efficient, scalable, update-enabled, and easy-to-use spatio-temporal data management system.This paper presents JUST, i.e., JD Urban Spatio-Temporal data engine, which can efficiently manage big spatio-temporal data in a convenient way. JUST incorporates the distributed NoSQL data store, i.e., Apache HBase, as the underlying storage, GeoMesa as the spatio-temporal data indexing tool, and Apache Spark as the execution engine. We creatively design two indexing techniques, i.e., Z2T and XZ2T, which accelerates spatio-temporal queries tremendously. Furthermore, we introduce a compression mechanism, which not only greatly reduces the storage cost, but also improves the query efficiency. To make JUST easy-to-use, we design and implement a complete SQL engine, with which all operations can be performed through a SQL-like query language, i.e., JustQL. JUST also supports inherently new data insertions and historical data updates without index reconstruction. JUST is deployed as a PaaS in JD with multi-users support. Many applications have been developed based on the SDKs provided by JUST. Extensive experiments are carried out with six state-of-the-art distributed spatio-temporal data management systems based on two real datasets and one synthetic dataset. The results show that JUST has a competitive query performance and is much more scalable than them.

Trajectory data is very useful for many urban applications. However, due to its spatio-temporal and high-volume properties, it is challenging to manage trajectory data. Existing trajectory data management frameworks suffer from scalability problem, and only support limited trajectory queries. This paper proposes a holistic distributed NoSQL trajectory storage engine, TrajMesa, based on GeoMesa, an open-source indexing toolkit for spatio-temporal data. TrajMesa adopts a novel storage schema, which reduces the storage size tremendously. We also devise novel indexing key designs, and propose a bunch of pruning strategies. TrajMesa can support plentiful queries efficiently, including ID-Temporal query, spatial range query, similarity query, and k-NN query. Experimental results show the powerful query efficiency and scalability of TrajMesa.

With the development of positioning technology, a large number of trajectories have been generated, which are very useful for many urban applications. However, it is challenging to manage trajectory data for its spatio-temporal dynamics and high-volume properties. Existing trajectory data management frameworks suffer from efficiency or scalability problem, and only support limited trajectory query types. This paper takes the first attempt to build a holistic distributed NoSQL trajectory storage engine, named TrajMesa, based on GeoMesa, an open-source indexing toolkit for spatio-temporal data. TrajMesa can manage a prohibitively large number of trajectories, and support plenty of query types efficiently. Specifically, we first design a novel trajectory storage schema, which reduces the storage size tremendously. We then devise a novel indexing key schema for time ranges, based on which ID temporal query can be supported efficiently. To reduce the amount of retrieved trajectory data for a spatial range query, we innovatively propose a position code to indicate the spatial location of trajectories accurately. We also propose a bunch of pruning strategies for similarity query and k-NN query in the NoSQL environment. Extensive experiments are conducted using two real datasets and one synthetic dataset, verifying the powerful query efficiency and scalability of TrajMesa.

Illegal vehicle parking is a common urban problem faced by major cities in the world, as it incurs traffic jams, which lead to air pollution and traffic accidents. Traditional approaches to detect illegal vehicle parking events rely highly on active human efforts, e.g., police patrols or surveillance cameras. However, these approaches are extremely ineffective to cover a large city. The massive and high quality sharing bike trajectories from Mobike offer us with a unique opportunity to design a ubiquitous illegal parking detection system, as most of the illegal parking events happen at curbsides and have significant impact on the bike users. Two main components are employed to mine the trajectories in our system: 1)~trajectory pre-processing, which filters outlier GPS points, performs map-matching and builds indexes for bike trajectories; and 2)~illegal parking detection, which models the normal trajectories, extracts features from the evaluation trajectories and utilizes a distribution test-based method to discover the illegal parking events. The system is deployed on the cloud internally used by Mobike. Finally, extensive experiments and many insightful case studies based on the massive trajectories in Beijing are presented.

Trajectory data preprocessing is to convert raw GPS logs into organized trajectories, which is a common, necessary but tedious task in many urban applications. This paper proposes CloudTP, a cloud-based flexible trajectory data preprocessing framework, to provide an efficient online service, easing the burdens of urban application builders. The proposed system is designed and implemented based on the cloud storage and parallel computing framework (i.e. Spark). Its features consist of 1) noise filtering, 2) trajectory segmentation, 3) map matching, and 4) index building. CloudTP is useful for both normal users and advanced users. By simply uploading trajectory datasets and setting corresponding parameters, normal users can get organized trajectories, statistics and visualizations on the cloud, while advanced users can also customize their own algorithms in any preprocessing module. Finally, usage scenarios are demonstrated to show the capability and flexibility of CloudTP.

A path query aims to find the trajectories that pass a given sequence of connected road segments within a time period. It is very useful in many urban applications, e.g., 1) traffic modeling, 2) frequent path mining, and 3) traffic anomaly detection. Existing solutions for path query are implemented based on single machines, which are not efficient for the following tasks: 1) indexing large-scale historical data; 2) handling real-time trajectory updates; and 3) processing concurrent path queries. In this paper, we design and implement a cloud-based path query processing framework based on Microsoft Azure. We modify the suffix tree structure to index the trajectories using Azure Table. The proposed system consists of two main parts: 1) backend processing, which performs the pre-processing and suffix index building with distributed computing platform (i.e., Storm) used to efficiently handle massive real-time trajectory updates; and 2) query processing, which answers path queries using Azure Storm to improve efficiency and overcome the I/O bottleneck. We evaluate the performance of our proposed system based on a real taxi dataset from Guiyang, China.

With the rapid development of location-acquisition techniques, massive trajectories are continuously generated. Many urban applications rely heavily on the data mining/analysis results of massive trajectory data. This demo presents a holistic data management system for both historical and real-time trajectory records based on a cloud platform, such as Microsoft Azure. The proposed system is able to efficiently support a variety of trajectory queries, including ID-Temporal query, Spatio-Temporal query, and Path-Temporal query. With these queries, we demonstrate that different urban applications can be realized in a much easier way.

The Central Carbon Metabolic Flux Database (CeCaFDB, available at http://www.cecafdb.org) is a manually curated, multipurpose and open-access database for the documentation, visualization and comparative analysis of the quantitative flux results of central carbon metabolism among microbes and animal cells. It encompasses records for more than 500 flux distributions among 36 organisms and includes information regarding the genotype, culture medium, growth conditions and other specific information gathered from hundreds of journal articles. In addition to its comprehensive literature-derived data, the CeCaFDB supports a common text search function among the data and interactive visualization of the curated flux distributions with compartmentation information based on the Cytoscape Web API, which facilitates data interpretation. The CeCaFDB offers four modules to calculate a similarity score or to perform an alignment between the flux distributions. One of the modules was built using an inter programming algorithm for flux distribution alignment that was specifically designed for this study. Based on these modules, the CeCaFDB also supports an extensive flux distribution comparison function among the curated data. The CeCaFDB is strenuously designed to address the broad demands of biochemists, metabolic engineers, systems biologists and members of the -omics community.

Forecasting the flow of crowds is of great importance to traffic management and public safety, and very challenging as it is affected by many complex factors, including spatial dependencies (nearby and distant), temporal dependencies (closeness, period, trend), and external conditions (e.g., weather and events). We propose a deep-learning-based approach, called ST-ResNet, to collectively forecast two types of crowd flows (i.e. inflow and outflow) in each and every region of a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the residual neural network framework to model the temporal closeness, period, and trend properties of crowd traffic. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We have developed a real-time system based on Microsoft Azure Cloud, called UrbanFlow, providing the crowd flow monitoring and forecasting in Guiyang City of China. In addition, we present an extensive experimental evaluation using two types of crowd flows in Beijing and New York City (NYC), where ST-ResNet outperforms nine well-known baselines., Comment: 21 pages, 16 figures. arXiv admin note: substantial text overlap with arXiv:1610.00081

With advances in location-acquisition techniques, such as GPS- embedded phones, an enormous volume of trajectory data is generated, by people, vehicles, and animals. This trajectory data is one of the most important data sources in many urban computing applications, e.g., traffic modeling, user profiling analysis, air quality inference, and resource allocation.To utilize large scale trajectory data efficiently and effectively, cloud computing platforms, e.g., Microsoft Azure, are the most convenient and economic way. However, traditional cloud computing platforms are not designed to deal with spatio-temporal data, such as trajectories. To this end, we design and implement a holistic cloud-based trajectory data management system on Microsoft Azure to bridge the gap between trajectory data and urban applications. Our system can efficiently store, index, and query large trajectory data with three functions: 1) trajectory ID-temporal query, 2) trajectory spatio-temporal query, and 3) trajectory mapmatching. The efficiency of the system is tested and tuned based on real-time trajectory data feeds. The system is currently used in many internal urban applications, as we will illustrate using case studies.

Advances in location-acquisition and wireless communication technologies have led to wider availability of spatio-temporal (ST) data, which has unique spatial properties (i.e. geographical hierarchy and distance) and temporal properties (i.e. closeness, period and trend). In this paper, we propose a Deep-learning-based prediction model for Spatio-Temporal data (DeepST). We leverage ST domain knowledge to design the architecture of DeepST, which is comprised of two components: spatio-temporal and global. The spatio-temporal component employs the framework of convolutional neural networks to simultaneously model spatial near and distant dependencies, and temporal closeness, period and trend. The global component is used to capture global factors, such as day of the week, weekday or weekend. Using DeepST, we build a real-time crowd flow forecasting system called UrbanFlow1. Experiment results on diverse ST datasets verify DeepST's ability to capture ST data's spatio-temporal properties, showing the advantages of DeepST beyond four baseline methods.

The existing user’s trajectory prediction methods considered little about the interrelation among users and would fail if the user historical trajectory data were lack. This paper presents a user’s trajectory prediction model and corresponding algorithms by the historical trajectories of users based on the trajectory cluster. The experimental results on MDC dataset show that the proposed method has great improvement in efficiency, accuracy, and scalability comparing with the traditional methods, and it also be applied to the situation where user’s historical trajectory data are lacked.

The recommendation system can efficiently solve the information overload in mobile Internet. Thus, how to effectively utilize context information to improve the accuracy of recommendation becomes the research focus in the field. This article puts forward a novel approach to realize the context-aware recommendation in mobile environments. It first gets users’ interest resonance with a hash-based interest resonance mining algorithm. Then, it calculates the association degree between the user and the item and then predicts the user’s rating on the item. Finally, it comprehensively figures out the recommending index. Moreover, this article also designs a personal recommendation model for the users and provides relevant decision-making coefficients. Experiments have demonstrated that our approach is superior to the traditional ones (RMP, RSTE, MD and BBBs) in both performance and efficiency.

We extract some physical and chemical features related to the occurrence of single nucleotide polymorphism (SNP) from three groups of sliding windows around SNP site, and then make the predictions about accuracy by using radial basis function (RBF) networks. The result of the forward sliding windows suggests that the accuracies and Matthews correlation coefficient (MCC values) ascend with the increasing of length of sliding windows. The accuracies range from 73.27 % to 80.69 %, and MCC values range from 0.465 to 0.614. The backward sliding windows and the sliding windows with fixed length three are designed to find the crucial sites related to SNP. The results imply that the occurrence possibility of SNP relies heavily on the above physical and chemical features of sites which are at a distance around 20 bases from the SNP site. Compared with the support vector machine (SVM), our RBF network approach has achieved more satisfactory results.

In this paper, we forecast the reading of an air quality monitoring station over the next 48 hours, using a data-driven method that considers current meteorological data, weather forecasts, and air quality data of the station and that of other stations within a few hundred kilometers. Our predictive model is comprised of four major components: 1) a linear regression-based temporal predictor to model the local factors of air quality, 2) a neural network-based spatial predictor to model global factors, 3) a dynamic aggregator combining the predictions of the spatial and temporal predictors according to meteorological data, and 4) an inflection predictor to capture sudden changes in air quality. We evaluate our model with data from 43 cities in China, surpassing the results of multiple baseline methods. We have deployed a system with the Chinese Ministry of Environmental Protection, providing 48-hour fine-grained air quality forecasts for four major Chinese cities every hour. The forecast function is also enabled on Microsoft Bing Map and MS cloud platform Azure. Our technology is general and can be applied globally for other cities.