Search

The rapid development of Internet of Things (IoT) enables people to track objects by deploying multiple sensors, e.g., to track people in indoor spaces using RFID sensors. Multi-sensor object-tracking data are usually produced as records, which are thereby organized into small files and written to servers. However, the small-size property and high arriving-rate of multi-sensor object-tracking data will result in poor I/O performance of file systems such as HDFS. In this paper, we propose the first read/write-optimized solution for storing multi-sensor object-tracking data on HDFS. In particular, we exploit a distributed caching mechanism and a parallel file-merging policy to improve the I/O performance of HDFS. With our design, object-tracking data are first cached by a Distributed Memory File System (DMFS) on top of HDFS. These data are further merged into large files, which are then flushed to HDFS in parallel. We demonstrate that this mechanism is able to improve the write throughput of HDFS and outperforms existing centralized-cache-based approaches. In addition, in order to improve the search performance of object queries over multi-sensor object-tracking data, we propose a Sensor-Dependence Graph (SDG) to model sensor dependence and further present an SDG-based algorithm to efficiently cluster sensors. The object-tracking data from the sensors in the same cluster are merged into the same large files, which can reduce file scans during query processing and therefore improve search performance. We conduct extensive experiments to evaluate the performance of our proposal. The results suggest the efficiency of our proposal with respect to disk-write throughput, memory-write throughput, search performance, and sensor clustering. [ABSTRACT FROM AUTHOR]

Recently, with the widely use of flash memory based solid state drives (SSDs), a lot of studies have been conducted on SSD-based data management, such as index structures, query processing, and buffer management schemes. This paper focuses on buffer schemes for SSD-based database systems. However, differing from previous studies, we concentrate on buffer schemes for tree indexes on SSDs. This work is motivated by the observation that access patterns on index pages are much different from those on data pages. Generally, in a typical tree index, e.g., B+-tree, the root and internal nodes have higher read frequencies than leaf nodes have. However, traditional SSD-oriented buffering methods do not consider this special feature of indexes, and thus is not efficient when used as index buffer management schemes. In this paper, we present a new buffering scheme for tree indexes on SSDs that is named Clean-First and Dirty-Redundant-Write (CFDRW) scheme. The contributions of CFDRW are manifold. First, it assigns priorities to index pages to reflect the differences of access patterns of the nodes in a tree index. Second, it uses priority and recency to detect the hotness of index pages and proposes a new replacement algorithm based on priority and recency of the buffered index pages. Third, it exploits the internal parallelism of SSDs and proposes to write out buffer pages in a coarse granularity, i.e., to write out several pages using one physical I/O operation. We compare our proposal on two commodity SSDs with many previous methods including LRU, LIRS, and six flash-memory-based buffering schemes, by using synthetic and real workloads. The results show that our proposal outperforms the competitors under all workloads and SSDs in terms of various metrics including hit ratio, read count, write count, and elapsed time. [ABSTRACT FROM AUTHOR]

R-tree has been widely used in spatial data management and data analysis to improve the performance of spatial data retrieval. However, the original R-tree is designed for magnetic disks, and has poor performance on flash memory, due to the special features of flash memory such as asymmetric read/write speeds (fast read, slow write) and the erase-before-write feature. Particularly, the original updating mechanism of R-tree usually has to update a few interior nodes when inserting an indexing item into or deleting an item from a leaf node, yielding many slow writes to flash memory. With the wide use of flash memory in many location-based fields, e.g., to store moving trajectories in intelligent transportation systems, how to optimize R-tree for flash memory has become a critical issue. In this paper, we propose a novel spatial index named Flash-Optimized R-tree that is optimized for flash memory. In particular, we propose to defer the node-splitting operations on R-tree by introducing overflow nodes, which results in an unbalanced tree structure. With this mechanism, we can reduce random writes to flash memory and improve the overall performance of R-tree. In addition, we present a new buffering scheme to efficiently cache the updates to the tree, which can further reduce random writes to flash memory. We conduct extensive experiments on real flash-memory storage devices as well as a flash memory simulation platform to evaluate the performance of our proposal, and the results suggest the efficiency of our proposal with respect to different metrics. [ABSTRACT FROM AUTHOR]