Search

Flash-based SSDs have become well established in the storage market, replacing magnetic disks in both enterprise and consumer computer systems. The performance characteristics of these new devices have prompted a considerable amount of research that aims at developing efficient data access methods. Early works attempted to reduce the expensive random writes, exploiting logging and batch write techniques, while more recent ones addressed query processing, taking advantage of the high internal parallelism of SSDs. 3D XPoint is a new nonvolatile memory technology that has emerged recently, featuring smaller access times and higher durability compared with flash. It is available both as block addressable secondary storage and as byte addressable persistent main memory. However, the high cost of 3D XPoint prevents, for the moment, its adoption in large scales. This renders hybrid storage systems utilizing NAND flash and 3D XPoint a sufficient alternative. In this work, we propose HyR-tree, a hybrid variant of R-tree that persists a part of the tree in the high performing 3D XPoint storage. HyR-tree identifies repeated access pattern to the data and uses these patterns to locate the most important nodes. The importance of a node is determined by the performance gain that derives from its placement within a 3D XPoint-based device. We experimentally evaluated HyR-tree using real devices and four different datasets. The acquired results show that our proposal achieves significant performance gains up to 40% in tree construction and up to 56% in range queries. [ABSTRACT FROM AUTHOR]

We present the full public release of all data from the Illustris simulation project. Illustris is a suite of large volume, cosmological hydrodynamical simulations run with the moving-mesh code Arepo and including a comprehensive set of physical models critical for following the formation and evolution of galaxies across cosmic time. Each simulates a volume of (106.5 Mpc) 3 and self-consistently evolves five different types of resolution elements from a starting redshift of z = 127 to the present day, z = 0 . These components are: dark matter particles, gas cells, passive gas tracers, stars and stellar wind particles, and supermassive black holes. This data release includes the snapshots at all 136 available redshifts, halo and subhalo catalogs at each snapshot, and two distinct merger trees. Six primary realizations of the Illustris volume are released, including the flagship Illustris-1 run. These include three resolution levels with the fiducial “full” baryonic physics model, and a dark matter only analog for each. In addition, we provide four distinct, high time resolution, smaller volume “subboxes”. The total data volume is ∼ 265 TB, including ∼ 800 full volume snapshots and ∼ 30,000 subbox snapshots. We describe the released data products as well as tools we have developed for their analysis. All data may be directly downloaded in its native HDF5 format. Additionally, we release a comprehensive, web-based API which allows programmatic access to search and data processing tasks. In both cases we provide example scripts and a getting-started guide in several languages: currently, IDL, Python, and Matlab. This paper addresses scientific issues relevant for the interpretation of the simulations, serves as a pointer to published and on-line documentation of the project, describes planned future additional data releases, and discusses technical aspects of the release. [ABSTRACT FROM AUTHOR]

Abstract: In most forested environments, boreal forest in particular, monitoring and evaluation of management activities would be facilitated by storage of multiscale data sets spanning long time periods. For this study, such a data repository based on a generic spatio-temporal conceptual data model was created using XML. The data repository containing changes in data content for a 1000ha forest estate between 1871 and 1994 was used to evaluate the data retrieval capabilities of XQuery. The data retrieval operations consisted of pivoting, sequencing, and aggregation. Pivoting transforms the generic attribute storage used in the data model into a more compact column-wise format. Sequencing is used to return a temporal sequence of values, and for the traversal of the object hierarchy. Aggregation aggregates the values of a variable over a grouping variable. The combinations of the basic retrieval operations were also investigated as well as the effect of time and object hierarchy on the retrieval operations. The conceptual match between the generic spatio-temporal data model and the XML data model was apparent at the data retrieval operations. The key properties of XQuery for the implementation of the retrieval operations were extendability, support for nested structures, and data transformation functionality. [Copyright &y& Elsevier]

Efficiently retrieving a subset of time series data in a specific sky region from large-scale astronomical observation data is important for some time-domain studies in astronomy, especially for research on long-term variable sources. Considering that the field of view of a telescope is generally larger than the effective sky region, the retrieval method at the full image level is relatively inefficient in terms of data transmission and storage. Therefore, this paper proposes a search engine dedicated to the efficient retrieval of time-series subimages in observed image data archives (Astro-TS3, Astro nomical T ime- S eries S ubimage S earch Engine). An in situ and resilient indexing scheme is proposed, and it can build indexes for archived image data without any modification of the original files and support incremental updating with newly archived data. A subimage cutting tool is implemented to extract a part of the image according to the range specified by the user and generate a new Flexible Image Transport System (FITS) file for the extracted subimage. With Astro-TS3, users only need to submit requests including the ranges of the time and sky region to retrieve the required data and then download the time-series subimages after the preview. The evaluation of Astro-TS3 is performed using actual archived data from three Antarctic Survey Telescopes (AST3). The experimental data show that the time for the index establishment and incremental update increases approximately linearly relative to the number of data files. For retrieval, the response time of the data retrieval is approximately linearly related to the number and size of the subimages. [ABSTRACT FROM AUTHOR]