Search

Data brokering systems aim to facilitate the exchange of data and models between disciplines in an increasingly transparent manner thereby accelerating scientific discovery. Researchers from many different, yet complimentary geoscience disciplines need to access cross field datasets from different fields with significantly different data formats, and in most cases differing time and space dimensionality than their own field commonly uses. This causes problems with large datasets with different time and space dimensions, as the difference in dimension often means that the entire dataset has to be read in order to provide the limited information the researcher is interested in. In this poster we present methods for removing the dimensionality from datasets, both physically on the data serving side as well demonstrate de- and redimensioning datasets from a broker based virtual perspective so the data brokering system can quickly access the smaller subset of data in the correct dimensionality for any given scientific field. What we did We de- and re-dimensioned the large reanalysis dataset, CFSR, to test alternative data paradigms to enhance the performance of single location extraction while maintain needed performace for spatial time step extraction. Results We were able to increase single location access performace by 10,000X, though spatial timestep access decreased by a factor of 10. Spatial requirements increased by a factor of 7 . Discussion Cross Science Data Issues Data brokering systems facilitate the exchange of data between disciplines, though the broker can not be responsible for optimizing the data structures for all sciences. An optimum data paradigm for one science is likely not efficient for other sciences, due to the native data dimensionalities of each field. Benchmark Quality While computational systems hosting each dataset are very dissimilar for this benchmark, it is fair to say that they are biased towards the CISL infrastructure. Hosting Space Requirements Other considerations that must be considered are the space requirements. Since the compression of shorter strings is less efficient than dense grids, and locational data has to be added to each point, the new paradigm for this study requires 4 times the storage space. Re-dimensioned Usage Over 22,000 single location and multi grid point requests have been made to TAMU and VTech Servers, estimated at ~ 3000 day equivalents from RDA subset service, which would not be possible with given computational resources.

Introduction Hydrologic modeling at daily or sub daily resolution requires accurate precipitation and temperature data, which is particularly important when planning land use changes and live management operations such as irrigation allocation. Weather gage data pose a variety of challenges, including the fact that they are nonexistent in many watersheds of interest. We show that by integrating global forecast products and reanalysis datasets from the atmospheric modeling community into the hydrologic models commonly used, we can circumvent some of the issues with point-of-measure weather gauges. Traditionally in hydrologic forecasting, a precipitation forecast is used from a nearby precipitation station on which the hydrologic model has been calibrated. Here we present the results of two studies. First, we compare calibrating a watershed model (SWAT) using weather forecasts from ensembles of raw gridded atmospheric models interpolated to the center of subbasins against using the closest weather gage measurements. Second, we perform a similar study using global reanalysis datasets. In addition, we look at what scales and radii using direct gridded model output may introduce equal or less error to watershed modeling projects than using the closest station. We demostrate: -There is validity to using reanalysis datasets in place of weather station data to force hydrologic modeling systems when nearby weather stations are not available or have inconsistent records -There is validity to using short term forecast archives in place of weather station data to force hydrologic modeling systems. -The distance to the where the model skill produces the same accuracy as the weather stations is closer than we thought. -Variables from the atmospheric models are physically linked, and we should likely keep them that way. Conclusion This study found that weather forecast archives and CFSR reanalysis data provide a potentially valuable alternative to “nearest weather station” data for hydrological modelers. Indeed, this study found that the CFSR data may actually be a better representation of the weather than point observations, at the scale of small watersheds (Presented Nov. 2011. Third International Forum on Water and Food (IFWF3). Tshwane, South Africa http://waterandfood.org/2012/04/22/third-international-forum-on-water-and-food/