Search

Managing huge volumes of data is a problem now, and will only become worse with the advent of exascale computing and next generation observational systems. An important recognition is that data needs to be more easily migrated between storage tiers. Here we present a new solution, the Near-Line Data store (NLDS), for managing data migration between user facing storage systems and tape by using an object storage cache. NLDS builds on lessons learned from previous experience developing the ESIWACE funded Joint Data Migration App (JDMA) and deploying it at the Centre for Environmental Data Analysis (CEDA). CEDA currently has over 50PB of data stored on a range of disk based storage systems. These systems are chosen on cost, power usage and accessibility via a network, and include three different types of POSIX disk and object storage. Tens of PB of additional data are also stored on tape. Each of these systems has different workflows, interfaces and latencies, causing difficulties for users. NLDS, developed with ESIWACE2 and other funding, is a multi-tiered storage solution using object storage as a front end to a tape library. Users interact with NLDS via a HTTP API, with a Python library and command-line client provided to support both programmatic and interactive use. Files transferred to NLDS are first written to the object storage, and a backup is made to tape. When the object storage is approaching capacity, a set of policies is interrogated to determine which files will be removed from it. Upon retrieving a file, NLDS may have to first transfer the file from tape to the object storage, if it has been deleted by the policies. This implements a multi-tier of hot (disk), warm (object storage) and cold (tape) storage via a single interface. While systems like this are not novel, NLDS is open source, designed for ease of redeployment elsewhere, and for use from both local storage and remote sites. NLDS is based around a microservice architecture, with a message exchange brokering communication between the microservices, the HTTP API and the storage solutions. The system is deployed via Kubernetes, with each microservice in its own Docker container, allowing the number of services to be scaled up or down, depending on the current load of NLDS. This provides a scalable, power efficient system while ensuring that no messages between microservices are lost. OAuth is used to authenticate and authorise users via a pluggable authentication layer. The use of object storage as the front end to the tape allows both local and remote cloud-based services to access the data, via a URL, so long as the user has the required credentials. NLDS is a a scalable solution to storing very large data for many users, with a user-friendly front end that is easily accessed via cloud computing. This talk will detail the architecture and discuss how the design meets the identified use cases.

Background: Chronic rhinosinusitis with nasal polyps affects approximately 2–4% of the general population, and long-term use of systemic corticosteroids is associated with adverse effects. The aim of this study was to assess the efficacy and safety of mepolizumab in adults with recurrent, refractory severe bilateral chronic rhinosinusitis with nasal polyps. Methods: SYNAPSE was a randomised, double-blind, placebo-controlled, parallel-group, phase 3 trial done at 93 centres, mainly hospitals, in 11 countries. Eligible patients were aged 18 years or older with recurrent, refractory, severe, bilateral nasal polyp symptoms (nasal obstruction symptom visual analogue scale [VAS] score of >5), were eligible for repeat nasal surgery (overall symptoms VAS score >7 and endoscopic nasal polyps score of ≥5, with a minimum score of 2 in each nasal cavity) despite standard of care treatment, and had to have at least one nasal surgery in the past 10 years. Patients were randomly assigned (1:1), using permuted block design, to receive either 100 mg mepolizumab subcutaneously or placebo once every 4 weeks, in addition to standard of care (mometasone furoate intranasal spray for at least 8 weeks before screening and during the study, saline nasal irrigations, systemic corticosteroids or antibiotics, or both), as required, for 52 weeks. Site staff, the central study team, and patients were masked to study treatment and absolute blood eosinophil counts. The coprimary endpoints were change from baseline in total endoscopic nasal polyp score at week 52 and in mean nasal obstruction VAS score during weeks 49–52, assessed in the intention-to-treat population (ITT). This study is registered with ClinicalTrials.gov, NCT03085797. Findings: From May 25, 2017, to Dec 12, 2018, 854 patients were screened for eligibility. 414 patients were randomly assigned with 407 included in the ITT population; 206 received mepolizumab and 201 received placebo. Total endoscopic nasal polyp score significantly improved at week 52 from baseline with mepolizumab versus placebo (adjusted difference in medians −0·73, 95% CI −1·11 to −0·34; p

Significance In 2012, an unusual outbreak of malaria occurred in Djibouti City followed by increasingly severe annual outbreaks. Investigations revealed the presence of an Asian mosquito species; Anopheles stephensi, which thrives in urban environments. An. stephensi has since been identified in Ethiopia and Sudan. By combining data for An. stephensi across its full range (Asia, Arabian Peninsula, Horn of Africa) with spatial models that identify the species’ preferred habitat, we provide evidence-based maps predicting the possible African locations where An. stephensi could establish if allowed to spread. Our results suggest over 126 million people in cities across Africa could be at risk. This supports the WHO’s call for targeted An. stephensi control and prioritized surveillance., In 2012, an unusual outbreak of urban malaria was reported from Djibouti City in the Horn of Africa and increasingly severe outbreaks have been reported annually ever since. Subsequent investigations discovered the presence of an Asian mosquito species; Anopheles stephensi, a species known to thrive in urban environments. Since that first report, An. stephensi has been identified in Ethiopia and Sudan, and this worrying development has prompted the World Health Organization (WHO) to publish a vector alert calling for active mosquito surveillance in the region. Using an up-to-date database of published locational records for An. stephensi across its full range (Asia, Arabian Peninsula, Horn of Africa) and a set of spatial models that identify the environmental conditions that characterize a species’ preferred habitat, we provide evidence-based maps predicting the possible locations across Africa where An. stephensi could establish if allowed to spread unchecked. Unsurprisingly, due to this species’ close association with man-made habitats, our maps predict a high probability of presence within many urban cities across Africa where our estimates suggest that over 126 million people reside. Our results strongly support the WHO’s call for surveillance and targeted vector control and provide a basis for the prioritization of surveillance.

The impacts of sea surface temperature (SST) anomalies and anthropogenic greenhouse gases on the likelihood of extreme drought occurring in the central United States in the year 2012 were investigated using large-ensemble simulations from a global atmospheric climate model. Two sets of experiments were conducted. In the first, the simulated hydroclimate of 2012 was compared to a baseline period (1986–2014) to investigate the impact of SSTs. In the second, the hydroclimate in a world with 2012-level anthropogenic forcing was compared to five “counterfactual” versions of a 2012 world under preindustrial forcing. SST anomalies in 2012 increased the simulated likelihood of an extreme summer precipitation deficit (e.g., the deficit with a 2% exceedance probability) by a factor of 5. The likelihood of an extreme summer soil moisture deficit increased by a similar amount, due in great part to a large spring soil moisture deficit carrying over into summer. An anthropogenic impact on precipitation was detectable in the simulations, doubling the likelihood of what would have been a rainfall deficit with a 2% exceedance probability under preindustrial-level forcings. Despite this reduction in rainfall, summer soil moisture during extreme drought was essentially unaffected by anthropogenic forcing because of 1) evapotranspiration declining roughly one-to-one with a decrease in precipitation due to severe water supply constraint and despite higher evaporative demand and 2) a decrease in stomatal conductance, and therefore a decrease in potential transpiration, with higher atmospheric CO2 concentrations.

Using object stores, and cloud based storage utilising the Amazon S3 HTTP API, presents a number of opportunities and challenges in the storing of climate data. The distributed nature of the objects allows large data sets to be broken into “fragments”, each fragment containing a subset of the data. This allows for parallel access to the fragments, improving the performance of reading the data across a network. However, this presents a number of problems. Firstly, determining the fragment size and the optimum method of splitting multi-dimensional data. Secondly, enabling meaningful search of data, when the data may be widely distributed.This poster will present a new method of splitting netCDF files into fragments and storing each fragment as an object in an S3 compatible object store. The location of the fragments, the metadata and dimensions for each climate variable are stored in a master file, which can be written to a location not within the object store, for example a POSIX file system on a SSD. This allows fast search of the metadata without having to reassemble the fragments. Each fragment is stored as a self-contained netCDF file allowing reconstruction of the data if the master file is lost.

Atmospheric modes of variability relevant for extreme temperature and precipitation events are evaluated in models currently being used for extreme event attribution. A 100 member initial condition ensemble of the global circulation model HadAM3P is compared with both the multi-model ensemble from the Coupled Model Inter-comparison Project, Phase 5 (CMIP5) and the CMIP5 atmosphere-only counterparts (AMIP5). The use of HadAM3P allows for huge ensembles to be computed relatively fast, thereby providing unique insights into the dynamics of extremes. The analysis focuses on mid Northern Latitudes (primarily Europe) during winter, and is compared with ERA-Interim reanalysis. The tri-modal Atlantic eddy-driven jet distribution is remarkably well captured in HadAM3P, but not so in the CMIP5 or AMIP5 multi-model mean, although individual models fare better. The well known underestimation of blocking in the Atlantic region is apparent in CMIP5 and AMIP5, and also, to a lesser extent, in HadAM3P. Pacific blocking features are well produced in all modeling initiatives. Blocking duration is biased towards models reproducing too many short-lived events in all three modelling systems. Associated storm tracks are too zonal over the Atlantic in the CMIP5 and AMIP5 ensembles, but better simulated in HadAM3P with the exception of being too weak over Western Europe. In all cases, the CMIP5 and AMIP5 performances were almost identical, suggesting that the biases in atmospheric modes considered here are not strongly coupled to SSTs, and perhaps other model characteristics such as resolution are more important. For event attribution studies, it is recommended that rather than taking statistics over the entire CMIP5 or AMIP5 available models, only models capable of producing the relevant dynamical phenomena be employed. ISSN:0930-7575 ISSN:1432-0894

Demonstrating the effect that climate change is having on regional weather is a subject which occupies climate scientists, government policy makers and the media. After an extreme weather event occurs, the question is often posed, ‘Was the event caused by anthropogenic climate change?’ Recently, a new branch of climate science (known as attribution) has sought to quantify how much the risk of extreme events occurring has increased or decreased due to climate change. One method of attribution uses very large ensembles of climate models computed via volunteer distributed computing. A recent advancement is the ability to run both a global climate model and a higher resolution regional climate model on a volunteer's home computer. Such a set-up allows the simulation of weather on a scale that is of most use to studies of the attribution of extreme events. This article introduces a global climate model that has been developed to simulate the climatology of all major land regions with reasonable accuracy. This then provides the boundary conditions to a regional climate model (which uses the same formulation but at higher resolution) to ensure that it can produce realistic climate and weather over any region of choice. The development process is documented and a comparison to previous coupled climate models and atmosphere-only climate models is made. The system (known as weather@home) by which the global model is coupled to a regional climate model and run on volunteers' home computers is then detailed. Finally, a validation of the whole system is performed, with a particular emphasis on how accurately the distributions of daily mean temperature and daily mean precipitation are modelled in a particular application over Europe. This builds confidence in the applicability of the weather@home system for event attribution studies.

The crucial question in the public debate of extreme events is increasingly whether and to what extent the event has been caused by anthropogenic warming. In this study we investigate this question using extreme summer precipitation events in England and Wales as an example for probabilistic event attribution using very large ensembles of regional climate model (RCM) simulations within the weather@home.net project. This allows us to analyse the statistics of high precipitation events in England and Wales, a region with a high quality precipitation observational dataset. Validating the model simulations against observations shows a credible shape of the distribution of 5-day precipitation, and thus confidence in the results. While the risk of extreme July precipitation events has at least doubled due to anthropogenic climate change in the modelling framework, no significant changes can be detected for the other two summer months. This study thus highlights the challenges of probabilistic event attribution of complex weather events and identifies the need to further decompose atmospheric features responsible for an event to occur for quantitative attribution analysis.

A number of studies have set out to obtain a range of atmosphere and ocean model behavior by perturbing parameters in a single climate model (perturbed physics ensemble: PPE). Early studies used shallow layer slab ocean or flux-adjusted coupled ocean-atmosphere models to obtain a broad range of behavior as characterized by climate sensitivity. A recent study reports a relatively narrow range of sensitivities (2.2–3.2°C) in a PPE of 35 coupled models without flux adjustment, raising the question whether previous broad ranges were an artifact of the use of models that were not in top-of-atmosphere (TOA) energy balance. Moreover, no PPE experiment has reported a large spread of behavior of the ocean compared to that exhibited in a multi-model ensemble (MME) such as Coupled Model Intercomparison Project phase 3 (CMIP3). In this work, we randomly perturb model parameters of a coupled ocean-atmosphere general circulation model using a space-filling design containing 10,000 combinations. The ensemble is run over the distributed computing platform of climateprediction.net under fixed pre-industrial forcing without flux adjustment. We resample a second, 20,000-member, ensemble with perturbations conditioned on the TOA fluxes from the first ensemble to not drift significantly from a realistic base state while targeting a range of behavior. Models within the targeted ensemble show realistic regional control climates when compared to the CMIP3 ensemble, although there is a bias in global mean surface temperature. The range of predicted equilibrium climate sensitivities of the targeted ensemble is substantially smaller than that obtained with flux adjustment, but larger than the range in the CMIP3 ensemble or in the 35-model un-flux-adjusted PPE in a recent study mentioned above. The Atlantic meridional overturning circulation in the targeted ensemble exhibits a spread in strength as wide as that found in the CMIP3 ensemble. We conclude that flux adjustment is not a pre-requisite for obtaining a broad spread of behavior in a perturbed physics ensemble.

The Intergovernmental Panel on Climate Change (IPCC) has accepted the invitation from the UNFCCC to provide a special report on the impacts of global warming of 1.5 °C above pre-industrial levels and related global greenhouse gas emission pathways. Many current experiments in, for example, the Coupled Model Inter-comparison Project (CMIP), are not specifically designed for informing this report. Here, we document the design of the Half a degree Additional warming, Projections, Prognosis and Impacts (HAPPI) experiment. HAPPI provides a framework for the generation of climate data describing how the climate, and in particular extreme weather, might differ from the present day in worlds that are 1.5 °C and 2.0 °C warmer than pre-industrial conditions. Output from participating climate models includes variables frequently used by a range of integrated assessment models. The key challenge is to separate the impact of an additional approximately half degree of warming from uncertainty in climate model responses and internal climate variability that dominate CMIP-style experiments. Large ensembles of simulations (> 50 members) of atmosphere-only models for three time slice experiments are proposed, each a decade in length; the first being the most recent observed 10-year period (2006–2015), the second two being estimates of the a similar decade but under 1.5 and 2 °C conditions a century in the future. We use the Representative Concentration Pathways 2.6 (RCP2.6) to provide the model boundary conditions for the 1.5 °C scenario, and a weighted combination of RCP2.6 and RCP4.5 for the 2 °C scenario.

During the winter of 2013/14, much of the UK experienced repeated intense rainfall events and flooding. This had a considerable impact on property and transport infrastructure. A key question is whether the burning of fossil fuels is changing the frequency of extremes, and if so to what extent. We assess the scale of the winter flooding before reviewing a broad range of Earth system drivers affecting UK rainfall. Some drivers can be potentially disregarded for these specific storms whereas others are likely to have increased their risk of occurrence. We discuss the requirements of hydrological models to transform rainfall into river flows and flooding. To determine any general changing flood risk, we argue that accurate modelling needs to capture evolving understanding of UK rainfall interactions with a broad set of factors. This includes changes to multiscale atmospheric, oceanic, solar and sea-ice features, and land-use and demographics. Ensembles of such model simulations may be needed to build probability distributions of extremes for both pre-industrial and contemporary concentration levels of atmospheric greenhouse gases.© 2014 Macmillan Publishers Limited. All rights reserved.

Changes in near surface air temperature (ΔT) in response to anthropogenic greenhouse gas forcing are expected to show spatial heterogeneity because energy and moisture fluxes are modulated by features of the landscape that are also heterogeneous at these spatial scales. Detecting statistically meaningful heterogeneity requires a combination of high spatial resolution and a large number of simulations. To investigate spatial variability of projected ΔT, we generated regional, high-resolution (25-km horizontal), large ensemble (100 members per year), climate simulations of western United States (US) for the periods 1985 – 2014 and 2030 – 2059, the latter with atmospheric constituent concentrations from the Representative Concentration Pathway 4.5. Using the large ensemble, 95% confidence interval sizes for grid-cell-scale temperature responses were on the order of 0.1 °C, compared to 1 °C from a single ensemble member only. In both winter and spring, the snow-albedo feedback statistically explains roughly half of the spatial variability in 'T. Simulated decreases in albedo exceed 0.1 in places, with rates of change in T per 0.1 decrease in albedo ranging from 0.3 to 1.4 °C. In summer, ΔT pattern in the northwest US is correlated with the pattern of decreasing precipitation. In all seasons, changing lapse rates in the low-to-middle troposphere may account for up to 0.2 °C differences in warming across the western US. Near the coast, a major control of spatial variation is the differential warming between sea and land.

A new climate modelling project has been developed for regional climate simulation and the attribution of weather and climate extremes over Australia and New Zealand. The project, known as weather@home Australia-New Zealand, uses public volunteers' home computers to run a moderate-resolution global atmospheric model with a nested regional model over the Australasian region. By harnessing the aggregated computing power of home computers, weather@home is able to generate an unprecedented number of simulations of possible weather under various climate scenarios. This combination of large ensemble sizes with high spatial resolution allows extreme events to be examined with more robust estimates of uncertainty. This paper provides an overview of the weather@home Australia-New Zealand project, including initial evaluation of the regional model performance. The model is seen to be capable of resolving many climate features that are important for the Australian and New Zealand regions, including the influence of El Niño-Southern Oscillation on driving natural climate variability. To date, 75 model simulations of the observed climate have been successfully integrated over the period 1985–2014 in a time-slice manner. In addition, multi-thousand member ensembles have also been generated for the years 2013, 2014 and 2015 under climate scenarios with and without the effect of human influences. All data generated by the project is freely available to the broader research community.

Incomplete understanding of three aspects of the climate system—equilibrium climate sensitivity, rate of ocean heat uptake and historical aerosol forcing—and the physical processes underlying them lead to uncertainties in our assessment of the global-mean temperature evolution in the twenty-first century 1,2 . Explorations of these uncertainties have so far relied on scaling approaches 3,4 , large ensembles of simplified climate models 1,2 , or small ensembles of complex coupled atmosphere‐ocean general circulation models 5,6 which under-represent uncertainties in key climate system properties derived from independent sources 7‐9 . Here we present results from a multi-thousand-member perturbed-physics ensemble of transient coupled atmosphere‐ocean general circulation model simulations. We find that model versions that reproduce observed surface temperature changes over the past 50 years show global-mean temperature increases of 1.4‐3 K by 2050, relative to 1961‐1990, under a mid-range forcing scenario. This range of warming is broadly consistent with the expert assessment provided by the Intergovernmental Panel on Climate Change Fourth Assessment Report 10 , but extends towards larger warming than observed in ensemblesof-opportunity 5 typically used for climate impact assessments. From our simulations, we conclude that warming by the middle of the twenty-first century that is stronger than earlier estimates is consistent with recent observed temperature changes and a mid-range ‘no mitigation’ scenario for greenhouse-gas emissions. In the latest generation of coupled atmosphereocean general circulation models (AOGCMs) contributing to the Coupled Model Intercomparison Project phase 3 (CMIP-3), uncertainties in key propertiescontrollingthetwenty-firstcenturyresponsetosustained anthropogenic greenhouse-gas forcing were not fully sampled, partially owing to a correlation between climate sensitivity and aerosol forcing 7,8 , a tendency to overestimate ocean heat uptake 11,12 and compensation between short-wave and long-wave feedbacks 9 . This complicates the interpretation of the ensemble spread as a direct uncertainty estimate, a point reflected in the fact that the ‘likely’ (>66% probability) uncertainty range on the transient response was explicitly subjectively assessed as40% to C60% of

A succession of storms reaching Southern England in the winter of 2013/2014 caused severe floods and £451 million insured losses. In a large ensemble of climate model simulations, we find that, as well as increasing the amount of moisture the atmosphere can hold, anthropogenic warming caused a small but significant increase in the number of January days with westerly flow, both of which increased extreme precipitation. Hydrological modelling indicates this increased extreme 30-day-average Thames river flows, and slightly increased daily peak flows, consistent with the understanding of the catchment’s sensitivity to longer-duration precipitation and changes in the role of snowmelt. Consequently, flood risk mapping shows a small increase in properties in the Thames catchment potentially at risk of riverine flooding, with a substantial range of uncertainty, demonstrating the importance of explicit modelling of impacts and relatively subtle changes in weather-related risks when quantifying present-day effects of human influence on climate.

climateprediction.net is a large public resource distributed scientific computing project. Members of the public download and run a full-scale climate model, donate their computing time to a large perturbed physics ensemble experiment to forecast the climate in the 21st century and submit their results back to the project. The amount of data generated is large, consisting of tens of thousands of individual runs each in the order of tens of megabytes. The overall dataset is, therefore, in the order of terabytes. Access and analysis of the data is further complicated by the reliance on donated, distributed, federated data servers. This paper will discuss the problems encountered when the data required for even a simple analysis is spread across several servers and how webservice technology can be used; how different user interfaces with varying levels of complexity and flexibility can be presented to the application scientists, how using existing web technologies such as HTTP, SOAP, XML, HTML and CGI can engender the reuse of code across interfaces; and how application scientists can be notified of their analysis' progress and results in an asynchronous architecture.

[1] In the summer 2010 Western Russia was hit by an extraordinary heat wave, with the region experiencing by far the warmest July since records began. Whether and to what extent this event is attributable to anthropogenic climate change is controversial. Dole et al. (2011) report the 2010 Russian heat wave was “mainly natural in origin” whereas Rahmstorf and Coumou (2011) write that with a probability of 80% “the 2010 July heat record would not have occurred” without the large-scale climate warming since 1980, most of which has been attributed to the anthropogenic increase in greenhouse gas concentrations. The latter explicitly state that their results “contradict those of Dole et al. (2011).” Here we use the results from a large ensemble simulation experiment with an atmospheric general circulation model to show that there is no substantive contradiction between these two papers, in that the same event can be both mostly internally-generated in terms of magnitude and mostly externally-driven in terms of occurrence-probability. The difference in conclusion between these two papers illustrates the importance of specifying precisely what question is being asked in addressing the issue of attribution of individual weather events to external drivers of climate.

In complex spatial models, as used to predict the climate response to greenhouse gas emissions, parameter variation within plausible bounds has major effects on model behavior of interest. Here, we present an unprecedentedly large ensemble of >57,000 climate model runs in which 10 parameters, initial conditions, hardware, and software used to run the model all have been varied. We relate information about the model runs to large-scale model behavior (equilibrium sensitivity of global mean temperature to a doubling of carbon dioxide). We demonstrate that effects of parameter, hardware, and software variation are detectable, complex, and interacting. However, we find most of the effects of parameter variation are caused by a small subset of parameters. Notably, the entrainment coefficient in clouds is associated with 30% of the variation seen in climate sensitivity, although both low and high values can give high climate sensitivity. We demonstrate that the effect of hardware and software is small relative to the effect of parameter variation and, over the wide range of systems tested, may be treated as equivalent to that caused by changes in initial conditions. We discuss the significance of these results in relation to the design and interpretation of climate modeling experiments and large-scale modeling more generally.

Using object stores, and cloud based storage utilising the Amazon S3 HTTP API, presents a number of opportunities and challenges in the storing of climate data. The distributed nature of the objects allows large data sets to be broken into “fragments”, each fragment containing a subset of the data. This allows for parallel access to the fragments, improving the performance of reading the data across a network. However, this presents a number of problems. Firstly, determining the fragment size and the optimum method of splitting multi-dimensional data. Secondly, enabling meaningful search of data, when the data may be widely distributed.This poster will present a new method of splitting netCDF files into fragments and storing each fragment as an object in an S3 compatible object store. The location of the fragments, the metadata and dimensions for each climate variable are stored in a master file, which can be written to a location not within the object store, for example a POSIX file system on a SSD. This allows fast search of the metadata without having to reassemble the fragments. Each fragment is stored as a self-contained netCDF file allowing reconstruction of the data if the master file is lost.

Geoscientific Model Development, 10 (5), ISSN:1991-9603, ISSN:1991-959X