Search

It is often assumed that weather regimes adequately characterize atmospheric circulation variability. However, regime classifications spanning many months and with a low number of regimes may not satisfy this assumption. The first aim of this study is to test such hypothesis for the Euro-Atlantic region. The second one is to extend the assessment of sub-seasonal forecast skill in predicting the frequencies of occurrence of the regimes beyond the winter season. Two regime classifications of four regimes each were obtained from sea level pressure anomalies clustered from October to March and from April to September respectively. Their spatial patterns were compared with those representing the annual cycle. Results highlight that the two regime classifications are able to reproduce most part of the patterns of the annual cycle, except during the transition weeks between the two periods, when patterns of the annual cycle resembling Atlantic Low regime are not also observed in any of the two classifications. Forecast skill of Atlantic Low was found to be similar to that of NAO+, the regime replacing Atlantic Low in the two classifications. Thus, although clustering yearly circulation data in two periods of 6 months each introduces a few deviations from the annual cycle of the regime patterns, it does not negatively affect sub-seasonal forecast skill. Beyond the winter season and the first ten forecast days, sub-seasonal forecasts of ECMWF are still able to achieve weekly frequency correlations of r = 0.5 for some regimes and start dates, including summer ones. ECMWF forecasts beat climatological forecasts in case of long-lasting regime events, and when measured by the fair continuous ranked probability skill score, but not when measured by the Brier skill score. Thus, more efforts have to be done yet in order to achieve minimum skill necessary to develop forecast products based on weather regimes outside winter season. The research leading to these results was funded by the Sub-seasonal to Seasonal climate forecasting for Energy (S2S4E) Project (GA776787), by the New European Wind Atlas (NEWA-II) project (PCIN-2016-029) and by the Ministerio de Economía y Competitividad (MINECO) as part of the CLINSA project (CGL2017-85791-R) and of the Juan de la Cierva-Incorporación Grant (IJCI-2016-29776). We also acknowledge the developers of the s2dverification software package (Manubens et al 2018), used for the data analysis and the visualization of the results presented in this work. The authors would also like to express their gratitude to Pierre-Antoine Bretonniére, Margarida Samsó and Núria Pérez-Zanón for their fundamental contribute in downloading and pre-processing data, and to Dr. Stefano Materia, Dr. Markus Donat, Dr. Simon Wild for their insightful comments and suggestions.

Skillful and reliable predictions of week-to-week rainfall variations in South America, two to three weeks ahead, are essential to protect lives, livelihoods, and ecosystems. We evaluate forecast performance for weekly rainfall in extended austral summer (November–March) in four contemporary subseasonal systems, including a new Brazilian model, at 1–5-week leads for 1999–2010. We measure performance by the correlation coefficient (in time) between predicted and observed rainfall; we measure skill by the Brier skill score for rainfall terciles against a climatological reference forecast. We assess unconditional performance (i.e., regardless of initial condition) and conditional performance based on the initial phase of the Madden–Julian oscillation (MJO) and El Niño–Southern Oscillation (ENSO). All models display substantial mean rainfall biases, including dry biases in Amazonia and wet biases near the Andes, which are established by week 1 and vary little thereafter. Unconditional performance extends to week 2 in all regions except for Amazonia and the Andes, but to week 3 only over northern, northeastern, and southeastern South America. Skill for upper- and lower-tercile rainfall extends only to week 1. Conditional performance is not systematically or significantly higher than unconditional performance; ENSO and MJO events provide limited “windows of opportunity” for improved S2S predictions that are region and model dependent. Conditional performance may be degraded by errors in predicted ENSO and MJO teleconnections to regional rainfall, even at short lead times.

Rainfall variability affects agriculture planning and water resource management. In extreme flood and drought events, lives and property are destroyed. This study aims to improve East Africa’s seasonal rainfall prediction by determining the impact of the standard eight Real-time Multivariate Madden–Julian Oscillation (MJO) (RMM) phases on rainfall and using sea surface temperature (SST) response to test the predictability of the March–May (MAM) and October–December (OND) main rainfall seasons over a period of 33 years (1981–2013). Pearson correlation patterns, composite maps, and regression analyses were applied, and the Brier skill score (BSS) and correlation coefficients (CC) were utilized as validation metrics. Low correspondence of rainfall to MJO 1 and MJO 2 was observed except for the months of November and December. Seasonally, MAM and OND correlation patterns with MJO 2 revealed enhanced rainfall over the highlands and insignificant correspondence over coastal areas. Conversely, enhanced MJO 8 corresponded to suppressed rainfall during the June–August season over the coast and the eastern highlands. MAM rainfall was shown to be predictable using Maritime Continent SST indices, with a BSS of 0.41, while OND rainfall was shown to be predictable using Atlantic and Maritime Continent SSTs with a BSS of 0.62. Positive and negative MJO 2 corresponded, respectively, to enhanced and suppressed rainfall during the OND season and was confirmed to be related to, respectively, a positive and negative Indian Ocean dipole (IOD). An IOD year could possibly be triggered by changes in MJO 2 amplitudes observed as early peaks between February and June.

Background: Several mortality prediction models (MPM) are used for predicting early (30-day) mortality following transcatheter aortic valve implantation (TAVI). Little is known about their predictive performance in external TAVI populations. We aim to externally validate established MPMs on a large TAVI dataset from the Netherlands Heart Registration (NHR). Methods: We included data from NHR-patients who underwent TAVI during 2013–2017. We calculated the predicted mortalities per MPM. We assessed the predictive performance by discrimination (Area Under Receiver Operating-characteristic Curve, AU-ROC); the Area Under Precision-Recall Curve, AU-PRC; calibration (using calibration-intercept and calibration-slope); Brier Score and Brier Skill Score. We also assessed the predictive performance among subgroups: tertiles of mortality-risk for non-survivors, gender, and access-route. Results: We included 6177 TAVI-patients with an observed early-mortality rate of 4.5% (n = 280). We applied seven MPMs (STS, EuroSCORE-I, EuroSCORE-II, ACC-TAVI, FRANCE-2, OBSERVANT, and German-AV) on our cohort. The highest AU-ROCs were 0.64 (95%CI 0.61–0.67) for ACC-TAVI and 0.63 (95%CI 0.60–0.67) for FRANCE-2. All MPMs had a very low AU-PRC of ≤0.09. ACC-TAVI had the best calibration-intercept and calibration-slope. Brier Score values ranged between 0.043 and 0.063. Brier Skill Score ranged between −0.47 and 0.004. ACC-TAVI and FRANCE-2 predicted high mortality-risk better than other MPMs. ACC-TAVI outperformed other MPMs in different subgroups. Conclusion: The ACC-TAVI model has relatively the best predictive performance. However, all models have poor predictive performance. Because of the poor discrimination, miscalibration and limited accuracy of the models there is a need to update the existing models or develop new TAVI-specific models for local populations.

To develop and evaluate a high-dimensional, data-driven model to identify patients at high risk of clinical deterioration from routinely collected electronic health record (EHR) data.In this single-center, retrospective cohort study, 488 patients with single-ventricle and shunt-dependent congenital heart disease6 months old were admitted to the cardiac intensive care unit before stage 2 palliation between 2014 and 2019. Using machine-learning techniques, we developed the Intensive care Warning Index (I-WIN), which systematically assessed 1028 regularly collected EHR variables (vital signs, medications, laboratory tests, and diagnoses) to identify patients in the cardiac intensive care unit at elevated risk of clinical deterioration. An ensemble of 5 extreme gradient boosting models was developed and validated on 203 cases (130 emergent endotracheal intubations, 34 cardiac arrests requiring cardiopulmonary resuscitation, 10 extracorporeal membrane oxygenation cannulations, and 29 cardiac arrests requiring cardiopulmonary resuscitation onto extracorporeal membrane oxygenation) and 378 control periods from 446 patients.At 4 hours before deterioration, the model achieved an area under the receiver operating characteristic curve of 0.92 (95% confidence interval, 0.84-0.98), 0.881 sensitivity, 0.776 positive predictive value, 0.862 specificity, and 0.571 Brier skill score. Performance remained high at 8 hours before deterioration with 0.815 (0.688-0.921) area under the receiver operating characteristic curve.I-WIN accurately predicted deterioration events in critically-ill infants with high-risk congenital heart disease up to 8 hours before deterioration, potentially allowing clinicians to target interventions. We propose a paradigm shift from conventional expert consensus-based selection of risk factors to a data-driven, machine-learning methodology for risk prediction. With the increased availability of data capture in EHRs, I-WIN can be extended to broader applications in data-rich environments in critical care.

Wind power ramps are the abrupt yet significant change in wind power productions. The information on the ordinal levels of impending wind power ramp could help power system operator to arm operation or ramping reserves in a timely manner. This paper presents novel approaches for regional wind power ramp level forecasting using real-time meso-scale wind speed measurements. Motivated by the correlation of the meso-scale wind speed measurements with the regional wind power data, the proposed approach utilizes multinomial logistic regression for wind power ramp forecasting. An approach that combines the probabilistic output of individual regressive models in a weighted manner is proposed, with the weights calculated by minimizing the Brier skill score of the combined model. The proposed methods are tested by using real-world data, and is compared with benchmark methods. The results reveal the effectiveness of the proposed approaches.

The purpose of this study is the selection of an appropriate coupled general circulation model (CGCM) for seasonal weather prediction of winter temperatures in two cities of Pakistan: Islamabad and Lahore. The data used in the study comprises 32 years (1969–2000) of real-time observations from the Pakistan Meteorological Department (PMD) and simulation data from CGCMs, including the European Centre for Medium Range Weather Forecasts (ECMWF) and the UK Met Office (UKMO). This study attempts to establish accurate average seasonal weather predictions using a single-model Ensemble Prediction System (EPS). Accurate seasonal prediction is important for developing an early warning system. The probabilistic forecast is calculated for the lower tercile (i.e. [0, 0.33]) keeping in view the trend of winter temperature for both cities. The initial conditions (ICs) obtained from the CGCMs have been alternatively processed for better results. The probabilistic forecast obtained from the EPS has been further verified against PMD observations and European Reanalysis Data (ERA-40) using two different measures of probabilistic skill: the Brier Skill Score (BSS) and the Relative Operating Characteristic Skill Score (ROCSS). The ECMWF and UKMO models were used for both of the cities under study. The results demonstrate that the ECMWF model is more suitable for Islamabad, whereas the UKMO model is more suitable for Lahore. The results from these models might be generalised for other areas at the same latitudes with similar topographic and climatic conditions.

Probabilistic forecasts of U.S. Drought Monitor (USDM) intensification over 2-, 4-, and 8-week time periods are developed based on recent anomalies in precipitation, evapotranspiration, and soil moisture. These statistical forecasts are computed using logistic regression with cross validation. While recent precipitation, evapotranspiration, and soil moisture do provide skillful forecasts, it is found that additional information on the current state of the USDM adds significant skill to the forecasts. The USDM state information takes the form of a metric that quantifies the “distance” from the next-higher drought category using a nondiscrete estimate of the current USDM state. This adds skill because USDM states that are close to the next-higher drought category are more likely to intensify than states that are farther from this threshold. The method shows skill over most of the United States but is most skillful over the north-central United States, where the cross-validated Brier skill score averages 0.20 for both 2- and 4-week forecasts. The 8-week forecasts are less skillful in most locations. The 2- and 4-week probabilities have very good reliability. The 8-week probabilities, on the other hand, are noticeably overconfident. For individual drought events, the method shows the most skill when forecasting high-amplitude flash droughts and when large regions of the United States are experiencing intensifying drought.

This thesis investigates the behaviour of the often used point-wise skill score, the MSESSini a.k.a. BSS, and develops new error metrics that, as opposed to point-wise metrics, take the spatial structure of morphological patterns into account. The MSESSini measures the relative accuracy of a morphological prediction over a prediction of zero morphological change, using the mean-squared error (MSE) as the accuracy measure. The main findings about the MSESSini are: 1) a generic ranking, based on values for MSESSini, has limited validity, since the zero change reference model fails to make model performance comparable across different prediction situations; 2) the combination of larger, persistent and smaller, intermittent scales of cumulative change may lead to an increase of skill with time, without the prediction on either of these scales becoming more skilful with time; 3) in the presence of inevitable location errors, the MSESSini favours predictions that underestimate the variance of cumulative bed changes and 4) existing methods to correct for measurement error are inconsistent in either their skill formulation or their suggested classification scheme. In order to overcome the inherent limitations of point-wise metrics, three novel diagnostic tools for the spatial validation of 2D morphological predictions are developed. First, a field deformation or warping method deforms the predictions towards the observations, minimizing the squared point-wise error. Error measures are formulated based on both the smooth displacement field between predictions and observations and the residual point-wise error field after the deformation. In contrast with the RMSE, the method captures the visual closeness of morphological patterns. Second, an optimal transport method defines the distance between predicted and observed morphological fields in terms of an optimal sediment transport field. The optimal corrective transport field moves the misplaced sediment from the predicted to the observed morphology at the lowest quadratic transportation cost. The root-mean-squared value of the optimal transport field, the RMSTE, is proposed as a new error metric. As opposed to the field deformation method, the optimal transport method is mass-conserving, parameter-free and symmetric. The RMSTE, unlike the RMSE, is able to discriminate between predictions that differ in the misplacement distance of predicted morphological features. It also avoids the consistent reward of the underestimation of morphological variability that the RMSE is prone to. Third, a scale-selective validation approach allows any metric to selectively address multiple spatial scales. It employs a smoothing filter in such a way that, in addition to the domain-averaged statistics, localized validation statistics and maps of prediction quality are obtained per scale (geographic extent or areal size of focus). The employed skill score weights how well the morphological structure and variability are simulated, while avoiding to reward the underestimation of variability. To fully describe prediction quality multiple metrics are required with a weighting determined by the goal of the simulation. Point-wise metrics should be supplemented with an error decomposition, as to avoid undesired underestimation of variability. Further, a set of performance metrics must include a metric, e.g. the RMSTE, that accounts for the spatial structure of the observed and predicted morphological fields.

OBJECTIVE There are no validated methods for predicting the timing of seizures. Using machine learning, we sought to forecast 24-hour risk of self-reported seizure from e-diaries. METHODS Data from 5,419 patients on SeizureTracker.com (including seizure count, type, and duration) were split into training (3,806 patients/1,665,215 patient-days) and testing (1,613 patients/549,588 patient-days) sets with no overlapping patients. An artificial intelligence (AI) program, consisting of recurrent networks followed by a multilayer perceptron ("deep learning" model), was trained to produce risk forecasts. Forecasts were made from a sliding window of 3-month diary history for each day of each patient's diary. After training, the model parameters were held constant and the testing set was scored. A rate-matched random (RMR) forecast was compared to the AI. Comparisons were made using the area under the receiver operating characteristic curve (AUC), a measure of binary discrimination performance, and the Brier score, a measure of forecast calibration. The Brier skill score (BSS) measured the improvement of the AI Brier score compared to the benchmark RMR Brier score. Confidence intervals (CIs) on performance statistics were obtained via bootstrapping. RESULTS The AUC was 0.86 (95% CI = 0.85-0.88) for AI and 0.83 (95% CI = 0.81-0.85) for RMR, favoring AI (p

There have been numerous statistical and dynamical downscaling model comparisons. However, differences in model skill can be distorted by inconsistencies in experimental set-up, inputs and output format. This paper harmonizes such factors when evaluating daily precipitation downscaled over the Iberian Peninsula by the Statistical DownScaling Model (SDSM) and two configurations of the dynamical Weather Research and Forecasting Model (WRF) (one with data assimilation (D) and one without (N)). The ERA-Interim reanalysis at 0.75 degrees resolution provides common inputs for spinning-up and driving the WRF model and calibrating SDSM. WRF runs and SDSM output were evaluated against ECA&D stations, TRMM, GPCP and EOBS gridded precipitation for 2010-2014 using the same suite of diagnostics. Differences between WRF and SDSM are comparable to observational uncertainty, but the relative skill of the downscaling techniques varies with diagnostic. The SDSM ensemble mean, WRF-D and ERAI have similar correlation scores (r=0.45-0.7), but there were large variations amongst SDSM ensemble members (r=0.3-0.6). The best Linear Error in Probability Space (LEPS=0.001-0.007) and simulations of precipitation amount were achieved by individual members of the SDSM ensemble. However, the Brier Skill Score shows these members do not improve the prediction by ERA-Interim, whereas precipitation occurrence is reproduced best by WRF-D. Similar skill was achieved by SDSM when applied to station or gridded precipitation data. Given the greater computational demands of WRF compared with SDSM, clear statements of expected value-added are needed when applying the former to climate impacts and adaptation research. This work was supported by Grants (CGL2013-45198-C2-1-R and CGL2016-76561-R, MINECO/ERDF, UE). S.J.G.R. was supported by a FPI Postdoctoral Research Grant (MINECO, BES-2014-069977). Additional funding was received from the University of the Basque Country (UPV/EHU, GIU 17/002). Intensive computational resources used in the project were provided by I2BASQUE. The authors thank the creators of the WRF/ARW and WRFDA systems for making them freely available to the community. The authors acknowledge the E-OBS dataset from the EU-FP6 project ENSEMBLES and the data providers in the ECA&D project. NOAA_OI_SST_V2 data provided by the NOAA/OAR/ESRL PSD, Boulder, Colorado, USA. Comments by two anonymous reviewers have significantly improved the manuscript.

We corrected monthly precipitation from 8 regional climate models using statistical bias correction. All models were corrected according to observations and parameters for bias correction were obtained for all models separately in every grid cells over European domain, using data between 1961 and 1990. Bias correction was validated in the period between 1991 and 2010 with RMSE, Brier score and Brier skill score. The results are encouraging, as mean and extremes were effectively corrected. After applying correction, large biases over Alps, at the East Adriatic cost, west coast of Norway and at the east end of the domain were removed. RMSE of corrected precipitation was lower than RMSE of simulated in 85% of European area and correction for all models failed in only 1.5% of European area. Also extremes were effectively corrected. According to the Brier skill score the probability for dry months was corrected in more than 52% of the European area and heavy precipitation events were corrected in almost 90% of the area. All validation measures suggest the correction of monthly precipitation was successful and therefore we can argue that the corrected precipitation fields will improve results of the climate impact models.

Operational global medium-range ensemble forecasts of tropical cyclone (TC) activity (genesis plus the subsequent track) are systematically evaluated to understand the skill of the state-of-the-art ensembles in forecasting TC activity as well as the relative benefits of a multicenter grand ensemble with respect to a single-model ensemble. The global ECMWF, JMA, NCEP, and UKMO ensembles are evaluated from 2010 to 2013 in seven TC basins around the world. The verification metric is the Brier skill score (BSS), which is calculated within a 3-day time window over a forecast length of 2 weeks to examine the skill from short- to medium-range time scales (0–14 days). These operational global medium-range ensembles are capable of providing guidance on TC activity forecasts that extends into week 2. Multicenter grand ensembles (MCGEs) tend to have better forecast skill (larger BSSs) than does the best single-model ensemble, which is the ECMWF ensemble in most verification time windows and most TC basins. The relative benefit of the MCGEs is relatively large in the north Indian Ocean and TC basins in the Southern Hemisphere where the BSS of the single-model ensemble is relatively small. The BSS metric and the reliability are found to be sensitive to the choice of threshold wind values that are used to define the model TCs.

This study attempts, for the first time, to predict the annual number of tropical cyclones (TCs) affecting New York State (NYS), as part of the effort of the New York State Resiliency Institute for Storms and Emergencies (RISE). A pure statistical prediction model and a statistical–dynamical hybrid prediction model have been developed based on the understanding of the physical mechanism between NYS TCs and associated large-scale climate variability. During the cold phase of El Niño–Southern Oscillation, significant circulation anomalies in the Atlantic Ocean provide favorable conditions for more recurving TCs into NYS. The pure statistical prediction model uses the sea surface temperature (SST) over the equatorial Pacific Ocean from the previous months. Cross validation shows that the correlation between the observed and predicted numbers of NYS TCs is 0.56 for the June 1979–2013 forecasts. Forecasts of the probability of one or more TCs impacting NYS have a Brier skill score of 0.35 compared to climatology. The statistical–dynamical hybrid prediction model uses Climate Forecast System, version 2, SST predictions, which are statistically downscaled to forecast the number of NYS TCs based on a stepwise regression model. Results indicate that the initial seasonal prediction for NYS TCs can be issued in February using the hybrid model, with an update in June using the pure statistical prediction model. Based on the statistical model, for 2014, the predicted number of TCs passing through NYS is 0.33 and the probability of one or more tropical cyclones crossing NYS is 30%, which are both below average and in agreement with the actual activity (0 NYS TCs).

Most solar flares originate in sunspot groups, where magnetic field changes lead to energy build-up and release. However, few flare-forecasting methods use information of sunspot-group evolution, instead focusing on static point-in-time observations. Here, a new forecast method is presented based upon the 24-hr evolution in McIntosh classification of sunspot groups. Evolution-dependent $\geqslant$C1.0 and $\geqslant$M1.0 flaring rates are found from NOAA-numbered sunspot groups over December 1988 to June 1996 (Solar Cycle 22; SC22) before converting to probabilities assuming Poisson statistics. These flaring probabilities are used to generate operational forecasts for sunspot groups over July 1996 to December 2008 (SC23), with performance studied by verification metrics. Major findings are: i) considering Brier skill score (BSS) for $\geqslant$C1.0 flares, the evolution-dependent McIntosh-Poisson method ($\text{BSS}_{\text{evolution}}=0.09$) performs better than the static McIntosh-Poisson method ($\text{BSS}_{\text{static}} = -0.09$); ii) low BSS values arise partly from both methods over-forecasting SC23 flares from the SC22 rates, symptomatic of $\geqslant$C1.0 rates in SC23 being on average $\approx$80% of those in SC22 (with $\geqslant$M1.0 being $\approx$50%); iii) applying a bias-correction factor to reduce the SC22 rates used in forecasting SC23 flares yields modest improvement in skill relative to climatology for both methods ($\mathrm{BSS}^{\mathrm{corr}}_{\mathrm{static}} = 0.09$ and $\mathrm{BSS}^{\mathrm{corr}}_{\mathrm{evolution}} = 0.20$) and improved forecast reliability diagrams., 21 pages, 9 figures

Based on the TIGGE multimodel ensemble forecasts taken from ECMWF, NCEP and UKMO, Bayesian Model Averaging (BMA) probabilistic forecasts of the 500hPa geopotential height over Northern Hemisphere with lead times of 1-15 days were conducted. A normal distribution was applied to the geopotential height. Comparison between the original ensemble forecasts and BMA forecasts suggests that the BMA forecasts perform better than the original ensemble forecasts for all lead times. The Brier Skill Score (BSS) of the BMA probabilistic forecasting shows that the BMA probabilistic forecasts are superior to the climatology forecasts. The original ensemble forecasts had a negative deviation, but the BMA prediction PDF was much better calibrated. Generally, the BMA method has significantly improved the quality of 500hPa geopotential height probabilistic forecasts.

NOAA’s second-generation retrospective forecast (reforecast) dataset was created using the currently operational Global Ensemble Forecast System (GEFS). It has the potential to accurately forecast daily reference evapotranspiration ETo and can be useful for water management. This study was conducted to evaluate daily ETo forecasts using the GEFS reforecasts in the southeastern United States (SEUS) and to incorporate the ETo forecasts into irrigation scheduling to explore the usefulness of the forecasts for water management. ETo was estimated using the Penman–Monteith equation, and ensemble forecasts were downscaled and bias corrected using a forecast analog approach. The overall forecast skill was evaluated using the linear error in probability space skill score, and the forecast in five categories (terciles and 10th and 90th percentiles) was evaluated using the Brier skill score, relative operating characteristic, and reliability diagrams. Irrigation scheduling was evaluated by water deficit WD forecasts, which were determined based on the agricultural reference index for drought (ARID) model driven by the GEFS-based ETo forecasts. All forecast skill was generally positive up to lead day 7 throughout the year, with higher skill in cooler months compared to warmer months. The GEFS reforecast improved ETo forecast skill for all lead days over the SEUS compared to the first-generation reforecast. The WD forecasts driven by the ETo forecasts showed higher accuracy and less uncertainty than the forecasts driven by climatology, indicating their usefulness for irrigation scheduling, hydrological forecasting, and water demand forecasting in the SEUS.

Cho, M.; Yoon, H.-D.; Do, K.; Son, S., and Kim, I.–H., 2019. Comparative study on the numerical simulation o bathymetric changes under storm condition. In: Lee, J.L.; Yoon, J.-S.; Cho, W.C.; Muin, M., and Lee, J. (eds.), The 3rd International Water Safety Symposium. Journal of Coastal Research, Special Issue No. 91, pp. 106-110. Coconut Creek (Florida), ISSN 0749-0208.Topographic changes in the nearshore mainly occur in strong storm condition. To understand bathymetric changes, attempts were made using several numerical models. Nowadays, Delft3D and XBeach are one of the most popular numerical models for predictions of morphodynamics in the nearshore. In this study, a comparative study of Delft3D and XBeach was performed to understand bathymetric changes under storm condition. The model parameters were calibrated with field observations. Storm surge and storm-induced wave field were re-created to simulate morphodynamics modelling. Bathymetric evolution, as well as shoreline changes were in a good agreement with field observations. BSS (Brier Skill Score) evaluation was applied to compare the accuracy of predictions of both models.

Gracia, V., Garcia, M., Grifoll, M. and Sanchez-Arcilla, A., 2013. Breaching of a barrier beach under extreme events. The role of morphodynamic simulations In: Conley, D.C., Masselink, G., Russell, P.E. and O’Hare, T.J. (eds.), Proceedings 12 th International Coastal Symposium (Plymouth, England), Journal of Coastal Research, Special Issue No. 65, pp. 951-956, ISSN 0749-0208. A study of three breaching episodes of the Trabucador barrier beach (Spanish Mediterranean coast) is presented. The analysis is done from a modeling perspective using XBEACH and SWAN. The morphodynamic model (XBEACH) has been validated with the available information. The so obtained Brier Skill Score index has been 0.44 which can be considered as acceptable. Results indicate that in all cases breaching was taking place after 8 hours of wave action (storm) or less. Besides, four main parameters control the final morphodynamic response of the barrier: the storm intensity, the water level and the initial emerged topography and the storm duration. Major breaching occurs when swell waves are in coincidence with high water levels whereas if they are uncoupled the barrier tends to be breached by a diversity of smaller channels.

This is the second part of a paper on the improved seasonal precipitation forecasts for the Asian monsoon using 16 atmosphere–ocean coupled models. This study utilizes a large suite of coupled atmosphere–ocean models; this second part largely addresses the skill of rainfall anomaly forecasts. These include both deterministic and probabilistic skill measures such as the RMS errors, anomaly correlations, equitable threat scores, and the Brier skill score. It was possible to improve the skills of rainfall climatology from the use of a downscaled multimodel superensemble to very high levels, and it is of interest to ask how far this methodology would go toward improving the skills of seasonal rainfall anomaly forecasts. It is possible to go through a sequence of multimodel post processing to improve upon these skills by using a dense rain gauge network over Asia, downscaling forecasts for each member model, and constructing a multimodel superensemble that benefits from the persistence of errors of the member models. This paper addresses the spinup issues of the downscaling and the superensemble results where the number of years of model data needed for training phase, for the downscaling, and for the construction of the superensemble, is addressed. In the context of cross validation, the training phase includes 14 seasons of monsoon data. The forecast phase is only one season; it is this season that was not included in the training phase each time. The relationship between data length and the number of models needed for enhanced skills is another issue that is addressed. Seasonal climate forecasts over the larger monsoon Asia domain and over the regional belts are evaluated. The superensemble forecasts invariably have the highest skill compared to the member models globally and regionally. This is largely due to the presence of large systematic errors in models that carry low seasonal prediction skills. Such models carry persistent signatures of systematic errors, and their errors are recognized by the multimodel superensemble. The probabilistic skills show that the superensemble-based forecasts carry a much higher reliability score compared to the member models. This implies that the superensemble-based forecasts are the most reliable among all the member models. It is possible to examine the performance of models and of the superensemble during periods of heavy monsoon rainfall versus those for deficient monsoon rainfall seasons. One of the conclusions of this study is that given the uncertainties in current modeling for seasonal rainfall forecasts, post processing of multimodel forecasts, using the superensemble methodology, seems to provide the most promising results for the rainfall anomaly forecasts. These results are confirmed by an additional skill metric where the RMS errors and the correlations of forecast skills are evaluated using a normalized precipitation anomaly for the forecasts and the observed estimates.

We assessed the capability of the regional climate model (RCM) approach in simulating the monthly 50th and 99th percentiles of marine wind speed and significant wave height on the Japanese coast during tropical cyclone season (June-October) of 2002-2004. To simulate the marine winds, we performed dynamical downscaling of the Japanese Reanalysis (JRA-25) using three RCMs. We found that, by applying the Brier skill score, coastal wind speed of the RCMs is closer to the observed, when compared to the JRA-25. The better performance of the RCM approach was found to come from a successful reduction of the bias rather than that of the root mean square error. The RCMs, however, simulate erroneously low (high) marine wind speed northeast of Japan (along the Pacific coast and Kuroshio Extension). The erroneous marine wind speed was found to be related with the sea surface temperature bias associated with the coarse horizontal resolution of the JRA-25. We employed the dynamically-downscaled marine winds to force a wave model (WaveWATCH-III), and found that the marine winds of the RCMs are more effective than the JRA-25 in reproducing significant wave height on the Japan Sea coast.

The main objective of this study is to analyse the relationship between atmospheric circulation and high precipitation in Poland, using selected classifications of circulation types. Daily precipitation totals from 48 synoptic stations and three classifications of circulation types ( Litynski, 1969; Niedzwiedz, 1981, 2008; Ustrnul, 1997 ) were used. These classifications were described and compared. The relationships between high precipitation and atmospheric circulation were analysed by (i) determining those circulation types that favour the occurrence of high precipitation, (ii) analysing the spatial differentiation of the probability of high precipitation events for chosen cyclonic circulation types and (iii) the usefulness of the classifications to analyse relations between high precipitation in Poland and atmospheric circulation was assessed. Any study of relations between atmospheric circulation and climate elements demands a preceding detailed assessment of the usefulness of a given classification for a defined problem. The results for relations between high precipitation occurrence and circulation depend on the rules applied to classify circulation types. Ustrnul’s classification seems to be best suited to study the relationship between heavy precipitation and circulation in the southern part of Poland. This result is based on two methods, the range of probabilities for high precipitation in each circulation type of a given classification (Pr diff ) and the Brier skill score (BSS). The Niedzwiedz classification mainly produces very good results in central southern Poland where its probability range is clearly wider than elsewhere. Both methods seem to be less useful to study the relationships between high precipitation and circulation in northern Poland. Litynski’s classification delivers a greater role of the cyclonic types in forming high precipitation, but no definite prevailing type can be identified at many of the stations. When analysing relations, regional classification of circulation should be applied rather than macro-scales ones.

A novel approach is presented for the evaluation of circulation type classifications (CTCs) in terms of their capability to predict surface climate variations. The approach is analogous to that for probabilistic meteorological forecasts and is based on the Brier skill score. This score is shown to take a particularly simple form in the context of CTCs and to quantify the resolution of a climate variable by the classifications. The sampling uncertainty of the skill can be estimated by means of nonparametric bootstrap resampling. The evaluation approach is applied for a systematic intercomparison of 71 CTCs (objective and manual, from COST Action 733) with respect to their ability to resolve daily precipitation in the Alpine region. For essentially all CTCs, the Brier skill score is found to be higher for weak and moderate compared to intense precipitation, for winter compared to summer, and over the north and west of the Alps compared to the south and east. Moreover, CTCs with a higher number of types exhibit better skill than CTCs with few types. Among CTCs with comparable type number, the best automatic classifications are found to outperform the best manual classifications. It is not possible to single out one ‘best’ classification for Alpine precipitation, but there is a small group showing particularly high skill.

Deterministic and fuzzy verification methods are compared to assess the Quantitative Precipitation Forecasts (QPF) performance of an hierarchy of models run at Meteo-France: two operational models (ARPEGE and ALADIN) and a prototype version of the high resolution model AROME. The reference data are 24-h accumulated rainfall values measured by the French climatological network of rain gauges. The deterministic forecasts are converted to frequencies of threshold exceedence in a neighbourhood in order to apply a fuzzy verification method, able to determine the influence of the double penalty. Local and regional versions of the Brier skill score (BSS) are computed and persistence is the selected reference system. An optimal size of the neighbourhood can be determined from the local version of the BSS. The regional version of the BSS increases with the size of the neighbourhood and provides useful information on the reduction of the error for large scales. All the scores show that the ALADIN 3DVAR implementation improves the quality of the ALADIN QPF in comparison to the ARPEGE QPF. Deterministic verification methods show that AROME improves the QPF for weak precipitation but produces too many false alarms for heavy rain. Moreover, the fuzzy approach removes more substantially the double penalty for the heavy rain forecasts of AROME than for the light precipitation of ALADIN. After recalibration the ALADIN QPF have better deterministic scores than the AROME QPF. Nevertheless, fuzzy verification methods prove the contrary: for scales larger than 50 km, the AROME scores exceed the ALADIN scores, but not significantly. Copyright © 2008 Royal Meteorological Society

This note describes how the widely used Brier and ranked probability skill scores (BSS and RPSS, respectively) can be correctly applied to quantify the potential skill of probabilistic multimodel ensemble forecasts. It builds upon the study of Weigel et al. where a revised RPSS, the so-called discrete ranked probability skill score (RPSSD), was derived, circumventing the known negative bias of the RPSS for small ensemble sizes. Since the BSS is a special case of the RPSS, a debiased discrete Brier skill score (BSSD) could be formulated in the same way. Here, the approach of Weigel et al., which so far was only applicable to single model ensembles, is generalized to weighted multimodel ensemble forecasts. By introducing an “effective ensemble size” characterizing the multimodel, the new generalized RPSSD can be expressed such that its structure becomes equivalent to the single model case. This is of practical importance for multimodel assessment studies, where the consequences of varying effective ensemble size need to be clearly distinguished from the true benefits of multimodel combination. The performance of the new generalized RPSSD formulation is illustrated in examples of weighted multimodel ensemble forecasts, both in a synthetic random forecasting context, and with real seasonal forecasts of operational models. A central conclusion of this study is that, for small ensemble sizes, multimodel assessment studies should not only be carried out on the basis of the classical RPSS, since true changes in predictability may be hidden by bias effects—a deficiency that can be overcome with the new generalized RPSSD.

The Brier score and Brier skill score are commonly used measures for evaluating probability forecasts. An oft-cited desirable property of these measures is that they are strictly proper, meaning that forecasters wishing to optimise the expected value of these measures should report their true beliefs and not ‘hedge’ the forecast by reporting probabilities that differ from their true beliefs. In this article, it is argued that the propriety of the Brier score only discourages hedging if the utility of the forecaster is a linear function of the score. In particular, it is demonstrated, that if known performance targets based on the score are imposed on the forecaster then this negates the propriety of the score. Forecasters wishing to maximise the probability of achieving the target value of the Brier score or Brier skill score should hedge their forecasts rather than report their true beliefs. In contrast, performance targets drawn from a uniform distribution which are unknown to the forecaster during the evaluation period will induce the utility function of the forecaster to be a linear function of the Brier score. This preserves the propriety of the scoring rule. © Crown Copyright 2007. Reproduced with the permission of the Controller of HMSO. Published by John Wiley & Sons, Ltd

This study aims at improving 0–3 day probabilistic forecasts of precipitation events in Norway. For this purpose a limited-area ensemble prediction system (LAMEPS) is tested. The horizontal resolution of LAMEPS is 28 km, and there are 31 levels in the vertical. The state variables provided as initial and lateral boundary conditions for the limited-area forecasts are perturbed using a dedicated version of the European Centre for Medium-Range Weather Forecasts (ECMWF) global ensemble prediction system, TEPS. These are constructed by combining initial and evolved singular vectors that at final time (48 h) are targeted to maximize the total energy in a domain containing northern Europe and adjacent sea areas. The resolution of TEPS is T255 with 40 levels. The test period includes 45 cases with 21 ensemble members in each case. We focus on 24 h accumulated precipitation rates with special emphasis on intense events. We also investigate a combination of TEPS and LAMEPS resulting in a system (NORLAMEPS) with 42 ensemble members. NORLAMEPS is compared with the 21-member LAMEPS and TEPS as well as the regular 51-member EPS run at ECMWF. The benefit of using targeted singular vectors is seen by comparing the 21-member TEPS with the 51-member operational EPS, as TEPS has considerably larger spread between ensemble members. For other measures, such as Brier Skill Score (BSS) and Relative Operating Characteristic (ROC) curves, the scores of the two systems are for most cases comparable, despite the difference in ensemble size. NORLAMEPS has the largest ensemble spread of all four ensemble systems studied in this paper, while EPS has the smallest spread. Nevertheless, EPS has higher BSS with NORLAMEPS approaching for the highest precipitation thresholds. For the area under the ROC curve, NORLAMEPS is comparable with or better than EPS for medium to large thresholds. Copyright © 2006 Royal Meteorological Society

We propose autocorrelation-robust asymptotic variances of the Brier score and Brier skill score, which are generally applicable in circumstances with weak serial correlation. An empirical application in macroeconomics underscores the importance of taking care of serial correlation. We find that the conventional variances are too conservative to account for the sampling variability in estimating the Brier (skill) score.

We derive autocorrelation-robust asymptotic variances of the Brier score and Brier skill score, which are generally applicable in circumstances with weak serial correlation. A simulation experiment and an empirical application from macroeconomics underscore the importance of taking care of serial correlation. We find that the conventional variances are too conservative to account for the sampling variability in estimating the Brier (skill) score.

The superensemble technique has been proven to be successful in producing a deterministic forecast superior not only to any of the individual models going into it, but also to the multimodel ensemble forecast. Research so far has been done on the superensemble as a deterministic forecast, and it has been shown that using the superensemble method leads to a significant reduction in rms errors. This paper investigates the skill of the superensemble as a probabilistic forecast, and it compares it with that of the multimodel ensemble. Using the Atmospheric Model Intercomparison Project (AMIP I) seasonal multimodel precipitation forecasts, probability forecasts are defined for the multimodel ensemble and for the multimodel superensemble. The Brier skill score of these forecasts is calculated for different thresholds of precipitation anomaly. It is shown that both the multimodel ensemble and the superensemble probability forecasts are much better than climatological forecast and that the superensemble ...

A decomposition of the Brier skill score shows that the performance of judgmental forecasts depends on seven components: environmental predictability, fidelity of the information system, match between environment and forecaster, reliability of information acquisition, reliability of information processing, conditional bias, and unconditional bias. These components provide a framework for research on the forecasting process. Selected literature addressing each component is reviewed, and implications for improving judgmental forecasting are discussed.

[1] The probabilistic skill of ensemble forecasts for the first month and the first season of the forecasts is assessed, where model uncertainty is represented by the a) multi-model, b) perturbed parameters, and c) stochastic parameterisation ensembles. The main foci of the assessment are the Brier Skill Score for near-surface temperature and precipitation over land areas and the spread-skill relationship of sea surface temperature in the tropical equatorial Pacific. On the monthly timescale, the ensemble forecast system with stochastic parameterisation provides overall the most skilful probabilistic forecasts. On the seasonal timescale the results depend on the variable under study: for near surface temperature the multi-model ensemble is most skilful for most land regions and for global land areas. For precipitation, the ensemble with stochastic parameterisation most often produces the highest scores on global and regional scales. Our results indicate that stochastic parameterisations should now be developed for multi-decadal climate predictions using earth-system models.

To investigate new ways to improve the prediction of gravel barrier beach erosion, XBeach, a processed-based numerical model capable of computing nearshore circulation and morphodynamics, including overwash and breaching, has been used in 1D mode to simulate: a) prototype-scale laboratory experiments; and b) storm response of a natural gravel barrier. For laboratory tests XBeach predicted beach face erosion well and demonstrated good quantitative skill (Brier skill score, BSS, typically 0.65). At field-scale, XBeach was used to simulate the response to a storm of a steep (average tanβ = 0.12), 4.5km-long and 100m–140m wide macrotidal gravel barrier (D50 = 2mm to 10mm). Although in this case XBeach predicted erosion with moderate accuracy (BSS = 0.60), the development of a berm was not reproduced well by the model. XBeach was also used to improve estimation of threshold conditions for overwash.

In the framework of the COST733 Action "Harmonisation and Applications of Weather Types Classifications for European Regions" a new circulation type classification software (hereafter, referred to as cost733class software) is developed. The cost733class software contains a variety of (European) classification methods and is flexible towards choice of domain of interest, input variables, time step, number of circulation types, sequencing and (weighted) target variables. This work introduces the capabilities of the cost733class software in which the resulting circulation types (CTs) from various circulation type classifications (CTCs) are applied on observed summer surface ozone concentrations in Central Europe. Firstly, the main characteristics of the CTCs in terms of circulation pattern frequencies are addressed using the baseline COST733 catalogue (cat 2.0), at present the latest product of the new cost733class software. In a second step, the probabilistic Brier skill score is used to quantify the explanatory power of all classifications in terms of the maximum 8 hourly mean ozone concentrations exceeding the 120-mu g/m(3) threshold; this was based on ozone concentrations from 130 Central European measurement stations. Averaged evaluation results over all stations indicate generally higher performance of CTCs with a higher number of types. Within the subset of methodologies with a similar number of types, the results suggest that the use of CTCs based on optimisation algorithms are performing slightly better than those which are based on other algorithms (predefined thresholds, principal component analysis and leader algorithms). The results are further elaborated by exploring additional capabilities of the cost733class software. Sensitivity experiments are performed using different domain sizes, input variables, seasonally based classifications and multiple-day sequencing. As an illustration, CTCs which are also conditioned towards temperature with various weights are derived and tested similarly. All results exploit a physical interpretation by adapting the environment-to-circulation approach, providing more detailed information on specific synoptic conditions prevailing on days with high surface ozone concentrations. This research does not intend to bring forward a favourite classification methodology or construct a statistical ozone forecasting tool but should be seen as an introduction to the possibilities of the cost733class software. It this respect, the results presented here can provide a basic user support for the cost733class software and the development of a more user- or application-specific CTC approach. Theoretical and Applied Climatology