Search

The performance of conceptual catchment runoff models may highly depend on the specific choice of calibration methods made by the user. Particle Swarm Optimization (PSO) and Differential Evolution (DE) are two well-known families of Evolutionary Algorithms that are widely used for calibration of hydrological and environmental models. In the present paper, five DE and five PSO optimization algorithms are compared regarding calibration of two conceptual models, namely the Swedish HBV model (Hydrologiska Byrans Vattenavdelning model) and the French GR4J model (modèle du Génie Rural à 4 paramètres Journalier) of the Kamienna catchment runoff. This catchment is located in the middle part of Poland. The main goal of the study was to find out whether DE or PSO algorithms would be better suited for calibration of conceptual rainfall-runoff models. In general, four out of five DE algorithms perform better than four out of five PSO methods, at least for the calibration data. However, one DE algorithm constantly performs very poorly, while one PSO algorithm is among the best optimizers. Large differences are observed between results obtained for calibration and validation data sets. Differences between optimization algorithms are lower for the GR4J than for the HBV model, probably because GR4J has fewer parameters to optimize than HBV.

COVID-19 disease, which highly affected global life in 2020, led to a rapid scientific response. Versatile optimization methods found their application in scientific studies related to COVID-19 pandemic. Differential Evolution (DE) and Particle Swarm Optimization (PSO) are two metaheuristics that for over two decades have been widely researched and used in various fields of science. In this paper a survey of DE and PSO applications for problems related with COVID-19 pandemic that were rapidly published in 2020 is presented from two different points of view: 1. practitioners seeking the appropriate method to solve particular problem, 2. experts in metaheuristics that are interested in methodological details, inter comparisons between different methods, and the ways for improvement. The effectiveness and popularity of DE and PSO is analyzed in the context of other metaheuristics used against COVID-19. It is found that in COVID-19 related studies: 1. DE and PSO are most frequently used for calibration of epidemiological models and image-based classification of patients or symptoms, but applications are versatile, even interconnecting the pandemic and humanities; 2. reporting on DE or PSO methodological details is often scarce, and the choices made are not necessarily appropriate for the particular algorithm or problem; 3. mainly the basic variants of DE and PSO that were proposed in the late XX century are applied, and research performed in recent two decades is rather ignored; 4. the number of citations and the availability of codes in various programming languages seems to be the main factors for choosing metaheuristics that are finally used.

Water temperature is one of the most important indicators of aquatic system, and accurate forecasting of water temperature is crucial for rivers. It is a complex process to accurately predict stream water temperature as it is impacted by a lot of factors (e.g., meteorological, hydrological, and morphological parameters). In recent years, with the development of computational capacity and artificial intelligence (AI), AI models have been gradually applied for river water temperature (RWT) forecasting. The current survey aims to provide a systematic review of the AI applications for modeling RWT. The review is to show the progression of advances in AI models. The pros and cons of the established AI models are discussed in detail. Overall, this research will provide references for hydrologists and water resources engineers and planners to better forecast RWT, which will benefit river ecosystem management.

Conceptual lumped rainfall-runoff models are frequently used for various environmental problems. To put them into practice, both the model calibration method and data series of the area-averaged precipitation and air temperature are needed. In the case when data from more than one measurement station are available, first the catchment-averaged meteorological data series are usually obtained by some method, and then they are used for calibration of a lumped rainfall-runoff model. However, various optimization methods could easily be applied to simultaneously calibrate both the aggregation weights attributed to various meteorological stations to obtain a lumped meteorological data series and the rainfall-runoff model parameters. This increases the problem dimensionality but allows the optimization procedure to choose the data that are most important for the rainfall-runoff process in a particular catchment, without a priori assumptions. We test the idea using two conceptual models, HBV and GR4J, and three mutually different, relatively recently proposed Evolutionary Computation and Swarm Intelligence optimization algorithms, that are applied to three catchments located in Poland and northwestern USA. We consider two cases: with and without the model error correction applied to the rainfall-runoff models. It is shown that for the calibration period, joint optimization of the weights used to aggregate the meteorological data and the parameters of the rainfall-runoff model improves the results. However, the results for the validation period are inconclusive and depend on the model, error correction, optimization algorithm, and catchment.

Among various stream temperature models those based on nonlinear regression frequently attract attention due to their simplicity and small number of required variables. Among such approaches the logistic regression model developed twenty years ago for weekly data is still widely used in various scientific studies that require quick and simple calculation of stream water temperature. The model has been modified a number of times in recent years to capture the relationship between daily stream water temperatures, air temperatures and discharge. In this study, we propose further modifications of the logistic regression model that do not require any additional variables that may be hard to measure. The proposed models capture the relationship between the stream temperature and the declination of the Sun, the air temperature and the discharge from a number of recent observations. The proposed approaches are tested on six rivers located in diverse orographic conditions of temperate climate zones of Europe and USA. Although the proposed models remain very simple, their performances are competitive against the performances of more advanced semi-physical or data-driven models.

Various methods are used in the literature for calibration of conceptual rainfall-runoff models. However, very rarely the question on the relation between the number of model runs (or function calls) and the quality of solutions found is asked. In this study two lumped conceptual rainfall-runoff models (HBV and GR4J with added snow module) are calibrated for five catchments, located in temperate climate zones of USA and Poland, by means of three modern variants of Evolutionary Computation and Swarm Intelligence optimization algorithms with four different maximum numbers of function calls set to 1000, 3000, 10,000 and 30,000. At the calibration stage, when more than 10,000 function calls is used, only marginal improvement in model performance has been found, irrespective of the catchment or calibration algorithm. For validation data, the relation between the number of function calls and model performance is even weaker, in some cases the longer calibration, the poorer modelling performance. It is also shown that the opinion on the model performance based on different popular hydrological criteria, like the Nash-Sutcliffe coefficient or the Persistence Index, may be misleading. This is because very similar, largely positive values of Nash-Sutcliffe coefficient obtained on different catchments may be accompanied by contradictory values of the Persistence Index.

A number of physical or data-driven models have been proposed to evaluate stream water temperatures based on hydrological and meteorological observations. However, physical models require a large amount of information that is frequently unavailable, while data-based models ignore the physical processes. Recently the air2stream model has been proposed as an intermediate alternative that is based on physical heat budget processes, but it is so simplified that the model may be applied like data-driven ones. However, the price for simplicity is the need to calibrate eight parameters that, although have some physical meaning, cannot be measured or evaluated a priori. As a result, applicability and performance of the air2stream model for a particular stream relies on the efficiency of the calibration method. The original air2stream model uses an inefficient 20-year old approach called Particle Swarm Optimization with inertia weight. This study aims at finding an effective and robust calibration method for the air2stream model. Twelve different optimization algorithms are examined on six different streams from northern USA (states of Washington, Oregon and New York), Poland and Switzerland, located in both high mountains, hilly and lowland areas. It is found that the performance of the air2stream model depends significantly on the calibration method. Two algorithms lead to the best results for each considered stream. The air2stream model, calibrated with the chosen optimization methods, performs favorably against classical streamwater temperature models. The MATLAB code of the air2stream model and the chosen calibration procedure (CoBiDE) are available as Supplementary Material on the Journal of Hydrology web page.

In the majority of studies aiming at stream temperature warming due to climate change just a single water temperature model is used. Choosing a single model may highly impact the conclusions from the study. In this paper four relatively different empirical or semi-empirical models: perceptron neural networks, product unit networks, extended logistic regression and air2stream were applied to project the impact of climate change on water temperature in rivers located in temperate climatic zones of the USA and Poland. The models were driven by daily air temperature and streamflow projected by the rainfall-runoff model. In the first step, the models were calibrated and validated. Then the projections of water temperature were derived for the historical periods and two future periods taking into account: (a) climate simulations from the CORDEX initiative (NA-CORDEX and EURO-CORDEX), (b) the GR4J rainfall-runoff model and (c) different water temperature models. The obtained results indicate that due to global warming, the stream temperatures are expected to increase by about 1–2 °C for 2021–2050 and by 2–3 °C for 2071–2100 periods. These changes are not uniformly distributed throughout the year. The largest warming in the USA is found in the summertime, in Poland – in spring and autumn. For some months the discrepancies in the projected stream temperature between various stream temperature models are large. Product unit neural network, logistic regression-based model or air2stream occasionally led to projections that differ from those obtained by the majority of models even by 2 °C. We strongly recommend using at least a few stream temperature models for analysing the impact of climate change on water temperatures or the fate of the aquatic ecosystem.

For about two decades neural networks are widely used for river temperature modelling. However, in recent years one has to distinguish between the “classical” shallow neural networks, and deep learning networks. The applicability of rapidly developing deep learning networks to stream water temperature modelling may be limited, but some methods developed for deep learning, if properly re-considered, may efficiently improve performance of shallow networks. Dropout is widely considered the method that allows deep learning networks to avoid overfitting to training data, facilitating its implementations to versatile problems. Recently the successful applicability of dropout for river temperature modelling by means of shallow multilayer perceptron neural networks has been introduced. In the present study we propose to use dropout solely for input neurons of product unit neural networks for the purpose of stream temperature modelling. We perform tests on data collected from six catchments located in temperate climate zones on two continents in various orographic conditions. We show that the average performance of product unit neural networks trained with input dropout is better than the average performance of product units without dropout, product units with dropout applied to every layer of the networks, multilayer perceptron neural networks with or without dropout, and the semi-physical air2stream model. The advantage of product unit neural networks with input dropout is statistically significant on hilly or mountainous catchments; the performance on flat ones is similar to the performances obtained from competitive models.

The air2water model is a widely used tool for the forecasting of lake water temperatures and projection of climate change. It has been applied in thousands of lakes worldwide. However, it employed the Particle Swarm Optimization (PSO) as the calibration method, which was proposed more than 20 years ago. In this study, 12 advanced optimization algorithms proposed during the recent few years were implemented to calibrate and validate the air2water model. Daily observed water temperature in temperate lakes (22 lowland lakes in Poland) were used to evaluate the model performance. Each optimizer was run 30 times with 3 different settings of the maximum number of function calls, 5000, 20,000, and 100,000. We have found that the PSO method used in the original air2water model performs relatively poor compared with most of the recent algorithms, and the majority of recently proposed algorithms reach the similar best solutions on many lakes when the highest number of function calls is allowed. However, only the HARD-DE (hierarchical archive-based mutation strategy with depth information of differential evolution) algorithm is never outperformed by any competitor, irrespective which lake and maximum number of function calls is considered. As a result, we highly recommend using HARD-DE as the calibration method for the air2water model in the future studies.

Artificial neural networks (ANNs) become widely used for runoff forecasting in numerous studies. Usually classical gradient-based methods are applied in ANN training and a single ANN model is used. To improve the modelling performance, in some papers ensemble aggregation approaches are used whilst in others, novel training methods are proposed. In this study, the usefulness of both concepts is analysed. First, the applicability of a large number of population-based metaheuristics to ANN training for runoff forecasting is tested on data collected from four catchments, namely upper Annapolis (Nova Scotia, Canada), Biala Tarnowska (Poland), upper Allier (France) and Axe Creek (Victoria, Australia). Then, the importance of the search for novel training methods is compared with the importance of the use of a very simple ANN ensemble aggregation approach. It is shown that although some metaheuristics may slightly outperform the classical gradient-based Levenberg-Marquardt algorithm for a specific catchment, none performs better for the majority of the tested ones. One may also point out a few metaheuristics that do not suit ANN training at all. On the other hand, application of even the simplest ensemble aggregation approach clearly improves the results when the ensemble members are trained by any suitable algorithms.EDITOR D. Koutsoyiannis; ASSOCIATE EDITOR E. Toth EDITOR D. Koutsoyiannis; ASSOCIATE EDITOR E. Toth

Although deep learning applicability in various fields of earth sciences is rapidly increasing, shallow multilayer-perceptron neural networks remain widely used for regression problems. Despite many clear distinctions between deep and shallow neural networks, some techniques developed for deep learning may help improve shallow models. Dropout, a simple approach to avoid overfitting by randomly skipping some nodes in a net during each training iteration, is among methodological features that made deep learning networks successful. In this study we give a review of dropout methods and empirically show that, when used together with early-stopping, dropout and its variant dropconnect could improve performance of shallow multi-layer perceptron neural networks. Shallow neural networks are applied to streamwater temperature modelling problems in six catchments, based on air temperature, river discharge and declination of the Sun. We found that when training of a particular neural network architecture that includes at least a few hidden nodes is repeated many times, dropout reduces the number of models that perform poorly on testing data, and hence improves the mean performance. If the number of inputs or hidden nodes is very low, dropout only disturbs training. However, nodes need to be dropped out with a much lower probability than in the case of deep neural networks (about 1%, instead of 10–50% for deep learning), due to a much smaller number of nodes in the network. Larger probabilities of dropping out nodes hinder convergence of the training algorithm and lead to poor results for both calibration and testing data. Dropconnect turned out to be slightly more effective than dropout.

Streamwater temperature may be severely affected by the global warming. Different types of models could be used to evaluate the regime of water temperatures in future climatic conditions, including artificial neural networks. As neural networks have no physical background, they require calibration of large number of parameters. This is typically done by gradient-based algorithms, however there is an ongoing debate on usefulness of metaheuristics for this task. In this paper more than ten Swarm Intelligence and Evolutionary Algorithms, including one developed especially for this study, are tested to train four kinds of artificial neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) for daily water temperature prediction in a natural river located in temperate climate zone. The results are compared with the ones obtained when the classical Levenberg-Marquardt algorithm is used. Finally, the ensemble aggregating approach is tested. Although the research confirms that most metaheuristics do not suite well for training any kind of neural networks, there are exceptions that include the newly proposed heuristic. However, the gain achieved when using even the best metaheuristics is low, comparing to the effort (computational time and complexity of such algorithms). Using ensemble aggregation approach seems to have higher impact on the model performance than seeking for new training methods.

Summary A number of methods have been proposed for the prediction of streamwater temperature based on various meteorological and hydrological variables. The present study shows a comparison of few types of data-driven neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) and nearest neighbour approach for short time streamwater temperature predictions in two natural catchments (mountainous and lowland) located in temperate climate zone, with snowy winters and hot summers. To allow wide applicability of such models, autoregressive inputs are not used and only easily available measurements are considered. Each neural network type is calibrated independently 100 times and the mean, median and standard deviation of the results are used for the comparison. Finally, the ensemble aggregation approach is tested. The results show that simple and popular multi-layer perceptron neural networks are in most cases not outperformed by more complex and advanced models. The choice of neural network is dependent on the way the models are compared. This may be a warning for anyone who wish to promote own models, that their superiority should be verified in different ways. The best results are obtained when mean, maximum and minimum daily air temperatures from the previous days are used as inputs, together with the current runoff and declination of the Sun from two recent days. The ensemble aggregation approach allows reducing the mean square error up to several percent, depending on the case, and noticeably diminishes differences in modelling performance obtained by various neural network types.

Summary Artificial neural networks (ANNs) becomes very popular tool in hydrology, especially in rainfall–runoff modelling. However, a number of issues should be addressed to apply this technique to a particular problem in an efficient way, including selection of network type, its architecture, proper optimization algorithm and a method to deal with overfitting of the data. The present paper addresses the last, rarely considered issue, namely comparison of methods to prevent multi-layer perceptron neural networks from overfitting of the training data in the case of daily catchment runoff modelling. Among a number of methods to avoid overfitting the early stopping, the noise injection and the weight decay have been known for about two decades, however only the first one is frequently applied in practice. Recently a new methodology called optimized approximation algorithm has been proposed in the literature. Overfitting of the training data leads to deterioration of generalization properties of the model and results in its untrustworthy performance when applied to novel measurements. Hence the purpose of the methods to avoid overfitting is somehow contradictory to the goal of optimization algorithms, which aims at finding the best possible solution in parameter space according to pre-defined objective function and available data. Moreover, different optimization algorithms may perform better for simpler or larger ANN architectures. This suggest the importance of proper coupling of different optimization algorithms, ANN architectures and methods to avoid overfitting of real-world data – an issue that is also studied in details in the present paper. The study is performed for Annapolis River catchment, characterized by significant seasonal changes in runoff, rapid floods during winter and spring, moderately dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice. The present paper shows that the elaborated noise injection method may prevent overfitting slightly better than the most popular early stopping approach. However, the implementation of noise injection to real-world problems is difficult and the final model performance depends significantly on a number of very technical details, what somehow limits its practical applicability. It is shown that optimized approximation algorithm does not improve the results obtained by older methods, possibly due to over-simplified criterion of stopping the algorithm. Extensive calculations reveal that Evolutionary Computation-based algorithm performs better for simpler ANN architectures, whereas classical gradient-based Levenberg–Marquardt algorithm is able to benefit from additional input variables, representing precipitation and snow cover from one more previous day, and from more complicated ANN architectures. This confirms that the curse of dimensionality has severe impact on the performance of Evolutionary Computing methods.

In this paper Product-Units neural networks (PUNNs), which probably have never been used within the field of hydrology, are introduced and applied for catchment runoff forecasting in cold climate zones. This type of neural networks, a subclass of higher order neural networks uses product nodes with inputs raised to exponential weights in one layer and well-known summation nodes in another layer. The present paper empirically shows that PUNNs with unbounded weights are difficult to train and do not perform well for catchment runoff forecasting. However, a very good predictive performance may be achieved when the weights are bounded within [−1, 1] interval. Several variants of optimization methods, mostly Differential Evolution-based algorithms, and a few approaches enabling good generalization capabilities of neural networks are compared in order to select the appropriate technique for PUNNs training. PUNNs with parameters bounded within [−1, 1] interval are shown to outperform Multi-Layer Perceptron neural networks and HBV conceptual model for runoff forecasting case study at Annapolis River, Nova Scotia, Canada. Gradient-based Levenberg–Marquardt algorithm and Evolutionary Computation-based Differential Evolution with Global and Local Neighborhood method turn out to be the most successful among the tested training algorithms. Surprisingly, in the case of Product-Units neural networks with weights bounded within [−1, 1] interval using noise injection or early stopping do not improve the results obtained when no method to avoid overfitting is used.

Summary Although neural networks have been widely applied to various hydrological problems, including river flow forecasting, for at least 15 years, they have usually been trained by means of gradient-based algorithms. Recently nature inspired Evolutionary Computation algorithms have rapidly developed as optimization methods able to cope not only with non-differentiable functions but also with a great number of local minima. Some of proposed Evolutionary Computation algorithms have been tested for neural networks training, but publications which compare their performance with gradient-based training methods are rare and present contradictory conclusions. The main goal of the present study is to verify the applicability of a number of recently developed Evolutionary Computation optimization methods, mostly from the Differential Evolution family, to multi-layer perceptron neural networks training for daily rainfall–runoff forecasting. In the present paper eight Evolutionary Computation methods, namely the first version of Differential Evolution (DE), Distributed DE with Explorative–Exploitative Population Families, Self-Adaptive DE, DE with Global and Local Neighbors, Grouping DE, JADE, Comprehensive Learning Particle Swarm Optimization and Efficient Population Utilization Strategy Particle Swarm Optimization are tested against the Levenberg–Marquardt algorithm – probably the most efficient in terms of speed and success rate among gradient-based methods. The Annapolis River catchment was selected as the area of this study due to its specific climatic conditions, characterized by significant seasonal changes in runoff, rapid floods, dry summers, severe winters with snowfall, snow melting, frequent freeze and thaw, and presence of river ice – conditions which make flow forecasting more troublesome. The overall performance of the Levenberg–Marquardt algorithm and the DE with Global and Local Neighbors method for neural networks training turns out to be superior to other Evolutionary Computation-based algorithms. The Levenberg–Marquardt optimization must be considered as the most efficient one due to its speed. Its drawback due to possible sticking in poor local optimum can be overcome by applying a multi-start approach.

The transient storage model of the transport of solutes in rivers may be used as a warning mechanism, when the values of its three parameters are known. It requires expensive and time-consuming tracer tests to be performed, as no reliable empirical formulae have been developed so far. In this paper the parameters of the transient storage model were evaluated by means of multi-layer perceptron artificial neural networks (ANN), using easily accessible hydraulic and morphometric data as inputs. The major obstacle was the scarcity of available data. For comparison purposes, the ANNs were trained by three optimization techniques, namely the Levenberg-Marquardt algorithm and two global approaches (much less popular in hydrological sciences): particle swarm optimization and differential evolution. Some minor modifications in these procedures, enabling the neural networks to avoid overfitting, were proposed. The ANNs revealed their superiority over available empirical formulae and the linear regression m...

In recent years sampling approaches have been used more widely than optimization algorithms to find parameters of conceptual rainfall–runoff models, but the difficulty of calibration of such models remains in dispute. The problem of finding a set of optimal parameters for conceptual rainfall–runoff models is interpreted differently in various studies, ranging from simple to relatively complex and difficult. In many papers, it is claimed that novel calibration approaches, so-called metaheuristics, outperform the older ones when applied to this task, but contradictory opinions are also plentiful. The present study aims at calibration of two simple lumped conceptual hydrological models, HBV and GR4J, by means of a large number of metaheuristic algorithms. The tests are performed on four catchments located in regions with relatively similar climatic conditions, but on different continents. The comparison shows that, although parameters found may somehow differ, the performance criteria achieved with simple lumped models calibrated by various metaheuristics are very similar and differences are insignificant from the hydrological point of view. However, occasionally some algorithms find slightly better solutions than those found by the vast majority of methods. This means that the problem of calibration of simple lumped HBV or GR4J models may be deceptive from the optimization perspective, as the vast majority of algorithms that follow a common evolutionary principle of survival of the fittest lead to sub-optimal solutions.

In this paper, Multi-Layer Perceptron and Radial-Basis Function Neural Networks, along with the Nearest Neighbour approach and linear regression are utilized for flash-flood forecasting in the mountainous Nysa Klodzka river catchment. It turned out that the Radial-Basis Function Neural Network is the best model for 3- and 6-h lead time prediction and the only reliable one for 9-h lead time forecasting for the largest flood used as a test case.

The prediction of temporal concentration profiles of a transported pollutant in a river is still a subject of ongoing research efforts worldwide. The present paper is aimed at studying the possibility of using Multi-Layer Perceptron Neural Networks to evaluate the whole concentration versus time profile at several cross-sections of a river under various flow conditions, using as little information about the river system as possible. In contrast with the earlier neural networks based work on longitudinal dispersion coefficients, this new approach relies more heavily on measurements of concentration collected during tracer tests over a range of flow conditions, but fewer hydraulic and morphological data are needed. The study is based upon 26 tracer experiments performed in a small river in Edinburgh, UK (Murray Burn) at various flow rates in a 540 m long reach. The only data used in this study were concentration measurements collected at 4 cross-sections, distances between the cross-sections and the injection site, time, as well as flow rate and water velocity, obtained according to the data measured at the 1st and 2nd cross-sections. The four main features of concentration versus time profiles at a particular cross-section, namely the peak concentration, the arrival time of the peak at the cross-section, and the shapes of the rising and falling limbs of the profile are modeled, and for each of them a separately designed neural network was used. There was also a variant investigated in which the conservation of the injected mass was assured by adjusting the predicted peak concentration. The neural network methods were compared with the unit peak attenuation curve concept. In general the neural networks predicted the main features of the concentration profiles satisfactorily. The predicted peak concentrations were generally better than those obtained using the unit peak attenuation method, and the method with mass-conservation assured generally performed better than the method that did not account for mass-conservation. Predictions of peak travel time were also better using the neural networks than the unit peak attenuation method. Including more data into the neural network training set clearly improved the prediction of the shapes of the concentration profiles. Similar improvements in peak concentration were less significant and the travel time prediction appeared to be largely unaffected.

Abstract Accurate application of the longitudinal dispersion model requires that specially designed experimental studies are performed in the river reach under consideration. Such studies are usually very expensive, so in order to quantify the longitudinal dispersion coefficient, as an alternative approach, various researchers have proposed numerous empirical formulae based on hydraulic and morphometric characteristics. The results are presented of the application of artificial neural networks as a parameter estimation technique. Five different cases were considered with the network trained for different arrangements of input nodes, such as channel depth, channel width, cross-sectionally averaged water velocity, shear velocity and sinuosity index. In the case where the sinuosity index is included as an input node, the results turned out to be better than those presented by other authors.