Search

This paper focuses on deriving and validating the fractional-order form of rectified linear unit activation function and its linear and nonlinear variants. The linear variants include the leaky and parametric, whereas the nonlinear variants include the exponential, sigmoid-weighted, and Gaussian error functions. Besides, a standard formula has been created and used while developing the fractional form of linear variants. Moreover, different expansion series such as Maclaurin and Taylor have been used while designing the fractional version of nonlinear variants. A simulation study has been conducted to validate the performance of all the developed fractional activation functions utilizing a single and multilayer neural network model and to compare them with their conventional counterparts. In this simulation study, a neural network model has been created to predict the system-generated power of a Texas wind turbine. The performance has been evaluated by varying the activation function in the hidden and output layers with the developed functions for single and multilayer networks. [ABSTRACT FROM AUTHOR]

The increase in the concentration of geological gas emissions in the atmosphere and particularly the increase of methane is considered by the majority of the scientific community as the main cause of global climate change. The main reasons that place methane at the center of interest, lie in its high global warming potential (GWP) and its lifetime in the atmosphere. Anthropogenic processes, like engineering geology ones, highly affect the daily profile of gasses in the atmosphere. Should direct measures be taken to reduce emissions of methane, immediate global warming mitigation could be achieved. Due to its significance, methane has been monitored by many space missions over the years and as of 2017 by the Sentinel-5P mission. Considering the above, we conclude that monitoring and predicting future methane concentration based on past data is of vital importance for the course of climate change over the next decades. To that end, we introduce a method exploiting state-of-the-art recurrent neural networks (RNNs), which have been proven particularly effective in regression problems, such as time-series forecasting. Aligned with the green artificial intelligence (AI) initiative, the paper at hand investigates the ability of different RNN architectures to predict future methane concentration in the most active regions of Texas, Pennsylvania and West Virginia, by using Sentinel-5P methane data and focusing on computational and complexity efficiency. We conduct several empirical studies and utilize the obtained results to conclude the most effective architecture for the specific use case, establishing a competitive prediction performance that reaches up to a 0.7578 mean squared error on the evaluation set. Yet, taking into consideration the overall efficiency of the investigated models, we conclude that the exploitation of RNN architectures with less number of layers and a restricted number of units, i.e., one recurrent layer with 8 neurons, is able to better compensate for competitive prediction performance, meanwhile sustaining lower computational complexity and execution time. Finally, we compare RNN models against deep neural networks along with the well-established support vector regression, clearly highlighting the supremacy of the recurrent ones, as well as discuss future extensions of the introduced work. [ABSTRACT FROM AUTHOR]

This article focuses on developing models that estimate suspended sediment concentrations (SSCs) for the Lower Brazos River, Texas, U.S. Historical samples of SSCs from gauge stations and satellite imagery from Landsat Missions and Sentinel Mission 2 were utilized to develop models to estimate SSCs for the Lower Brazos River. The models used in this study to accomplish this goal include support vector machines (SVMs), artificial neural networks (ANNs), extreme learning machines (ELMs), and exponential relationships. In addition, flow measurements were used to develop rating curves to estimate SSCs for the Brazos River as a baseline comparison of the models that used satellite imagery to estimate SSCs. The models were evaluated using a Taylor Diagram analysis on the test data set developed for the Brazos River data. Fifteen of the models developed using satellite imagery as inputs performed with a coefficient of determination R2 above 0.69, with the three best performing models having an R2 of 0.83 to 0.85. One of the best performing models was then utilized to estimate the SSCs before, during, and after Hurricane Harvey to evaluate the impact of this storm on the sediment dynamics along the Lower Brazos River and the model's ability to estimate SSCs. [ABSTRACT FROM AUTHOR]

Activation functions play an important role in deep learning models by introducing non-linearity to the output of a neuron, enabling the network to learn complex patterns and non-linear relationships in data and make predictions on more complex tasks. Deep learning models' most commonly used activation functions are Purelin, Sigmoid, Tansig, Rectified Linear Unit (ReLU), and Exponential Linear Unit (ELU), which exhibit limitations such as non-differentiability, vanishing gradients, and neuron inactivity with negative values. These functions are typically defined over a finite range, and their outputs are integers or real numbers. Using fractional calculus in designing activation functions for neural networks has shown promise in improving the performance of deep learning models in specific applications. These activation functions can capture more complex non-linearities than traditional integer-order activation functions, improving performance on tasks such as image classification and time series prediction. This paper focuses on deriving and testing linear and non-linear fractional-order forms of activation functions and their variants. The linear activation function includes Purelin. In contrast, the non-linear activation functions are Binary Step, Sigmoid, Tansig, ReLU, ELU, Gaussian Error Linear Unit (GELU), Hexpo, and their variants. Besides, the standard formula has been implemented and used in developing the fractional-order linear activation function. Furthermore, various expansion series, such as Euler and Maclaurin, have been used to design non-linear fractional-order activation functions and their variants. The single- and multi-layer fractional-order neural network models have been developed using the designed fractional-order activation functions. The simulation study uses developed fractional-order neural network models for predicting the Texas wind turbine systems' generated power. The performance of single and multi-layer fractional-order neural network models has been evaluated by changing the activation functions in the hidden layer while keeping the Purelin function constant at the output layer. Experiments on neural network models demonstrate that the designed fractional-order activation functions outperform traditional functions like Sigmoid, Tansig, ReLU, ELU, and their variants, effectively addressing limitations. • Activation functions are derived into fractional functions using fractional calculus. • The fractional neural network models have been made by adopting derived functions. • A Case study has been done on the Texas wind turbine with the models to predict the power. • The models' performance has been evaluated using the fractional functions. [ABSTRACT FROM AUTHOR]

Accurate estimation of reference evapotranspiration (ETr) is important for irrigation planning, water resource management, and preserving agricultural and forest habitats. The widely used Penman–Monteith equation (ASCE-PM) estimates ETr across various timescales using ground weather station data. However, discrepancies persist between estimated ETr and measured ETr obtained from weighing lysimeters (ETr-lys), particularly in advective environments. This study assessed different machine learning (ML) models in comparison to ASCE-PM for ETr estimation in highly advective conditions. Various variable combinations, representing both radiation and aerodynamic components, were organized for evaluation. Eleven datasets (DT) were created for the daily timescale, while seven were established for hourly and quarter-hourly timescales. ML models were optimized by a genetic algorithm (GA) and included support vector regression (GA-SVR), random forest (GA-RF), artificial neural networks (GA-ANN), and extreme learning machines (GA-ELM). Meteorological data and direct measurements of well-watered alfalfa grown under reference ET conditions obtained from weighing lysimeters and a nearby weather station in Bushland, Texas (1996–1998), were used for training and testing. Model performance was assessed using metrics such as root mean square error (RMSE), mean absolute error (MAE), mean bias error (MBE), and coefficient of determination (R2). ASCE-PM consistently underestimated alfalfa ET across all timescales (above 7.5 mm/day, 0.6 mm/h, and 0.2 mm/h daily, hourly, and quarter-hourly, respectively). On hourly and quarter-hourly timescales, datasets predominantly composed of radiation components or a blend of radiation and aerodynamic components demonstrated superior performance. Conversely, datasets primarily composed of aerodynamic components exhibited enhanced performance on a daily timescale. Overall, GA-ELM outperformed the other models and was thus recommended for ETr estimation at all timescales. The findings emphasize the significance of ML models in accurately estimating ETr across varying temporal resolutions, crucial for effective water management, water resources, and agricultural planning. [ABSTRACT FROM AUTHOR]

Among numerous functions performed by the battery management system (BMS), online estimation of the state of health (SOH) is an essential and challenging task to be accomplished periodically. In electric vehicle (EV) applications, accurate SOH estimation minimizes failure risk and improves reliability by predicting battery health conditions. The challenge of accurate estimation of SOH is based on the uncertain dynamic operating condition of the EVs and the complex nonlinear electrochemical characteristics exhibited by the lithium-ion battery. This paper presents an artificial neural network (ANN) classifier experimentally validated for the SOH estimation of lithium-ion batteries. The ANN-based classifier model is trained experimentally at room temperature under dynamic variable load conditions. Based on SOH characterization, the training is done using features such as the relative values of voltage, state of charge (SOC), state of energy (SOE) across a buffer, and the instantaneous states of SOC and SOE. At implementation, due to the slow dynamics of SOH, the algorithm is triggered on a large-scale periodicity to extract these features into buffers. The features are then applied as input to the trained model for SOH estimation. The classifier is validated experimentally under dynamic varying load, constant load, and step load conditions. The model accuracies for validation data are 96.2%, 96.6%, and 93.8% for the respective load conditions. It is further demonstrated that the model can be applied on multiple cell types of similar specifications with an accuracy of about 96.7%. The performance of the model analyzed with the confusion matrices is consistent with the requirements of the automotive industry. The classifier was tested on a Texas F28379D microcontroller unit (MCU) board. The result shows that an average real-time execution speed of 8.34 µs is possible with a negligible memory occupation. [ABSTRACT FROM AUTHOR]

Waterflooding is a widely used secondary oil recovery technique. The oil and gas industry uses a complex reservoir numerical simulation and reservoir engineering analysis to forecast production curves from waterflooding projects. The application of such standard methods at the stage of assessing the potential of a huge number of projects could be computationally inefficient and requires a lot of effort. This paper demonstrates the applicability of machine learning to rate the outcome of waterflooding applied to an oil reservoir. We also explore the relationship of project evaluations by operators at the final stages with several performance metrics for forecasting. Real data about several thousand waterflooding projects in Texas are used in the current study. We compare the ML models rankings of the waterflooding efficiency and the expert rankings. Linear regression models along with neural networks and gradient boosting on decision threes are considered. We show that machine learning models allow reducing computational complexity and can be useful for rating the reservoirs, with respect to the effectiveness of waterflooding. [ABSTRACT FROM AUTHOR]

Abudu, S., J.P. King, Z. Sheng, 2011. Comparison of the Performance of Statistical Models in Forecasting Monthly Total Dissolved Solids in the Rio Grande. Journal of the American Water Resources Association (JAWRA) 48(1): 10-23. DOI: 10.1111/j.1752-1688.2011.00587.x Abstract: This paper presents the application of autoregressive integrated moving average (ARIMA), transfer function-noise (TFN), and artificial neural networks (ANNs) modeling approaches in forecasting monthly total dissolved solids (TDS) of water in the Rio Grande at El Paso, Texas. Predictability analysis was performed between the precipitation, temperature, streamflow rates at the site, releases from upstream reservoirs, and monthly TDS using cross-correlation statistical tests. The chi-square test results indicated that the average monthly temperature and precipitation did not show significant predictability on monthly TDS series. The performances of one- to three-month-ahead model forecasts for the testing period of 1984-1994 showed that the TFN model that incorporated the streamflow rates at the site and Caballo Reservoir release improved monthly TDS forecasts slightly better than the ARIMA models. Except for one-month-ahead forecasts, the ANN models using the streamflow rates at the site as inputs resulted in no significant improvements over the TFN models at two-month-ahead and three-month-ahead forecasts. For three-month-ahead forecasts, the simple ARIMA showed similar performance compared to all other models. The results of this study suggested that simple deseasonalized ARIMA models could be used in one- to three-month-ahead TDS forecasting at the study site with a simple, explicit model structure and similar model performance as the TFN and ANN models for better water management in the Basin. [ABSTRACT FROM AUTHOR]

• Coupling of ANN waste forecasts with GIS collection route optimization. • GIS routes with minimum travel distances are sensitive to waste composition. • Results of 36 scenarios yield changes in travel distance of up to 19.9% • Dual compartment trucks reduce travel distance but increase waste collection time. Combining an artificial neural network (ANN) waste prediction model with a geographic information system (GIS) waste collection route optimization, the paper shows how the compositional features of waste materials affect the optimized truck route time, distance, and air emissions. Using data from Austin, Texas, USA, a nonlinear autoregressive ANN model is used to predict the waste generation rate of the recycling and garbage streams for the year 2023 in four sub-areas of the city. This ANN model resulted in mean absolute percentage errors ranging from 10.92% to 16.51%. Modified compositions of the recycling and garbage streams are then used as inputs, along with the year 2023 generation rates, to create 6 modified and 3 non-modified scenarios that reflect possible future changes in waste composition. These waste stream scenarios are then used as input parameters to determine optimal waste collection routes with minimal travel distance in each of the four sub-areas using the GIS vehicle routing problem network analysis tool. Results of these 36 scenarios yield changes in travel distance of up to 19.9%, when compared to the non-modified composition. Further, dual compartment trucks were compared to single compartment trucks and found to save between 10.3 and 16.0% in travel distance and slightly reduce emissions but had a 15.7–19.8% increase in collection time. Results suggest temporal changes in waste composition and characteristics are important in GIS route optimization studies. [ABSTRACT FROM AUTHOR]

This paper studies the advantages and disadvantages of different machine learning techniques in predicting ground-motion intensity measures given source characteristics, source-to-site distance, and local site conditions. Typically, linear regression-based models with predefined equations and coefficients are used in ground motion prediction. However, restrictions of the linear regression models may limit their capabilities in extracting complex nonlinear behaviors in the data. Therefore, the present paper comparatively investigates potential benefits from employing other machine learning techniques as statistical method in ground motion prediction such as Artificial Neural Network, Random Forest, and Support Vector Machine. This study quantifies event-to-event and site-to-site variability of the ground motions by implementing them as random effect terms to reduce the aleatory uncertainty. All the algorithms are trained using a selected database of 4528 ground-motions, including 376 seismic events with magnitude 3 to 5.8, recorded over the hypocentral distance range of 4–500 km in Oklahoma, Kansas, and Texas since 2005. The results indicate the algorithms satisfy some physically sound characteristics such as magnitude scaling distance dependency without requiring predefined equations or coefficients. Moreover, it is found that, when sufficient data is available, all the alternative algorithms tend to provide more accurate estimates compared to the conventional linear regression-based method, and particularly, Random Forest outperforms the other algorithms. However, the conventional method is a better tool when limited data is available. • ML-based framework is proposed to develop data-driven GMMs. • ML algorithms are adjusted to consider random effect parameters in GMMs. • ML-based GMMs provide higher accuracy than conventional methods of GMMs. • Physically sound features are observed in ML techniques. [ABSTRACT FROM AUTHOR]

The winter storm Uri that occurred in February 2021 affected many regions in Canada, the United States, and Mexico. The State of Texas was severely impacted due to the failure in the electricity supply infrastructure compounded by its limited connectivity to other grid systems in the United States. The georeferenced estimation of the storm's impact is crucial for response and recovery. However, such information was not available until several months afterward, mainly due to the time-consuming and costly assessment processes. The latency to provide timely information particularly impacted people in the economically disadvantaged communities, who lack resources to ameliorate the impact of the storm. This work explores the potential for disaster impact estimation based on the analysis of instant social media content, which can provide actionable information to assist first responders, volunteers, governments, and the general public. In our prototype, a deep neural network (DNN) uses geolocated social media content (texts, images, and videos) to provide monetary assessments of the damage at zip code level caused by Uri, achieving up to 70% accuracy. In addition, the performance analysis across geographical regions shows that the fully trained model is able to estimate the damage for economically disadvantaged regions, such as West Texas. Our methods have the potential to promote social equity by guiding the deployment or recovery resources to the regions where it is needed based on damage assessment. [ABSTRACT FROM AUTHOR]

Regional hurricane risk is often assessed assuming a static housing inventory, yet a region's housing inventory changes continually. Failing to include changes in the built environment in hurricane risk modeling can substantially underestimate expected losses. This study uses publicly available data and a long short-term memory (LSTM) neural network model to forecast the annual number of housing units for each of 1000 individual counties in the southeastern United States over the next 20 years. When evaluated using testing data, the estimated number of housing units was almost always (97.3 % of the time), no more than 1 percentage point different than the observed number, predictive errors that are acceptable for most practical purposes. Comparisons suggest the LSTM outperforms the autoregressive integrated moving average (ARIMA) and simpler linear trend models. The housing unit projections can help facilitate a quantification of changes in future expected losses and other impacts caused by hurricanes. For example, this study finds that if a hurricane with characteristics similar to Hurricane Harvey were to impact southeastern Texas in 20 years, the residential property and flood losses would be nearly USD 4 billion (38 %) greater due to the expected increase of 1.3 million new housing units (41 %) in the region. [ABSTRACT FROM AUTHOR]

An integrated space–time artificial neural network (ANN) model inspired by the governing groundwater flow equation was developed to test whether a single ANN is capable of modeling regional groundwater flow systems. Model-independent entropy measures and random forest (RF)-based feature selection procedures were used to identify suitable inputs for ANNs. L2 regularization, five-fold cross-validation, and an adaptive stochastic gradient descent (ADAM) algorithm led to a parsimonious ANN model for a 30 691 km 2 agriculturally intensive area in the Ogallala Aquifer of Texas. The model testing at 38 independent wells during the 1956–2008 calibration period showed no overfitting issues and highlighted the model's ability to capture both the observed spatial dependence and temporal variability. The forecasting period (2009–2015) was marked by extreme climate variability in the region and served to evaluate the extrapolation capabilities of the model. While ANN models are universal interpolators, the model was able to capture the general trends and provide groundwater level estimates that were better than using historical means. Model sensitivity analysis indicated that pumping was the most sensitive process. Incorporation of spatial variability was more critical than capturing temporal persistence. The use of the standardized precipitation–evapotranspiration index (SPEI) as a surrogate for pumping was generally adequate but was unable to capture the heterogeneous groundwater extraction preferences of farmers under extreme climate conditions. [ABSTRACT FROM AUTHOR]

With the increasing vulnerability of groundwater resources, especially in coastal regions, there is a growing need to monitor changes in groundwater storage (GWS). Estimations of GWS have been conducted extensively at regional to global scales using GRACE and GRACE-FO observations. The major goal of this study was to evaluate the applicability of uninterrupted monthly GRACE-derived terrestrial water storage (TWSGRACE) records in facilitating detection of long- and short-term hydroclimatic events affecting the GWS in a coastal area. The TWSGRACE data gap was filled with reconstructed values from multi-linear regression (MLR) and artificial neural network (ANN) models and used to estimate changes in GWS in the Texas coastal region (Gulf Coast and Carrizo–Wilcox Aquifers) between 2002 and 2019. The reconstructed TWSGRACE, along with soil moisture storage (SMS) from land surface models (LSMs), and surface water storage (SWS) were used to estimate the GRACE-derived GWS (GWSGRACE), validated against the GWS estimated from groundwater level observations (GWSwell) and extreme hydroclimatic event records. The results of this study show: (1) Good agreement between the predicted TWSGRACE data gaps from the MLR and ANN models with high accuracy of predictions; (2) good agreement between the GWSGRACE and GWSwell records (CC = 0.56, p-value < 0.01) for the 2011–2019 period for which continuous GWLwell data exists, thus validating the approach and increasing confidence in using the reconstructed TWSGRACE data to monitor coastal GWS; (3) a significant decline in the coastal GWSGRACE, at a rate of 0.35 ± 0.078 km3·yr−1 (p-value < 0.01), for the 2002–2019 period; and (4) the reliable applicability of GWSGRACE records in detecting multi-year drought and wet periods with good accuracy: Two drought periods were identified between 2005–2006 and 2010–2015, with significant respective depletion rates of −8.9 ± 0.95 km3·yr−1 and −2.67 ± 0.44 km3·yr−1 and one wet period between 2007 and 2010 with a significant increasing rate of 2.6 ± 0.63 km3·yr−1. Thus, this study provides a reliable approach to examine the long- and short-term trends in GWS in response to changing climate conditions with significant implications for water management practices and improved decision-making capabilities. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks (ANN) have been widely applied in hydrologic and water quality (H/WQ) modeling in the past three decades. Many studies have demonstrated an ANN's capability to successfully estimate daily streamflow from meteorological data on the watershed level. One major challenge of ANN streamflow modeling is finding the optimal network structure with good generalization capability while ameliorating model overfitting. This study empirically examines two types of model selection approaches for simulating streamflow time series: the out-of-sample approach using blocked cross-validation (BlockedCV) and an in-sample approach that is based on Akaike's information criterion (AIC) and Bayesian information criterion (BIC). A three-layer feed-forward neural network using a back-propagation algorithm is utilized to create the streamflow models in this study. The rainfall–streamflow relationship of two adjacent, small watersheds in the San Antonio region in south-central Texas are modeled on a daily time scale. The model selection results of the two approaches are compared, and some commonly used performance measures (PMs) are generated on the stand-alone testing datasets to evaluate the models selected by the two approaches. This study finds that, in general, the out-of-sample and in-sample approaches do not converge to the same model selection results, with AIC and BIC selecting simpler models than BlockedCV. The ANNs were found to have good performance in both study watersheds, with BlockedCV selected models having a Nash–Sutcliffe coefficient of efficiency (NSE) of 0.581 and 0.658, and AIC/BIC selected models having a poorer NSE of 0.574 and 0.310, for the two study watersheds. Overall, out-of-sample BlockedCV selected models with better predictive ability and is preferable to model streamflow time series. [ABSTRACT FROM AUTHOR]

A neural network, using input from the Eta Model and upper air soundings, has been developed for the probability of precipitation (POP) and quantitative precipitation forecast (QPF) for the Dallas-Fort Worth, Texas, area. Forecasts from two years were verified against a network of 36 rain gauges. The resulting forecasts were remarkably sharp, with over 70% of the PoP forecasts being less than 5% or greater than 95%. Of the 436 days with forecasts of less than 5% PoP, no rain occurred on 435 days. On the 111 days with forecasts of greater than 95% PoP, rain always occurred. The linear correlation between the forecast and observed precipitation amount was 0.95. Equitable threat scores for threshold precipitation amounts from 0.05 in. (∼1 mm) to 1 in. (∼25 mm) are 0.63 or higher, with maximum values over 0.86. Combining the PoP and QPF products indicates that for very high PoPs, the correlation between the QPF and observations is higher than for lower PoPs. In addition, 61 of the 70 observed rains of at least 0.5 in. (12.7 mm) are associated with PoPs greater than 85%. As a result, the system indicates a potential for more accurate precipitation forecasting. [ABSTRACT FROM AUTHOR]

Digital elevation models (DEMs) have become ubiquitous and remarkably effective in the field of earth sciences as a tool to characterize surface topography. All DEMs have a degree of inherent error and uncertainty that is propagated to subsequent models and analyses, which can lead to misinterpretation and inaccurate estimates. A new method was developed to estimate local DEM errors and implement corrections while quantifying the uncertainties of the implemented corrections. The method is based on the flexibility and ability to model complex problems with ensemble neural networks (ENNs). The method was developed to be applied to any DEM created from a corresponding set of elevation points (point cloud) and a set of ground truth measurements. The method was developed and tested using hyperspatial resolution terrestrial laser scanning (TLS) data (sub-centimeter point spacing) collected from a marsh site located along the southern portion of the Texas Gulf Coast, USA. ENNs improve the overall DEM accuracy in the study area by 68% for six model inputs and by 75% for 12 model inputs corresponding to root mean square errors (RMSEs) of 0.056 and 0.045 m, respectively. The 12-input model provides more accurate tolerance interval estimates, particularly for vegetated areas. The accuracy of the method is confirmed based on an independent data set. Although the method still underestimates the 95% tolerance interval, 8% below the 95% target, results show that it is able to quantify the spatial variability in uncertainties due to a relationship between vegetation/land cover and accuracy of the DEM for the study area. There are still opportunities and challenges in improving and confirming the applicability of this method for different study sites and data sets. [ABSTRACT FROM AUTHOR]