Search

A fitness landscape analysis of the loss surfaces produced by product unit neural networks is performed in order to gain a better understanding of the impact of product units on the characteristics of the loss surfaces. The loss surface characteristics of product unit neural networks are then compared to the characteristics of loss surfaces produced by neural networks that make use of summation units. The failure of certain optimization algorithms in training product neural networks is explained through trends observed between loss surface characteristics and optimization algorithm performance. The paper shows that the loss surfaces of product unit neural networks have extremely large gradients with many deep ravines and valleys, which explains why gradient-based optimization algorithms fail at training these neural networks. [ABSTRACT FROM AUTHOR]

A fitness landscape analysis of the loss surfaces produced by product unit neural networks is performed in order to gain a better understanding of the impact of product units on the characteristics of the loss surfaces. The loss surface characteristics of product unit neural networks are then compared to the characteristics of loss surfaces produced by neural networks that make use of summation units. The failure of certain optimization algorithms in training product neural networks is explained through trends observed between loss surface characteristics and optimization algorithm performance. The paper shows that the loss surfaces of product unit neural networks have extremely large gradients with many deep ravines and valleys, which explains why gradient-based optimization algorithms fail at training these neural networks.

The properties of biodiesel differ depending on feedstock fatty acid content. Moreover, biodiesel fatty acid composition influences the combustion process. For these reasons, noise emissions of a direct injection Perkins diesel engine fueled with olive pomace oil methyl ester (monounsaturated methyl esters) and palm oil methyl ester (saturated methyl esters) were studied under several steady-state engine operating conditions. In this work, different approaches for sound prediction of the engine based on Neural Network (NN) models, such as Product Unit NN (PUNN), Radial Basis Function NN (RBFNN) and response surface models have been proposed. Error was measured considering Mean Square Error (MSE) and Standard Error of Prediction (SEP). It can be concluded that the use of a hybrid model combining PU and RBF improves noise prediction accuracy, providing an acceptable value of both MSE and SEP when monounsaturated methyl ester/diesel fuel blends are used. However, best results for saturated methyl ester/diesel fuel blends were achieved by PUNN model. Whereas taking into account the simplicity of the model, PUNN model is the most appropriate for both monounsaturated and saturated methyl ester/diesel fuel blends. Response surface models have shown worse results based on the coefficient of correlation. Also, the effect of independent variables in the models has been studied and an inverse relationship between frequency and engine noise has been found. [ABSTRACT FROM AUTHOR]

Time series forecasting (TSF) consists on estimating models to predict future values based on previously observed values of time series, and it can be applied to solve many real-world problems. TSF has been traditionally tackled by considering autoregressive neural networks (ARNNs) or recurrent neural networks (RNNs), where hidden nodes are usually configured using additive activation functions, such as sigmoidal functions. ARNNs are based on a short-term memory of the time series in the form of lagged time series values used as inputs, while RNNs include a long-term memory structure. The objective of this paper is twofold. First, it explores the potential of multiplicative nodes for ARNNs, by considering product unit (PU) activation functions, motivated by the fact that PUs are specially useful for modelling highly correlated features, such as the lagged time series values used as inputs for ARNNs. Second, it proposes a new hybrid RNN model based on PUs, by estimating the PU outputs from the combination of a long-term reservoir and the short-term lagged time series values. A complete set of experiments with 29 data sets shows competitive performance for both model proposals, and a set of statistical tests confirms that they achieve the state of the art in TSF, with specially promising results for the proposed hybrid RNN. The experiments in this paper show that the recurrent model is very competitive for relatively large time series, where longer forecast horizons are required, while the autoregressive model is a good selection if the data set is small or if a low computational cost is needed. [ABSTRACT FROM AUTHOR]

In this paper we tackle the problem of massive missing data reconstruction in ocean buoys, with an evolutionary product unit neural network (EPUNN). When considering a large number of buoys to reconstruct missing data, it is sometimes difficult to find a common period of completeness (without missing data on it) in the data to form a proper training and test set. In this paper we solve this issue by using partial reconstruction, which are then used as inputs of the EPUNN, with linear models. Missing data reconstruction in several phases or steps is then proposed. In this work we also show the potential of EPUNN to obtain simple, interpretable models in spite of the non-linear characteristic of the neural network, much simpler than the commonly used sigmoid-based neural systems. In the experimental section of the paper we show the performance of the proposed approach in a real case of massive missing data reconstruction in 6 wave-rider buoys at the Gulf of Alaska. [ABSTRACT FROM AUTHOR]

Abstract: In order to ensure the success of new product developments and to study different alternatives of designs before their manufacture, it is primordial to assess identification models. This practice is an extensive one in the automotive industry. Automotive manufacturers invest a lot of effort and money to improve the vibro-acoustics performance of their products because they have to comply with the noise emission standards. International standards, commonly known as pass-by and coast-by noise test, define a procedure for measuring vehicle noise at different receptor positions. The aim of this work is to develop a novel model which can be used in pass-by noise test in vehicles based on ensembles of hybrid evolutionary product unit or radial basis function neural networks (EPUNNs or ERBFNNs) at high frequencies. Statistical models and ensembles of hybrid EPUNN and ERBFNN approaches have been used to develop different noise identification models. The results obtained using different ensembles of hybrid EPUNNs and ERBFNNs show that the functional model and the hybrid algorithms proposed provide a very accurate identification compared to other statistical methodologies used to solve this regression problem. [Copyright &y& Elsevier]

Abstract: Auralization through binaural transfer path analysis and synthesis is a useful tool to analyze how contributions from different sources affect the perception of sound. This paper presents a novel model based on the auralization of sound sources through the study of the behavior of the system with respect to frequency. The proposed approach is a combined model using the airborne source quantification (ASQ) technique for low-mid frequencies (⩽2.5kHz) and Evolutionary Product-Unit Neural Networks (EPUNNs) for high frequencies (>2.5kHz), which improve overall accuracy. The accuracy of all models has been evaluated in terms of the Mean Squared Error (MSE) and the Standard Error of Prediction (SEP), the combined model obtaining the smallest value for high frequencies. Moreover, the best prediction model was established based on sound quality metrics, the proposed method showing better accuracy than the ASQ technique at high frequencies in terms of loudness, sharpness and 1/3rd octave bands. [Copyright &y& Elsevier]

Abstract: Two of the major challenges of the European Union policy are to reduce both greenhouse gas (GHG) emissions and the dependence on fossil-based fuels. Biodiesel, produced through the transesterification of vegetable oils or animal fats, seems to be an excellent support nowadays, new raw materials to produce biodiesel are under study, provided that the production of biodiesel from edible oils is controversial for some social organizations. Olive pomace oil derives from the oil left in the olive fruit pulp (once olive oil has been extracted) and may be an option to considerer since it exhibits high neutral flavor that makes it undesirable for consumption, unless it is treated and further blended with virgin olive oil. Therefore, other uses for this oil could be soap production or its recycling to produce biodiesel. In this paper, noise emissions of a direct injection diesel engine Perkins fueled with olive pomace oil methyl ester (OPME) at several steady-state engine operating conditions were studied. This study proposes the use of different approaches for sound prediction of a diesel engine, based on artificial neural network (such as Evolutionary Product Unit Neural Networks (PUNNs) and Radial Basic Function Neural Networks (RBFNNs)) and response surface models. Their accuracies are compared in terms of Mean Square Error (MSE) and Standard Error of Prediction (SEP). It can be concluded that the use of PUNN improves the accuracy achieving acceptable values for both MSE and SEP by means of a simpler model than the combined PU and RBF NN proposed model. Moreover, it was found that the variable power does not explain the noise value prediction, the noise emitted by the engine is inversely related to the 1/3rd octave band of the frequency value and diesel fuel noise plays the most important role and influence in the PUNN model. Response surface models are rejected, due to their unacceptable accuracy in terms of noise prediction. [Copyright &y& Elsevier]

Combinations of physical and statistical wind speed forecasting models are frequently used in wind speed prediction problems arising in wind farms management. Artificial neural networks can be used in these models as a final step to obtain accurate wind speed predictions. The aim of this work is to determine the potential of evolutionary product unit neural networks (EPUNNs) for improving the accuracy and interpretation of these systems. Traditional neural network and EPUNN approaches have been used to develop different wind speed prediction models. The results obtained using different EPUNN models show that the functional model and the hybrid algorithms proposed provide very accurate prediction compared with standard neural networks used to solve this regression problem. One of the main advantages of the application of these EPUNNs has been the possibility of obtaining some interpretation of the non-linear relation predicted by the model, as will be shown in real data of a wind farm in Spain. [ABSTRACT FROM AUTHOR]

Abstract: This work investigates the ability of multilogistic regression models including nonlinear effects of the covariates as a multi-class pattern recognition technique to discriminate highly overlapping analytical signals using a very short number of input covariates. For this purpose, three methodologies recently reported by us were applied based on the combination of linear and nonlinear terms which are transformations of the linear ones by using evolutionary product unit neural networks. To test this approach, drinking water samples contaminated with volatile organic compounds such as benzene, toluene, xylene and their mixtures were classified in seven classes through the very close data provided by their headspace-mass spectrometric analysis. Instead of using the total ion current profile provided by the MS detector as input covariates, the three-parameter Gaussian curve associated to it was used as linear covariates for the standard multilogistic regression model, whereas the product unit basic functions or their combination with the linear covariates were used for the nonlinear models. The hybrid nonlinear model, pruned by a backward stepwise method, provided the best classification results with a correctly classified rate for the training and generalization sets of 100% and 76.2%, respectively. The reduced dimensions of the proposed model: only three terms, namely one initial covariate and two basis product units, enabled to infer interesting interpretations from a chemical point of view. [Copyright &y& Elsevier]

Abstract: A new approach to predict the growth/no growth interface of Listeria monocytogenes as a function of storage temperature, pH, citric acid (CA) and ascorbic acid (AA) is presented. A linear logistic regression procedure was performed and a non-linear model was obtained by adding new variables by means of a Neural Network model based on Product Units (PUNN). The classification efficiency of the training data set and the generalization data of the new Logistic Regression PUNN model (LRPU) were compared with Linear Logistic Regression (LLR) and Polynomial Logistic Regression (PLR) models. 92% of the total cases from the LRPU model were correctly classified, an improvement on the percentage obtained using the PLR model (90%) and significantly higher than the results obtained with the LLR model, 80%. On the other hand predictions of LRPU were closer to data observed which permits to design proper formulations in minimally processed foods. This novel methodology can be applied to predictive microbiology for describing growth/no growth interface of food-borne microorganisms such as L. monocytogenes. The optimal balance is trying to find models with an acceptable interpretation capacity and with good ability to fit the data on the boundaries of variable range. The results obtained conclude that these kinds of models might well be very a valuable tool for mathematical modeling. [Copyright &y& Elsevier]

Abstract: The potential of the product unit neural networks built by using different memetic evolutionary algorithms for the simultaneous determination of mixtures of analytes based on dynamic responses was investigated. For this purpose, three methodologies for obtaining the structure and weights of neural networks were proposed, based on the combination of the evolutionary programming algorithm, a clustering process and a local improvement procedure carried out by the Levenberg–Marquardt algorithm. To test these approaches, two phenol derivatives, pyrogallol and gallic acid, were quantified in mixtures based on their perturbation effect in a classical oscillating chemical system, namely, the Belousov–Zhabotinskyi reaction. The four-parameter Weibull curve associated with the profile of perturbation response estimated by the Levenberg–Marquardt method was used as input data for the models. Straightforward network topologies 4:3:1 (13 weights) and 4:2:1 (9 weights) for pyrogallol and gallic acid, respectively, allowed the analytes to be quantified with great accuracy (mean standard error of prediction for the generalization test) and precision (standard deviation) of 2.45% and 0.21 for pyrogallol and 7.61% and 1.63 for gallic acid. The selected model can be easily implemented via software by using simple quantification equations, from which significant chemical remarks can be inferred. Finally, the product unit neural network modelling offered better results when compared with sigmoidal neural networks and response surface analysis. [Copyright &y& Elsevier]

• Product unit neural networks (PUNN) are implemented for stream temperature modelling. • Stream temperature modelling is based on atmospheric and hydrological interactions. • Deep learning-based dropout variant is proposed for shallow PUNN. • Impact of various dropout variants is tested for PUNN. • PUNN with proposed dropout outperforms data-based and semi-physical models. For about two decades neural networks are widely used for river temperature modelling. However, in recent years one has to distinguish between the "classical" shallow neural networks, and deep learning networks. The applicability of rapidly developing deep learning networks to stream water temperature modelling may be limited, but some methods developed for deep learning, if properly re-considered, may efficiently improve performance of shallow networks. Dropout is widely considered the method that allows deep learning networks to avoid overfitting to training data, facilitating its implementations to versatile problems. Recently the successful applicability of dropout for river temperature modelling by means of shallow multilayer perceptron neural networks has been introduced. In the present study we propose to use dropout solely for input neurons of product unit neural networks for the purpose of stream temperature modelling. We perform tests on data collected from six catchments located in temperate climate zones on two continents in various orographic conditions. We show that the average performance of product unit neural networks trained with input dropout is better than the average performance of product units without dropout, product units with dropout applied to every layer of the networks, multilayer perceptron neural networks with or without dropout, and the semi-physical air2stream model. The advantage of product unit neural networks with input dropout is statistically significant on hilly or mountainous catchments; the performance on flat ones is similar to the performances obtained from competitive models. [ABSTRACT FROM AUTHOR]