Search

This manuscript presents the design of a deep differential neural network (DDNN) for pattern classification. First, we proposed a DDNN topology with three layers, whose learning laws are derived from a Lyapunov analysis, justifying local asymptotic convergence of the classification error and the weights of the DDNN. Then, an extension to include an arbitrary number of hidden layers in the DDNN is analyzed. The learning laws for this general form of the DDNN offer a contribution to the deep learning framework for signal classification with biological nature and dynamic structures. The DDNN is used to classify electroencephalographic signals from volunteers that perform an identification graphical test. The classification results show exponential growth in the signal classification accuracy from 82 percent with one layer to 100 percent with three hidden layers. Working with DDNN instead of static deep neural networks (SDNN) represents a set of advantages, such as processing time and training period reduction up to almost 100 times, and the increment of the classification accuracy while working with less hidden layers than working with SDNN, which are highly dependent on their topology and the number of neurons in each layer. The DDNN employed fewer neurons due to the induced feedback characteristic. [ABSTRACT FROM AUTHOR]

This study aimed to develop a semi non-parametric model of the die casting process of aluminum alloys. This model uses a hierarchical artificial neural network (HANN), with a structure motivated by the relationships of the metals which define the characteristics of the aluminum alloy. These settings depend on the content of seven metals (Sn, Zn, Mn, Cu, Si, Ni, and Mg). The relation between these metals and the alloy characteristics oriented the HANN structure. A distributed back-propagation learning modified with the Levenberg-Marquardt method served to adjust the HANN weights. Two complementary validation methods justified the application of this novel hybrid non-parametric modelling structure. The training set came from standards composition proposed by different international organizations. A set of real aluminum alloys and the experimental results describing their characteristics formed the validation test. An average accuracy value of 3.65% confirmed the ability of the HANN to reproduce the relation between the metal content and the alloy characteristics. These values confirmed how the oriented HANN may predict the aluminum alloy characteristics as function of the metal distribution. This result offers a different alternative to the prediction of aluminum alloy properties using the metal composition as input information. [ABSTRACT FROM AUTHOR]

The aim of this study was to develop an adaptive state estimator with discontinuous parameter adjustment law for the catalytic ozonation system. A nonlinear transformation defined an equivalent system presented in chain-of-integrators form with uncertain structure in the dynamics of the last state. A step-by-step state estimator using a sequence of super-twisting algorithms (STAs) estimated the unmeasured states of the uncertain system. A class of differential neural network (DNN) with discontinuous learning law served to estimate the uncertain section of the catalytic process. The learning method was developed by implementing a strong lower-semi-continuous Lyapunov function. The method used to generate the laws that adjusted the weights, also yields the estimation of the parameters included in the catalytic ozonation system. A set of numerical simulations demonstrated the application of the DNN-based state observer to solve the estimation of the non-measurable information in the catalytic ozonation system. The available output signal was the concentration of the ozone gas at the output of the reactor. This was the only information used by the observer. The state estimator with discontinuous learning laws was also evaluated with experimental information obtained by a catalytic ozonation system using NiO as catalyst and phtalic acid as model contaminant. The effect of aggregating the DNN in the observer structure was compared with the observer using only the sequence of STA. The superior performance of the observer developed in this study was confirmed by evaluating the mean square error of the identification error. [ABSTRACT FROM AUTHOR]

Most of the previous work on identification involves systems described by ordinary differential equations (ODEs). Many industrial processes and physical phenomena, however, should be modeled using partial differential equations (PDEs) which offer both spatial and temporal distributions that are simply not available with ODE models. Systems described by a PDE belong to a class of system called distributed parameter system (DPS). This article presents a method for solving the problem of identification of uncertain DPSs using a differential neural network (DNN). The DPS, assumed to be described by a PDE, is approximated using the finite element method (FEM). The FEM discretizes the domain into a set of distributed and connected nodes, thereby, allowing a representation of the DPS in a finite number of ODEs. The proposed DNN follows the same interconnection structure of the FEM, thus allowing the DNN to identify the FEM approximation of the DPS in both 2D and 3D domains. Lyapunov's second method was used to derive adaptive learning laws for the proposed DNN structure. The identification algorithm, here developed in Nvidia's CUDA/C to reduce the execution time, runs mostly on the graphics processing unit (GPU). A physical experiment served to validate the 2D case. In the experiment, the DNN followed the trajectory of 57 markers that were placed on an undulating square piece of silk. The proposed DNN is compared against a method based on principal component analysis and an artificial neural network trained with group search optimization. In addition to the 2D case, a simulation validated the 3D case, where input data for the DNN was generated by solving a PDE with appropriate initial and boundary conditions over an unitary domain. Results show that the proposed FEM-based DNN approximates the dynamic behavior of both a real 2D and a simulated 3D system. [ABSTRACT FROM AUTHOR]

Recently, the study of dynamic systems and signals in the frequency domain motivates the emergence of new tools. In particular, electrophysiological and communications signals in the complex domain can be analyzed but hardly, they can be modeled. This problem promotes an attractive field of researching in system theory. As a consequence, adaptive algorithms like neural networks are interesting tools to deal with the identification problem of this kind of systems. In this study, a new learning process for recurrent neural network applied on complex-valued discrete-time nonlinear systems is proposed. The Lyapunov stability framework is applied to obtain the corresponding learning laws by means of the so-called Lyapunov control functions. The region where the identification error converges is defined by the power of uncertainties and perturbations that affects the nonlinear discrete-time complex system. This zone is obtained as an alternative result of the same Lyapunov analysis. An off-line training algorithm is derived in order to reduce the size of the convergence zone. The training is executed using a set of some off-line measurements coming from the uncertain system. Numerical results are developed to prove the efficiency of the methodology proposed in this study. A first example is oriented to identify the dynamics of a nonlinear discrete time complex-valued system and the second one to model the dynamics of an electrophysiological signal separated in magnitude and phase. [ABSTRACT FROM AUTHOR]

Abstract: In nature, most systems show nonlinear complex behaviors. Among other characteristics, plants present a high degree of oscillation over time. Adaptive algorithms used to approximate such difficult behaviors show some important deficiencies. Many adaptive non-parametric methods cannot reconstruct the trajectories of such complex dynamics. Differential neural networks (DNNs) are no exception. When just one DNN is applied to achieve an approximation, the identification error may significantly differ from zero. A natural trick to overcome this difficulty is to increase the number of neurons or to increase the number of layers. Another possible suggestion is to define a set of neural networks working together (usually in parallel). The members of such a set each work on well-defined trajectories contained in specific subspaces in which the uncertain system may evolve. Nevertheless, a decision system is required to define the contribution of each DNN in the final identification scheme. One of the most successful methodologies for constructing this selector is based on a Takagi–Sugeno (TS) inference system. This paper discusses how to combine the identification properties offered by a continuous neural network and the characteristic decision capabilities of fuzzy methods. The selection of which neural network is activated depends on the decision achieved by a TS fuzzy system. The convergence of this algorithm is proved using a quadratic Lyapunov function. A complete description of the learning laws used for the set of DNN identifiers is also obtained. The Chen circuit and the Rabinovich–Fabrikant system are used to demonstrate the superior performance achieved by this mixed DNN and fuzzy system, usually called a neuro-fuzzy system. [Copyright &y& Elsevier]

There are many examples in science and engineering which are reduced to a set of partial differential equations (PDEs) through a process of mathematical modelling. Nevertheless there exist many sources of uncertainties around the aforementioned mathematical representation. Moreover, to find exact solutions of those PDEs is not a trivial task especially if the PDE is described in two or more dimensions. It is well known that neural networks can approximate a large set of continuous functions defined on a compact set to an arbitrary accuracy. In this article, a strategy based on the differential neural network (DNN) for the non-parametric identification of a mathematical model described by a class of two-dimensional (2D) PDEs is proposed. The adaptive laws for weights ensure the ‘practical stability’ of the DNN-trajectories to the parabolic 2D-PDE states. To verify the qualitative behaviour of the suggested methodology, here a non-parametric modelling problem for a distributed parameter plant is analysed. [ABSTRACT FROM AUTHOR]

Abstract: In the present paper a mathematical model of a gas–gas reaction between ozone and benzene in a tubular reactor is considered. Usually, mathematical models of chemical process are governed by a set of ordinary differential equations assuming that the corresponding concentration dynamics depends only on time. On the other hand, the spatial distribution of the mass, energy and concentrations may be observed in the case of a more complex model structure that demands the use of models described by partial differential equations. The example of such complex model describing, the reaction between benzene and ozone in the gas phase, is considered here. The approach suggested in this study is based on the differential neural network (DNN) technique which permits to convert the task of mathematical modeling of a tubular reactor containing an uncertain (not well-defined) dynamics to a non-parametric identification problem. The asymptotic convergence of the obtained identification error to an ellipsoidal zone containing the origin is shown using the Lyapunov-like analysis. The coincidence between the benzene and ozone concentrations variation calculated by the suggested DNN-algorithm and those generated by a kinetic model is shown to be good enough. [Copyright &y& Elsevier]

The most important aspect of differential neural networks dynamics is related to their weights properties. This is a consequence of the complex non-linear structure describing the learning matrix differential equations, which are associated with adaptive capability of this kind of neural network. So far, there is no analytical demonstration about the weights stability. In fact, this is the main inconvenience in designing real applications for differential neural network observers. This study deals with the stability proof for the weights dynamics using an adaptive procedure to adjust the weights ordinary differential equations. Three different examples (two of them were realised by numerical simulations and the last one was carried out using real biofiltering process data) demonstrated the good performance of the suggested approach. [ABSTRACT FROM AUTHOR]

Abstract: Biofiltration is an economical and environmentally friendly process to eliminate air pollutants. Results obtained by different authors showed the enhanced performance of the fungal biofiltering systems. Consequently, there is a necessity to develop methodologies not only to design more efficient reactors but to control the reaction behavior under different conditions: pollutants feeding, air flows, humidity and biomass production. In this study, a continuous neural network observer was designed to predict the toluene vapors elimination capacity (EC) in a fungal biofilter. The observer uses the carbon dioxide (CO2) production and the pressure drop (DP) (on line measurements) as input information. The differential neural network observer proved to be a useful tool to reconstruct the immeasurable on-line variable (EC). The observer was successfully tested under different reaction conditions proving the robustness of estimation process. This software sensor may be helpful to derive adaptive control functions optimizing the biofilter reaction development. [Copyright &y& Elsevier]

A dynamic neural network is applied to estimate the state of the “phenol-water-ozone” chemical system. A new method based on dynamic neural observers with sliding mode (signum) term is used to estimate the dynamics of decomposition of phenols by ozone and to identify their kinetic parameters without the use of any process model. Decomposition of phenols and their mixtures by ozone in a semi-batch reactor is regarded as a dynamic process with an uncertain model (“black box”). Only the content of gaseous ozone is measured at the reactor output during ozonization. Variations of this variable are used to construct a total characteristic curve of the ozonization process. A dynamic state observer is used to estimate the phenol ozonization constant at different pH values from 2 to 12. Experimental data on decomposition dynamics are in good agreement with their estimates. Our method is helpful in on-line monitoring of water purification process without the use of special chemical sensors. [ABSTRACT FROM AUTHOR]

• The paper presents four recurrent and differential artificial neural networks (ANN) structures to construct different versions of dynamic automatic pattern classifiers. • Two different annotated and validated databases of diverse physiological signals were used to evaluate the capacities of all the ANNs proposed in this study. • Two validation methods were used to justify the application of dynamic ANNs as pattern classifiers: generalization-regularization and k-fold cross validation. • The recurrent neural network was also implemented in a 32-bits microcontroller embedded device. In the last few years, recurrent and continuous algorithms have became key factors in the solution of diverse pattern recognition problems. The main goal of this study is to introduce four classes of recurrent and continuous artificial neural networks (ANN) that can be implemented for pattern recognition of electrophysiological signals. Such networks are generally known as dynamic neural networks (DNN). The proposed DNN based pattern recognizer uses biosignals raw data as input. This processing method allows capturing the signal time dynamics, which is considered as an intrinsic characteristic of physiology signals. Therefore, recurrent and differential ANN structures were developed to construct different versions of dynamic automatic pattern recognizer. The first one describes the application of Recurrent Neural Networks (RNN) to enforce the biosignal analysis which evolves over time with a fixed sampling period. Three different DNNs with continuous dynamics are introduced. Differential neural network (DifNN) with the capability of learning the evolution of the signal in continuous time, a time-delay neural network (TDNN) for classification is implemented to consider the time-delayed characteristics of the electrophysiological signals and a complex valued neural network (CVNN) which considered the signals to be classified may be pre-processed with a frequency analysis technique. Two different databases of diverse physiological signals are used in this study to validate the application of dynamic neural networks. A first database considers electromiographic (EMG) signals which are tested using the DifNN, TDNN and CVNN. The second database includes gait in Parkinson's disease database signals which are used in the evaluation procedure of RNN. Two validation methods are used to justify the application of dynamic ANNs as pattern recognizer for the EMG activities and the health level classification of patients suffering from Parkinson's: generalization-regularization and the k -fold cross validation. The accuracy estimation and the confusion matrix evaluation confirm the superiority of the proposed approach compared to classical feed-forward ANN pattern recognizer. The particular case of the RNN is also implemented in a 32-bits micro-controller embedded device. [ABSTRACT FROM AUTHOR]

This study reports the design and implementation of a pattern recognition algorithm aimed to classify electroencephalographic (EEG) signals based on a class of dynamic neural networks (NN) described by time delay differential equations (TDNN). This kind of NN introduces the signal windowing process used in different pattern classification methods. The development of the classifier included a new set of learning laws that considered the impact of delayed information on the classifier structure. Both, the training and the validation processes were completely designed and evaluated in this study. The training method for this kind of NN was obtained by applying the Lyapunov theory stability analysis. The accuracy of training process was characterized in terms of the number of delays. A parallel structure (similar to an associative memory) with fixed (obtained after training) weights was used to execute the validation stage. Two methods were considered to validate the pattern classification method: a generalization-regularization and the k -fold cross validation processes ( k = 5). Two different classes were considered: normal EEG and patients with previous confirmed neurological diagnosis. The first one contains the EEG signals from 100 healthy patients while the second contains information of epileptic seizures from the same number of patients. The pattern classification algorithm achieved a correct classification percentage of 92.12% using the information of the entire database. In comparison with similar pattern classification methods that considered the same database, the proposed CNN proved to achieve the same or even better correct classification results without pre-treating the EEG raw signal. This new type of classifier working in continuous time but using the delayed information of the input seems to be a reliable option to develop an accurate classification of windowed EEG signals. [ABSTRACT FROM AUTHOR]

This study reports the design and implementation of a pattern recognition algorithm to classify electroencephalographic (EEG) signals based on artificial neural networks (NN) described by ordinary differential equations (ODEs). The training method for this kind of continuous NN (CNN) was developed according to the Lyapunov theory stability analysis. A parallel structure with fixed weights was proposed to perform the classification stage. The pattern recognition efficiency was validated by two methods, a generalization–regularization and a k -fold cross validation ( k = 5 ). The classifier was applied on two different databases. The first one was made up by signals collected from patients suffering of epilepsy and it is divided in five different classes. The second database was made up by 90 single EEG trials, divided in three classes. Each class corresponds to a different visual evoked potential. The pattern recognition algorithm achieved a maximum correct classification percentage of 97.2% using the information of the entire database. This value was similar to some results previously reported when this database was used for testing pattern classification. However, these results were obtained when only two classes were considered for the testing. The result reported in this study used the whole set of signals (five different classes). In comparison with similar pattern recognition methods that even considered less number of classes, the proposed CNN proved to achieve the same or even better correct classification results. [ABSTRACT FROM AUTHOR]

Time-delay systems have been successfully used to represent the complexity of some dynamic systems. Time-delay is often used for modeling many real systems. Among others, biological and chemical plants have been described using time-delay terms with better results than those models that have not consider them. However, getting those models represented a challenge and sometimes the results were not so satisfactory. Non-parametric modeling offered an alternative to obtain suitable and usable models. Continuous neural networks (CNN) have been considered as a real alternative to provide models over uncertain non-parametric systems. This article introduces the design of a specific class of non-parametric model for uncertain time-delay system based on CNN considering the so-called delayed learning laws analysis. The convergence analysis as well as the learning laws were produced by means of a Lyapunov–Krasovskii functional. Three examples were developed to demonstrate the effectiveness of the modeling process forced by the identifier proposed in this study. The first example was a simple nonlinear model used as benchmark example. The second example regarded the human immunodeficiency virus dynamic behavior is used to show the performance of the suggested non-parametric identifier based on CNN for no fictitious neither academic models. Finally, a third example describing the evolution of hepatitis B virus served to test the identifier presented in this study and was also useful to provide evidence of its superior performance against a non-delayed identifier based on CNN. [ABSTRACT FROM AUTHOR]

Sequential processes appear naturally in all types of industries. Biotechnology is a good example of such schemes. Wastewater treatment using microbiological activity is a particular case having all the characteristics of sequential methods. Sulfate reduction as pre-treatment followed by the decomposition of sulfated compounds using adapted microorganisms is the sequential nonlinear process with state constrains analyzed in this paper. Modeling this procedure is still a difficult task because the number of elements involved in the reaction. This paper presents an adaptive algorithm to obtain a suitable model of this process using continuous neural networks. The adaptive model preserves the sequential nature of the process as well as the bounded nature of all states. The neural network is proposed as a system identifier in terms of the hybrid systems theory. The Lyapunov stability method is used to demonstrate the convergence of the identifier states to the real concentrations of the microbiological system. Experimental results and their corresponding simulation using the adaptive model based on neural networks confirm the theoretical results described in this paper. [ABSTRACT FROM AUTHOR]