Search

We have evaluated the uncertainty analysis of the 3D electrical tomography inverse problem using model reduction via singular-value decomposition and performed sampling of the nonlinear equivalence region via an explorative member of the particle swarm optimization (PSO) family. The procedure begins with the local inversion of the observed data to find a good resistivity model located in the nonlinear equivalence region. Then, the dimensionality is reduced via the spectral decomposition of the 3D geophysical model. Finally, the exploration of the uncertainty space is performed via an exploratory version of PSO (RR-PSO). This sampling methodology does not prejudge where the initial model comes from as long as this model has a geologic meaning. The 3D subsurface conductivity distribution is arranged as a 2D matrix by ordering the conductivity values contained in a given earth section as a column array and stacking parallel sections as columns of the matrix. There are three basic modes of ordering: mode 1 and mode 2, by using vertical sections in two perpendicular directions, and mode 3, by using horizontal sections. The spectral decomposition is then performed using these three 2D modes. Using this approach, it is possible to sample the uncertainty space of the 3D electrical resistivity inverse problem very efficiently. This methodology is intrinsically parallelizable and could be run for different initial models simultaneously. We found the application to a synthetic data set that is well-known in the literature related to this subject, obtaining a set of surviving geophysical models located in the nonlinear equivalence region that can be used to approximate numerically the posterior distribution of the geophysical model parameters (frequentist approach). Based on these models, it is possible to perform the probabilistic segmentation of the inverse solution found, meanwhile answering geophysical questions with its corresponding uncertainty assessment. This methodology has a general character could be applied to any other 3D nonlinear inverse problems by implementing their corresponding forward model.

Linear and nonlinear discrete inverse problems are by genesis ill-posed, that is, several solutions exist compatible with the prior information and fit the observed data within the same error bounds (equivalent models). This fact and the presence of noise and modelling errors cause their solution to be uncertain. Uncertainty analysis and solution appraisal is a crucial step in inverse modelling, and it is typically accomplished via random sampling methodologies and Bayesian approaches (provided a good knowledge of the probabilistic distributions of the model and data spaces is at disposal) or via exploratory global optimization algorithms that perform an approximate posterior sampling of the nonlinear equivalence region. This paper demonstrates that a simple method to sample the uncertainty space in linear and nonlinear problems consists in performing different inversions of random data bags. This novel methodology is called data kit inversion and it is inspired in the bootstrapping technique in statistical inference, switching in this case to the existence of other equivalent models, instead of using in estimating the properties of a given estimator. We present its application to different synthetic linear and nonlinear inverse problems and also to the inversion of a real dataset of gravimetric inversion in the Atacama Desert and also to 1D seismic and AVO inversion. We show that using partial information (random data bags) it is possible to sample different models of the equivalence region (uncertainty space) in order to explore the existence of different plausible geophysical scenarios according to the geophysical model that it is adopted. This algorithm has a general character and can be used in different fields of science and technology .

Nonlinear high-dimensional optimization problems are generally ill-posed and ill-conditioned, with different sets of models located in one or different disconnected valleys of the cost function landscape with similar values. This situation generates uncertainty in the identification of the optimum model parameters that should be translated to the decision that has to be made. Therefore, the analysis of the uncertainty space of this type of problems is required in order to adopt robust decisions. This is done by sampling the cost function topography within the intricate solution set valleys that belong to the nonlinear equivalence region and validating the existence of different scenarios. This is also the case of the portfolio optimization problem, which admits multiple solutions depending on the expected return-risk ratio. As the popular wisdom says, you cannot have the butter and the money of having sold it. This mathematical situation is known as a Pareto front, which aims to show the boundary of the nonlinear equivalence region of the corresponding decision problem. In this chapter, we introduce the concept of uncertainty in high-dimensional problems, proposing the particle swarm optimization family as a parameter free-tuning global algorithm, capable of sampling the nonlinear equivalent region in parallel with the optimization. For that purpose, these algorithms should be exploratory. This feature is related to the automatic tuning of the PSO parameters in the neighborhood of the stochastic second-order stability region of the particle trajectories. These algorithms are faster than Monte Carlo methods. The chapter concludes with the application of PSO to the portfolio optimization problem of the IBEX-35, which is the main stock market index of the Bolsa de Madrid. In this case, the cost function is constructed in a way that the investors seek to maximize the portfolio’s expected return subjected to a given risk. The optimization of the portfolio composition follows a previous selection of the stocks.

The prediction of the dynamics of the COVID-19 outbreak and the corresponding needs of the health care system (COVID-19 patients’ admissions, the number of critically ill patients, need for intensive care units, etc.) is based on the combination of a limited growth model (Verhulst model) and a short-term predictive model that allows predictions to be made for the following day. In both cases, the uncertainty analysis of the prediction is performed, i.e., the set of equivalent models that adjust the historical data with the same accuracy. This set of models provides the posterior distribution of the parameters of the predictive model that adjusts the historical series. It can be extrapolated to the same analyzed time series (e.g., the number of infected individuals per day) or to another time series of interest to which it is correlated and used, e.g., to predict the number of patients admitted to urgent care units, the number of critically ill patients, or the total number of admissions, which are directly related to health needs. These models can be regionalized, that is, the predictions can be made at the local level if data are disaggregated. We show that the Verhulst and the Gompertz models provide similar results and can be also used to monitor and predict new outbreaks. However, the Verhulst model seems to be easier to interpret and to use.

Inverse problems are special kind of optimization problems where observed data (noise included) enters into the optimization cost function, having an important impact in the solution (geophysical model parameters) that has been found. Therefore, the most important step in inversion is uncertainty analysis and model appraisal. We revise the uncertainty concept in linear and nonlinear inverse problems through linear algebra and optimization concepts, relating them to the well-known Bayesian Approaches and maximum likelihood methods. We also provide very practical insights to deal with uncertainty in high-dimensional nonlinear inverse problems through the use of model reduction and forward model approximations. This approach is also valid for classification problems where the forward model is unknown and it has to be constructed. The aim of this paper is not to provide new original results but showing the researchers why uncertainty assessment is a compulsory step in inversion and how to deal with it in a simpler way.

Background Phenotype prediction problems are usually considered ill-posed, as the amount of samples is very limited with respect to the scrutinized genetic probes. This fact complicates the sampling of the defective genetic pathways due to the high number of possible discriminatory genetic networks involved. In this research, we outline three novel sampling algorithms utilized to identify, classify and characterize the defective pathways in phenotype prediction problems, such as the Fisher’s ratio sampler, the Holdout sampler and the Random sampler, and apply each one to the analysis of genetic pathways involved in tumor behavior and outcomes of triple negative breast cancers (TNBC). Altered biological pathways are identified using the most frequently sampled genes and are compared to those obtained via Bayesian Networks (BNs). Results Random, Fisher’s ratio and Holdout samplers were more accurate and robust than BNs, while providing comparable insights about disease genomics. Conclusions The three samplers tested are good alternatives to Bayesian Networks since they are less computationally demanding algorithms. Importantly, this analysis confirms the concept of “biological invariance” since the altered pathways should be independent of the sampling methodology and the classifier used for their inference. Nevertheless, still some modifications are needed in the Bayesian networks to be able to sample correctly the uncertainty space in phenotype prediction problems, since the probabilistic parameterization of the uncertainty space is not unique and the use of the optimum network might falsify the pathways analysis.

In this chapter, we show the application of deep neural networks (DNNs) in phenotype prediction problems for the interpretation of microarray data from disease patients and healthy controls to better understand the genetic pathways that are involved. These problems have a very underdetermined character since the number of control variables (monitored genetic probes) is always much larger than the number of observed data (control and disease samples). Therefore, the aim of these methods should be to perform the sampling of their uncertainty space, which is composed of the set of genetic signatures that predict the phenotype with similar predictive accuracy in order to establish the genetic pathways altered by the disease. The methodology presented in this chapter samples the uncertainty space of the phenotype prediction problem via DNN with a prior probability induced by the prior discriminatory power of each individual gene as measured by their Fisher’s ratio, established within the set of differentially expressed genes. The DNN serves to evaluate the posterior predictive accuracy of the genetic networks that are sampled. The high discriminatory genetic networks sampled in the uncertainty space serve to understand the causes of disease development and finding new therapeutic targets and performing drug repositioning. We show the application to a microarray dataset associated with a specific rare disease called inclusion body myositis (IBM).

Most of the geophysical inverse problems in geophysical exploration consist in detecting, locating and outlining the shape of geophysical anomalous bodies imbedded into a quasi-homogeneous background by analyzing their effect in the geophysical signature. The inversion algorithm that is currently used creates a very fine mesh in the model space to approximate the shapes and the values of the anomalous bodies and the geophysical structure of the geological background. This approach results in discrete inverse problems with a huge uncertainty space, and the common way of stabilizing the inversion consists in introducing a reference model (through prior information) to define the set of correctness of geophysical models. We present a different way of dealing with the high underdetermined character of this kind of problems, consisting in solving the inverse problem using a low dimensional parameterization that provides an approximate solution of the anomaly via Particle Swarm Optimization (PSO). This methodology has been designed for anomaly detection in geological set-ups that correspond with this kind of problem. We show its application to synthetic and real cases in gravimetric inversion performing at the same time uncertainty analysis of the solution. We have compared two different parameterizations for the geophysical anomalies (polygons and ellipses), showing that we have obtained similar results. This methodology outperforms the common least squares method with regularization.

Accurate prediction of protein stability changes resulting from amino acid substitutions is of utmost importance in medicine to better understand which mutations are deleterious, leading to diseases, and which are neutral. Since conducting wet lab experiments to get a better understanding of protein mutations is costly and time consuming, and because of huge number of possible mutations the need of computational methods that could accurately predict effects of amino acid mutations is of greatest importance. In this research, we present a robust methodology to predict the energy changes of a proteins upon mutations. The proposed prediction scheme is based on two step algorithm that is a Holdout Random Sampler followed by a neural network model for regression. The Holdout Random Sampler is utilized to analysis the energy change, the corresponding uncertainty, and to obtain a set of admissible energy changes, expressed as a cumulative distribution function. These values are further utilized to train a simple neural network model that can predict the energy changes. Results were blindly tested (validated) against experimental energy changes, giving Pearson correlation coefficients of 0.66 for Single Point Mutations and 0.77 for Multiple Point Mutations. These results confirm the successfulness of our method, since it outperforms majority of previous studies in this field.

In this paper we present a methodology to perform geophysical inversion of large scale linear systems via a covariance-free orthogonal transformation: the Discrete Cosine Transform (DCT). The methodology consists in compressing the matrix of the linear system as a digital image and using the interesting properties of orthogonal transformations to define an approximation of the Moore-Penrose pseudo-inverse. This methodology is also highly scalable since the model reduction achieved by these techniques increases with the number of parameters of the linear system involved due to the high correlation needed for these parameters to accomplish very detailed forward predictions, and allows for a very fast computation of the inverse problem solution. We show the application of this methodology to a simple synthetic 2D gravimetric problem for different dimensionalities and different levels of white Gaussian noise, and to a synthetic linear system whose system matrix has been generated via geostatistical simulation to produce a random field with a given spatial correlation. The numerical results show that the DCT pseudoinverse outperforms the classical least-squares techniques, mainly in presence of noise, since the solutions that are obtained are more stable and fit the observed data with a lowest RMS error. Besides, we show that the model reduction is a very effective way of parameter regularization when the conditioning of the reduced DCT matrix is taken into account. We finally show its application to the inversion of a real gravity profile in the Atacama Desert (north Chile) obtaining very successful results in this nonlinear inverse problem. The methodology presented here has a general character and can be applied to solve any linear and nonlinear inverse problems (through linearization) arising in technology and particularly in geophysics, independently of the geophysical model discretization and dimensionality. Nevertheless, the results shown in this paper are better in the case of ill-conditioned inverse problems for which the matrix compression is more efficient. In that sense, a natural extension of this methodology would be its application to the set of normal equations. This article is protected by copyright. All rights reserved

In this paper, we present the uncertainty analysis of the 2D electrical tomography inverse problem using model reduction and performing the sampling via an explorative member of the Particle Swarm Optimization (PSO) family, called the Regressive-Regressive PSO (RR-PSO). The procedure begins with a local inversion to find a good resistivity model located in the nonlinear equivalence region of the set of plausible solutions. The dimension of this geophysical model is then reduced using spectral decomposition, and the uncertainty space is explored via PSO. Using this approach, we show that it is possible to sample the uncertainty space of the electrical tomography inverse problem. We illustrate this methodology with the application to a synthetic and a real dataset coming from a karstic geological set-up. By computing the uncertainty of the inverse solution, it is possible to perform the segmentation of the resistivity images issued from inversion. This segmentation is based on the set of equivalent models that have been sampled, and makes it possible to answer geophysical questions in a probabilistic way, performing risk analysis. This article is protected by copyright. All rights reserved

Introduction: Drug discovery is the process through which potential new compounds are identified by means of biology, chemistry, and pharmacology. Due to the high complexity of genomic data, AI techniques are increasingly needed to help reduce this and aid the adoption of optimal decisions. Phenotypic prediction is of particular use to drug discovery and precision medicine where sets of genes that predict a given phenotype are determined. Phenotypic prediction is an undetermined problem given that the number of monitored genetic probes markedly exceeds the number of collected samples (from patients). This imbalance creates ambiguity in the characterization of the biological pathways that are responsible for disease development. Areas covered: In this paper, the authors present AI methodologies that perform a robust deep sampling of altered genetic pathways to locate new therapeutic targets, assist in drug repurposing and speed up and optimize the drug selection process. Expert opinion: AI is a potential solution to a number of drug discovery problems, though one should, bear in mind that the quality of data predicts the overall quality of the prediction, as in any modeling task in data science. The use of transparent methodologies is crucial, particularly in drug repositioning/repurposing in rare diseases.

We present a preliminary analysis about the use of convolutional neural networks (CNNs) for the early detection of breast cancer via infrared thermography. The two main challenges of using CNNs are having at disposal a large set of images and the required processing time. The thermographies were obtained from Vision Lab and the calculations were implemented using Fast.ai and Pytorch libraries, which offer excellent results in image classification. Different architectures of convolutional neural networks were compared and the best results were obtained with resnet34 and resnet50, reaching a predictive accuracy of 100% in blind validation. Other arquitectures also provided high classification accuracies. Deep neural networks provide excellent results in the early detection of breast cancer via infrared thermographies, with technical and computational resources that can be easily implemented in medical practice. Further research is needed to asses the probabilistic localization of the tumor regions using larger sets of annotated images and assessing the uncertainty of these techniques in the diagnosis.

We discuss the use of the regularized linear discriminant analysis (LDA) as a model reduction technique combined with particle swarm optimization (PSO) in protein tertiary structure prediction, followed by structure refinement based on singular value decomposition (SVD) and PSO. The algorithm presented in this paper corresponds to the category of template-based modeling. The algorithm performs a preselection of protein templates before constructing a lower dimensional subspace via a regularized LDA. The protein coordinates in the reduced spaced are sampled using a highly explorative optimization algorithm, regressive&ndash, regressive PSO (RR-PSO). The obtained structure is then projected onto a reduced space via singular value decomposition and further optimized via RR-PSO to carry out a structure refinement. The final structures are similar to those predicted by best structure prediction tools, such as Rossetta and Zhang servers. The main advantage of our methodology is that alleviates the ill-posed character of protein structure prediction problems related to high dimensional optimization. It is also capable of sampling a wide range of conformational space due to the application of a regularized linear discriminant analysis, which allows us to expand the differences over a reduced basis set.

We discuss the relationship between the problem of protein tertiary structure prediction from the amino acid sequence and the uncertainty analysis. The algorithm presented in this paper belongs to the category of decoy-based modeling, where different known protein models are used to establish a low dimensional space via Principal Component Analysis. The low dimensional space is utilized to perform an energy optimization via a family of very explorative Particle Swarm Optimizers to find the global minimum. The aim of this procedure is to get a representative sample of the nonlinear equivalent region, that is, protein models that have their energy lower than a certain energy bound. The posterior analysis of this family provides very valuable information about the backbone structure of the native conformation and its possible alternate states. This methodology has the advantage to be simple and fast and can help to the refinement of tertiary protein structure. We comprehensively illustrate the performance of our algorithm on one protein from the CASP-9 protein structure prediction experiment. We also provide a theoretical analysis of the energy landscape found in the tertiary structure protein inverse problem, explaining why model reduction techniques (principal component analysis in this case) serve to alleviate the ill-posed character of this high dimensional optimization problem. In addition, we expand the computational benchmark with a summary of other CASP-9 proteins in the Appendix.

We discuss applicability of principal component analysis (PCA) for protein tertiary structure prediction from amino acid sequence. The algorithm presented in this paper belongs to the category of protein refinement models and involves establishing a low-dimensional space where the sampling (and optimization) is carried out via particle swarm optimizer (PSO). The reduced space is found via PCA performed for a set of low-energy protein models previously found using different optimization techniques. A high frequency term is added into this expansion by projecting the best decoy into the PCA basis set and calculating the residual model. This term is aimed at providing high frequency details in the energy optimization. The goal of this research is to analyze how the dimensionality reduction affects the prediction capability of the PSO procedure. For that purpose, different proteins from the Critical Assessment of Techniques for Protein Structure Prediction experiments were modeled. In all the cases, both the energy of the best decoy and the distance to the native structure have decreased. Our analysis also shows how the predicted backbone structure of native conformation and of alternative low energy states varies with respect to the PCA dimensionality. Generally speaking, the reconstruction can be successfully achieved with 10 principal components and the high frequency term. We also provide a computational analysis of protein energy landscape for the inverse problem of reconstructing structure from the reduced number of principal components, showing that the dimensionality reduction alleviates the ill-posed character of this high-dimensional energy optimization problem. The procedure explained in this paper is very fast and allows testing different PCA expansions. Our results show that PSO improves the energy of the best decoy used in the PCA when the adequate number of PCA terms is considered.

In this paper, we compare different sampling algorithms used for identifying the defective pathways in highly underdetermined phenotype prediction problems. The first algorithm (Fisher’s ratio sampler) selects the most discriminatory genes and samples the high discriminatory genetic networks according to a prior probability that it is proportional to their individual Fisher’s ratio. The second one (holdout sampler) is inspired by the bootstrapping procedure used in regression analysis and uses the minimum-scale signatures found in different random hold outs to establish the most frequently sampled genes. The third one is a pure random sampler which randomly builds networks of differentially expressed genes. In all these algorithms, the likelihood of the different networks is established via leave one out cross-validation (LOOCV), and the posterior analysis of the most frequently sampled genes serves to establish the altered biological pathways. These algorithms are compared to the results obtained via Bayesian Networks (BNs). We show the application of these algorithms to a microarray dataset concerning Triple Negative Breast Cancers. This comparison shows that the Random, Fisher’s ratio and Holdout samplers are most effective than BNs, and all provide similar insights about the genetic mechanisms that are involved in this disease. Therefore, it can be concluded that all these samplers are good alternatives to Bayesian Networks which much lower computational demands. Besides this analysis confirms the insight that the altered pathways should be independent of the sampling methodology and the classifier that is used to infer them.

We discuss applicability of Principal Component Analysis and Particle Swarm Optimization in protein tertiary structure prediction. The proposed algorithm is based on establishing a low-dimensional space where the sampling (and optimization) is carried out via Particle Swarm Optimizer (PSO). The reduced space is found via Principal Component Analysis (PCA) performed for a set of previously found low-energy protein models. A high frequency term is added into this expansion by projecting the best decoy into the PCA basis set and calculating the residual model. Our results show that PSO improves the energy of the best decoy used in the PCA considering an adequate number of PCA terms.

We discuss the use of the Singular Value Decomposition as a model reduction technique in Protein Tertiary Structure prediction, alongside to the uncertainty analysis associated to the tertiary protein predictions via Particle Swarm Optimization (PSO). The algorithm presented in this paper corresponds to the category of the decoy-based modelling, since it first finds a good protein model located in the low energy region of the protein energy landscape, that is used to establish a three-dimensional space where the free-energy optimization and search is performed via an exploratory version of PSO. The ultimate goal of this algorithm is to get a representative sample of the protein backbone structure and the alternate states in an energy region equivalent or lower than the one corresponding to the protein model that is used to establish the expansion (model reduction), obtaining as result other protein structures that are closer to the native structure and a measure of the uncertainty in the protein tertiary protein reconstruction. The strength of this methodology is that it is simple and fast, and serves to alleviate the ill-posed character of the protein structure prediction problem, which is very highly dimensional, improving the results when it is performed in a good protein model of the low energy region. To prove this fact numerically we present the results of the application of the SVD-PSO algorithm to a set of proteins of the CASP competition whose native’s structures are known.

In this paper, we introduce the holdout sampler to find the defective pathways in high underdetermined phenotype prediction problems. This sampling algorithm is inspired by the bootstrapping procedure used in regression analysis to established confidence bounds. We show that working with partial information (data bags) serves to sample the linear uncertainty region in a simple regression problem, mainly along the axis of greatest uncertainty that corresponds to the smallest singular value of the system matrix. This procedure applied to a phenotype prediction problem, considered as a generalized prediction problem between the set of genetic signatures and the set of classes in which the phenotype is divided, serves to unravel the ensemble of altered pathways in the transcriptome that are involved in the disease development. The algorithm looks for the minimum-scale genetic signature in each random holdout and the likelihood (predictive accuracy) is established using the validation dataset via a nearest-neighbor classifier. The posterior analysis serves to identify the header genes that most-frequently appear in the different hold-outs and are therefore robust to a partial lack of samples. These genes are used to establish the genetic pathways and the biological processes involved in the disease progression. This algorithm is much faster, robust and simpler than Bayesian Networks. We show its application to a microarray dataset concerning a type of breast cancers with poor prognoses (TNBC).

Introduction It has become clear that noise generated during the assay and analytical processes has the ability to disrupt accurate interpretation of genomic studies. Not only does such noise impact the scientific validity and costs of studies, but when assessed in the context of clinically translatable indications such as phenotype prediction, it can lead to inaccurate conclusions that could ultimately impact patients. We applied a sequence of ranking methods to damp noise associated with microarray outputs, and then tested the utility of the approach in three disease indications using publically available datasets. Materials and methods This study was performed in three phases. We first theoretically analyzed the effect of noise in phenotype prediction problems showing that it can be expressed as a modeling error that partially falsifies the pathways. Secondly, via synthetic modeling, we performed the sensitivity analysis for the main gene ranking methods to different types of noise. Finally, we studied the predictive accuracy of the gene lists provided by these ranking methods in synthetic data and in three different datasets related to cancer, rare and neurodegenerative diseases to better understand the translational aspects of our findings. Results and discussion In the case of synthetic modeling, we showed that Fisher’s Ratio (FR) was the most robust gene ranking method in terms of precision for all the types of noise at different levels. Significance Analysis of Microarrays (SAM) provided slightly lower performance and the rest of the methods (fold change, entropy and maximum percentile distance) were much less precise and accurate. The predictive accuracy of the smallest set of high discriminatory probes was similar for all the methods in the case of Gaussian and Log-Gaussian noise. In the case of class assignment noise, the predictive accuracy of SAM and FR is higher. Finally, for real datasets (Chronic Lymphocytic Leukemia, Inclusion Body Myositis and Amyotrophic Lateral Sclerosis) we found that FR and SAM provided the highest predictive accuracies with the smallest number of genes. Biological pathways were found with an expanded list of genes whose discriminatory power has been established via FR. Conclusions We have shown that noise in expression data and class assignment partially falsifies the sets of discriminatory probes in phenotype prediction problems. FR and SAM better exploit the principle of parsimony and are able to find subsets with less number of high discriminatory genes. The predictive accuracy and the precision are two different metrics to select the important genes, since in the presence of noise the most predictive genes do not completely coincide with those that are related to the phenotype. Based on the synthetic results, FR and SAM are recommended to unravel the biological pathways that are involved in the disease development.

History matching provides to reservoir engineers an improved spatial distribution of physical properties to be used in forecasting the reservoir response for field management. The ill-posed character of the history-matching problem yields nonuniqueness and numerical instabilities that increase with the reservoir complexity. These features might cause local optimization methods to provide unpredictable results not being able to discriminate among the multiple models that fit the observed data (production history). Also, the high dimensionality of the inverse problem impedes estimation of uncertainties using classical Markov-chain Monte Carlo methods. We attenuated the ill-conditioned character of this history-matching inverse problem by reducing the model complexity using a spatial principal component basis and by combining as observables flow production measurements and time-lapse seismic crosswell tomographic images. Additionally the inverse problem was solved in a stochastic framework. For this purpose, we used a family of particle swarm optimization (PSO) optimizers that have been deduced from a physical analogy of the swarm system. For a synthetic sand-and-shale reservoir, we analyzed the performance of the different PSO optimizers, both in terms of exploration and convergence rate for two different reservoir models with different complexity and under the presence of different levels of white Gaussian noise added to the synthetic observed data. We demonstrated that PSO optimizers have a very good convergence rate for this example, and provide in addition, approximate measures of uncertainty around the optimum facies model. The PSO algorithms are robust in presence of noise, which is always the case for real data.

PSO is an optimization technique inspired by the social behavior of individuals in nature (swarms) that has been successfully used in many different engineering fields. In addition, the PSO algorithm can be physically interpreted as a stochastic damped mass–spring system. This analogy has served to introduce the PSO continuous model and to deduce a whole family of PSO algorithms using different finite-differences schemes. These algorithms are characterized in terms of convergence by their respective first and second order stability regions. The performance of these new algorithms is first checked using synthetic functions showing a degree of ill-posedness similar to that found in many geophysical inverse problems having their global minimum located on a very narrow flat valley or surrounded by multiple local minima. Finally we present the application of these PSO algorithms to the analysis and solution of a VES inverse problem associated with a seawater intrusion in a coastal aquifer in southern Spain. PSO family members are successfully compared to other well known global optimization algorithms (binary genetic algorithms and simulated annealing) in terms of their respective convergence curves and the sea water intrusion depth posterior histograms.

Transmission tomography methods show a great sensibility to data variability, which eventually includes data errors, often present in field experiments. Local optimization methods, traditionally used to solve this inverse problem, are very sensitive to these difficulties, failing to converge properly in the presence of spurious data. Regularization methods partially cope with these weaknesses, damping the instabilities.

This paper is devoted to present the stochastic stability analysis of a novel PSO version, the aligned PSO, and its application to the face recognition problem using supervised learning techniques. Its application to the ORL database provides 100% median identification accuracy over 100 independent runs.

We summarize, for marine electromagnetic inverse problems, a newly developed inverse solution appraisal and non-linear uncertainty estimation method based on parameter reduction techniques and efficient posterior model space sampling. This method uses model compression methods to decorrelate parameters in an inverse solution and represent all feasible posterior models as linear combinations of a small number of model-derived basis vectors and corresponding coefficients. This allows us to reduce the posterior sampling problem by orders of magnitude. We further contract this reduced-dimensional posterior space by confining all acceptable models to a set of bounds mapped from our original parameter space. As a final step to increase efficiency, we implement a geometric sampling scheme that we use to approximate our restricted posterior by generating feasible models on adaptive, optimally-sparse grids. The sampled equi-feasible models are accepted according to a data misfit threshold and constitute an optimally-sparse representation of the restricted posterior model space. Although very efficient, our method imposes a bias in the posterior space by truncating the basis expansion during the model reduction step. To investigate this, we compare two types of fast and scalable bases, the discrete cosine transform and singular value decomposition. We demonstrate that while the choice of base does influence the type of models sampled and the model rejection rates, the posterior statistics are generally compatible between the methods providing confidence in the uncertainty estimations. For the marine electromagnetic problem, we show that a representative ensemble of equivalent inverse solutions can be generated for realistically-sized inverse problems and that solution appraisal and uncertainty inference follow directly from ensemble statistics.

Particle swarm optimization (PSO) is a Swarm Intelligence technique used for optimization motivated by the social behavior of individuals in large groups in nature. The damped mass-spring analogy known as the PSO continuous model allowed us to derive a whole family of particle swarm optimizers with different properties with regard to their exploitation/exploration balance. Using the theory of stochastic differential and difference equations, we fully characterize the stability behavior of these algorithms. PSO and RR-PSO are the most performant algorithms of this family in terms of rate of convergence. Other family members have better exploration capabilities. The so called four point algorithms use more information of previous iterations to update the particles positions and trajectories and seem to be more exploratory than most of the 3 points versions. Finally, based on the done analysis, we can affirm that the PSO optimizers are not heuristic algorithms since there exist mathematical results that can be used to explain their consistency/convergence.

In this paper we present a powerful set of Particle Swarm optimizers for inverse modeling. Their design is based on the interpretation of the swarm dynamics as a stochastic damped mass-spring system. All the PSO optimizers have very different exploitation and exploration capabilities. Their convergence can be related to the stability of their first and second order moments of the particle trajectories. Based on these results we present their corresponding cloud algorithms where each particle in the swarm has different inertia (damping) and acceleration (rigidity) constants. These algorithms show a very good balance between exploration and exploitation and their use avoids the tuning of the PSO parameters. These algorithms have been successfully applied to environmental geophysics and petroleum reservoir engineering where the combined use of model reduction techniques allow posterior sampling in high dimensional spaces.

PSO is an optimization technique that has been successfully used in many different engineering fields. The PSO algorithm can be physically interpreted as a stochastic damped mass-spring system. This analogy served us to introduce the PSO continuous model and to deduce a whole family of PSO algorithms arising from the discretization of the PSO continuous model. We also analyze their respective first order and second order stability regions from the stochastic point of view. Their performance is also checked using synthetic functions showing a degree of illposedness similar to that found in many geophysical inverse problems. Finally we present the application of these algorithms to the analysis and solution of a VES inverse problem associated to a seawater intrusion in a coastal aquifer in South Spain.

In this paper, we propose different ensemble learning algorithms and their application to the face recognition problem. Three types of attributes are used for image representation: statistical, spectral, and segmentation features and regional descriptors. Classification is performed by nearest neighbor using different p-norms defined in the corresponding spaces of attributes. In this approach, each attribute together with its corresponding type of the analysis (local or global) and the distance criterion (norm or cosine), define a different classifier. The classification is unsupervised since no class information is used to improve the design of the different classifiers. Three different versions of ensemble classifiers are proposed in this paper: CAV1, CAV2, and CBAG, being the main differences among them the way the image candidates that perform the consensus are selected. The main results shown in this paper are the following: 1. The statistical attributes (local histogram and percentiles) are the individual classifiers that provided the higher accuracies, followed by the spectral methods (DWT), and the regional features (texture analysis). 2. No single attribute is able to provide systematically 100% accuracy over the ORL database. 3. The accuracy and stability of the classification is increased by consensus classification (ensemble learning techniques). 4. Optimum results are obtained by reducing the number of classifiers taking into account their diversity, and by optimizing the parameters of these classifiers using a member of the Particle Swarm Optimization (PSO) family. These results are in accord with the conclusions that are presented in the literature using ensemble learning methodologies, that is, it is possible to build strong classifiers by assembling different weak (or simple) classifiers based on different and diverse image attributes. Due to these encouraging results, future research will be devoted to the use of supervised ensemble techniques in face recognition and in other important biometric problems.

The PSO algorithm can be physically interpreted as a stochastic damped mass-spring system: the so-called PSO continuous model. Furthermore, PSO corresponds to a particular discretization of the PSO continuous model. Based on this mechanical analogy we derived in the past a family of PSO-like versions, where the acceleration is discretized using a centered scheme and the velocity of the particles can be regressive (GPSO), progressive (CP-GPSO) or centered (CC-GPSO). Although the first and second order trajectories of these algorithms are isomorphic, CC-GPSO and CP-GPSO are very different from GPSO. In this paper we present two other PSO-like methods: PP-GPSO and RR-GPSO. These algorithms correspond respectively to progressive and regressive discretizations in acceleration and velocity. PP-PSO has the same velocity update than GPSO, but the velocities used to update the trajectories are delayed one iteration, thus, PP-PSO acts as a Jacobi system updating positions and velocities at the same time. RR-GPSO is similar to a GPSO with stochastic constriction factor. Both versions have a very different behavior from GPSO and the other family members introduced in the past: CC-PSO and CP-PSO. RR-PSO seems to have the greatest convergence rate and its good parameter sets can be calculated analytically since they are along a straight line located in the first order stability region. Conversely PP-PSO seems to be a more explorative version, although the behavior of these algorithms can be partly problem dependent. Both exhibit a very peculiar behavior, very different from other family members, and thus they can be called distant PSO relatives. RR-PSO have the greatest convergence rate of all family members for a wide range of benchmark functions with different numerical complexity in 10, 30 and 50 dimensions. These algorithms have been succesfully applied for protein secondary structure prediction and in oil and gas reservoir optimization.