Search

Highlights • Model discrepancy prediction for dynamic system using Bayesian state estimation. • Using neural network as a surrogate sensor. • Predicting discrepancy for multi-component systems under untested loading. Abstract We present a methodology to predict the discrepancy in modeling dynamic system response for untested system inputs. In particular, the methodology focuses on multi-component systems, where discrepancy in the system response is caused by model form errors in some of the system components. We combine a deterministic time-series artificial neural network (ANN) with Bayesian state estimation to formulate a relationship between inputs to the system and the corresponding discrepancies in the model estimates of system responses. We also identify the system components affected by model form errors, and predict discrepancies even in unmeasured locations of the system. The proposed approach is illustrated on a linear multi-degree of freedom dynamic system, and on an air cycle machine – a multi-component dynamic system. [ABSTRACT FROM AUTHOR]

Highlights • Quantify the uncertainty in bulk thermal conductivity of Si. • Efficient UQ through dimension reduction. • Sensitivity analysis of the Stillinger-Weber potential parameters. • Low-dimensional surrogate construction. • Fast calibration in a Bayesian setting. Abstract This paper develops an efficient methodology for both forward and inverse problems in uncertainty quantification with respect to molecular dynamics simulation. Specifically, our objectives are to investigate the impact of uncertainty in the Stillinger-Weber (SW) potential parameters on NEMD-based predictions of bulk thermal conductivity of silicon (forward problem), and perform a Bayesian calibration of these parameters using experimental data (inverse problem). However, both analyses typically require tens of thousands of model evaluations and therefore, relying purely on atomistic simulations would be impractical. The common strategy of building a surrogate model in the space of the uncertain parameters is also unaffordable due to the need for many training evaluations using atomistic simulations. Therefore, computational effort is minimized in this paper by reducing the dimensionality of the input space of the surrogate model by first computing the so-called active subspace. The active subspace is found to be 1-dimensional, indicating enormous scope for dimension reduction and computational savings. A surrogate model is then built in the 1-dimensional subspace to help quantify the variability of the bulk thermal conductivity, and is shown to have reasonable accuracy. The active subspace is also used to perform efficient global sensitivity analysis (GSA) of the SW parameters. Finally, we use the active subspace-based surrogate model for fast calibration of SW parameters in a Bayesian setting. [ABSTRACT FROM AUTHOR]

Existing methods for the computation of global sensitivity indices are challenged by both number of input-output samples required and the presence of dependent or correlated variables. First, a methodology is developed to increase the efficiency of sensitivity computations with independent variables by incorporating optimal space-filling quasi-random sequences into an existing importance sampling-based kernel regression sensitivity method. Two prominent situations where parameter correlations cannot be ignored, however, are (1) posterior distributions of calibrated parameters and (2) transient, coupled simulations. Therefore, the sensitivity methodology is generalized to dependent variables allowing for efficient post-calibration sensitivity analyses using input-output samples obtained directly from Bayesian calibration. These methods are illustrated using coupled, aerothermal simulations where it is observed that model errors and parameter correlations control the sensitivity estimates until coupling effects become dominant over time. [ABSTRACT FROM AUTHOR]

The accuracy of model-based reliability analysis is affected by the uncertainty regarding the model parameters used to predict the behavior of the engineering system. The uncertainty in the model parameters can be reduced by combining prior knowledge about the parameters with observed data regarding system inputs and outputs. In some cases, the information about the observations is only available as abstracted data, where the original raw data have been reduced to a summarized representation. Common forms of abstracted data include summary statistics, such as the mean and variance for continuous variables and observed frequencies for discrete variables. In the context of reliability analysis, a common form of available information is summarized reliability data for various mechanical components (e.g., failure rates or failure probabilities) instead of detailed actual test data. This paper presents a methodology for updating the model parameters using these abstracted data forms through a Bayesian network. First, the concept of a statistics function is developed and linked to the abstracted data forms. The concept of arc reversal is then exploited to transform the Bayesian network to a form that can be used to incorporate the statistics function and thereby enable the updating of the model parameters. Several numerical examples are used to demonstrate the applicability and generality of the proposed method for several different forms of abstracted data. [ABSTRACT FROM AUTHOR]

An uncertainty aggregation and reduction framework is presented for structure-material performance prediction. Different types of uncertainty sources, structural analysis model, and material performance prediction model are connected through a Bayesian network for systematic uncertainty aggregation analysis. To reduce the uncertainty in the computational structure-material performance prediction model, Bayesian updating using experimental observation data is investigated based on the Bayesian network. It is observed that the Bayesian updating results will have large error if the model cannot accurately represent the actual physics, and that this error will be propagated to the predicted performance distribution. To address this issue, this paper proposes a novel uncertainty reduction method by integrating Bayesian calibration with model validation adaptively. The observation domain of the quantity of interest is first discretized into multiple segments. An adaptive algorithm is then developed to perform model validation and Bayesian updating over these observation segments sequentially. Only information from observation segments where the model prediction is highly reliable is used for Bayesian updating; this is found to increase the effectiveness and efficiency of uncertainty reduction. A composite rotorcraft hub component fatigue life prediction model, which combines a finite element structural analysis model and a material damage model, is used to demonstrate the proposed method. [ABSTRACT FROM AUTHOR]

Inference is one key objective in a Bayesian network (BN), and it aims to estimate the posterior distributions of state variables based on evidence (observations). While efficient analytical inference algorithms (either approximate or exact) for BN with discrete variables have been well-established in the literature, the inference in BN with continuous variables is still challenging if the BN is non-linear and/or non-Gaussian. In this case we can either discretize the continuous variable and utilize the inference approaches for discrete BN; or we have to use sampling-based methods such as MCMC for static BN and particle filter for dynamic BN. This paper proposes a network collapsing technique based on the concept of probability integral transform to convert a multi-layer BN to an equivalent simple two-layer BN, so that the unscented Kalman filter can be applied to the collapsed BN and the posterior distributions of state variables can be obtained analytically. For dynamic BN, the proposed method is also able to propagate the state variables to the next time step analytically using the unscented transform, based on the assumption that the posterior distributions of state variables are Gaussian. Thus the proposed method achieves a very fast approximate solution, making it particularly suitable for dynamic BN where inference and uncertainty propagation are required over many time steps. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach for multi-objective optimization under both aleatory and epistemic sources of uncertainty. Given paired samples of the inputs and outputs from the system analysis model, a Bayesian network (BN) is built to represent the joint probability distribution of the inputs and outputs. In each design iteration, the optimizer provides the values of the design variables to the BN, and copula-based sampling is used to rapidly generate samples of the output variables conditioned on the input values. Samples from the conditional distributions are used to evaluate the objectives and constraints, which are fed back to the optimizer for further iteration. The proposed approach is formulated in the context of reliability-based design optimization (RBDO). The joint probability of multiple objectives and constraints is included in the formulation. The Bayesian network along with conditional sampling is exploited to select training points that enable effective construction of the Pareto front. A vehicle side impact problem is employed to demonstrate the proposed methodology. [ABSTRACT FROM AUTHOR]

This paper presents a dynamic fuzzy stochastic neural network model for nonparametric system identification using ambient vibration data. The model is developed to handle two types of imprecision in the sensed data: fuzzy information and measurement uncertainties. The dimension of the input vector is determined by using the false nearest neighbor approach. A Bayesian information criterion is applied to obtain the optimum number of stochastic neurons in the model. A fuzzy C-means clustering algorithm is employed as a data mining tool to divide the sensed data into clusters with common features. The fuzzy stochastic model is created by combining the fuzzy clusters of input vectors with the radial basis activation functions in the stochastic neural network. A natural gradient method is developed based on the Kullback–Leibler distance criterion for quick convergence of the model training. The model is validated using a power density pseudospectrum approach and a Bayesian hypothesis testing-based metric. The proposed methodology is investigated with numerically simulated data from a Markov Chain model and a two-story planar frame, and experimentally sensed data from ambient vibration data of a benchmark structure. [ABSTRACT FROM AUTHOR]

Accelerated life testing (ALT) design is usually performed based on assumptions of life distributions, stress-life relationship, and empirical reliability models. Time-dependent reliability analysis on the other hand seeks to predict product and system life distribution based on physics-informed simulation models. This paper proposes an ALT design framework that takes advantages of both types of analyses. For a given testing plan, the corresponding life distributions under different stress levels are estimated based on time-dependent reliability analysis. Because both aleatory and epistemic uncertainty sources are involved in the reliability analysis, ALT data is used in this paper to update the epistemic uncertainty using Bayesian statistics. The variance of reliability estimation at the nominal stress level is then estimated based on the updated time-dependent reliability analysis model. A design optimization model is formulated to minimize the overall expected testing cost with constraint on confidence of variance of the reliability estimate. Computational effort for solving the optimization model is minimized in three directions: (i) efficient time-dependent reliability analysis method; (ii) a surrogate model is constructed for time-dependent reliability under different stress levels; and (iii) the ALT design optimization model is decoupled into a deterministic design optimization model and a probabilistic analysis model. A cantilever beam and a helicopter rotor hub are used to demonstrate the proposed method. The results show the effectiveness of the proposed ALT design optimization model. Copyright © 2015 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This paper proposes a probabilistic framework to include both aleatory and epistemic uncertainty within model-based reliability estimation of engineering systems for individual limit states. Epistemic uncertainty is considered due to both data and model sources. Sparse point and/or interval data regarding the input random variables leads to uncertainty regarding their distribution types, distribution parameters, and correlations; this statistical uncertainty is included in the reliability analysis through a combination of likelihood-based representation, Bayesian hypothesis testing, and Bayesian model averaging techniques. Model errors, which include numerical solution errors and model form errors, are quantified through Gaussian process models and included in the reliability analysis. The probability integral transform is used to develop an auxiliary variable approach that facilitates a single-level representation of both aleatory and epistemic uncertainty. This strategy results in an efficient single-loop implementation of Monte Carlo simulation (MCS) and FORM/SORM techniques for reliability estimation under both aleatory and epistemic uncertainty. Two engineering examples are used to demonstrate the proposed methodology. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach for efficient uncertainty quantification and propagation in multidisciplinary analysis with a large number of coupling variables. The proposed methodology has three elements: Bayesian network, copula-based sampling, and principal component analysis. The Bayesian network represents the joint distribution of multiple variables through marginal distributions and conditional probabilities, and it updates the distributions based on new data. This paper uses this concept to develop a novel approach for probabilistic multidisciplinary analysis, that is, inference of distributions of the coupling variables by enforcing the interdisciplinary compatibility condition (which is treated similar to data for updating). The Bayesian network is built using only a few iterations of the coupled multidisciplinary analysis, without iterating until convergence. A copula-based sampling technique is employed for efficient sampling from the joint and conditional distributions. Further savings are achieved through dimension reduction using principal component analysis. A mathematical example and an aeroelastic analysis of an aircraft wing are used to demonstrate the proposed probabilistic multidisciplinary analysis methodology. [ABSTRACT FROM AUTHOR]

Calibration of model parameters is an essential step in predicting the response of a complicated system, but the lack of data at the system level makes it impossible to conduct this quantification directly. In such a situation, system model parameters are estimated using tests at lower levels of complexity which share the same model parameters with the system. For such a multi-level problem, this paper proposes a methodology to quantify the uncertainty in the system level prediction by integrating calibration, validation and sensitivity analysis at different levels. The proposed approach considers the validity of the models used for parameter estimation at lower levels, as well as the relevance at the lower level to the prediction at the system level. The model validity is evaluated using a model reliability metric, and models with multivariate output are considered. The relevance is quantified by comparing Sobol indices at the lower level and system level, thus measuring the extent to which a lower level test represents the characteristics of the system so that the calibration results can be reliably used in the system level. Finally the results of calibration, validation and relevance analysis are integrated in a roll-up method to predict the system output. [ABSTRACT FROM AUTHOR]

This paper investigates the use of structural dynamics computational models with multiple levels of fidelity in the calibration of system parameters. Different types of models may be available for the estimation of unmeasured system properties, with different levels of physics fidelity, mesh resolution and boundary condition assumptions. In order to infer these system properties, Bayesian calibration uses information from multiple sources (including experimental data and prior knowledge), and comprehensively quantifies the uncertainty in the calibration parameters. Estimating the posteriors is done using Markov Chain Monte Carlo sampling, which requires a large number of computations, thus making the use of a high-fidelity model for calibration prohibitively expensive. On the other hand, use of a low-fidelity model could lead to significant error in calibration and prediction. Therefore, this paper develops an approach for model parameter calibration with a low-fidelity model corrected using higher fidelity simulations, and investigates the trade-off between accuracy and computational effort. The methodology is illustrated for a curved panel located in the vicinity of a hypersonic aircraft engine, subjected to acoustic loading. Two models (a frequency response analysis and a full time history analysis) are combined to calibrate the damping characteristics of the panel. [ABSTRACT FROM AUTHOR]

This paper proposes a systematic framework using Bayesian networks to aggregate the uncertainty from multiple sources for the purpose of uncertainty quantification (UQ) in the prediction of performance of a manufacturing process. Energy consumption, one of the key metrics of manufacturing process sustainability performance, is used to illustrate the proposed methodology. The prediction of energy consumption is not straightforward due to the presence of uncertainty in many process variables and the models used for prediction. The uncertainty is both aleatory (statistical) and epistemic (lack of knowledge); both sources of uncertainty are considered in the proposed UQ methodology. The uncertainty sources occur at different stages of the manufacturing process and do not combine in a straightforward manner, thus a Bayesian network approach is found to be advantageous in uncertainty aggregation. A dimension reduction approach through variance-based global sensitivity analysis is proposed to reduce the number of variables in the system and facilitate scalability in high-dimensional problems. The proposed methodologies for uncertainty quantification and dimension reduction are demonstrated using two examples – an injection molding process and a welding process. [ABSTRACT FROM AUTHOR]

This paper proposes a Bayesian methodology to integrate model verification, validation, and calibration activities for the purpose of overall uncertainty quantification in different types of engineering systems. The methodology is first developed for single-level models, and then extended to systems that are studied using multi-level models that interact with each other. Two types of interactions amongst multi-level models are considered: (1) Type-I, where the output of a lower-level model (component and/or subsystem) becomes an input to a higher level system model, and (2) Type-II, where parameters of the system model are inferred using lower-level models and tests (that describe simplified components and/or isolated physics). The various models, their inputs, parameters, and outputs, experimental data, and various sources of model error are connected through a Bayesian network. The results of calibration, verification, and validation with respect to each individual model are integrated using the principles of conditional probability and total probability, and propagated through the Bayesian network in order to quantify the overall system-level prediction uncertainty. The proposed methodology is illustrated with numerical examples that deal with heat conduction and structural dynamics. [ABSTRACT FROM AUTHOR]

Probabilistic models are desirable for diagnosis and prognosis to account for many sources of uncertainty. However, the probability distributions of system variables and the conditional probability relationships between components or subsystems may be unknown or incomplete. Useful information for such systems may be available from multiple sources and in varying formats such as operational and laboratory data, mathematical models, reliability data, or expert opinion. Conventional modeling techniques do not generally include all such information. This paper presents a methodology for constructing a system model for use in diagnosis and prognosis in the presence of such heterogeneous information. An attractive modeling paradigm for this methodology is the dynamic Bayesian network (DBN). A systematic approach is developed to incorporate various types of data in the DBN and to learn the probability distributions of the system variables as well as the conditional probability relationships between them. The structure and distribution parameters of the DBN are obtained via established learning methods. The proposed methodology is demonstrated for a hydraulic actuator system. Copyright © 2013 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Quantitative model validation is playing an increasingly important role in performance and reliability assessment of a complex system whenever computer modelling and simulation are involved. The foci of this paper are to pursue a Bayesian probabilistic approach to quantitative model validation with non-normality data, considering data uncertainty and to investigate the impact of normality assumption on validation accuracy. The Box–Cox transformation method is employed to convert the non-normality data, with the purpose of facilitating the overall validation assessment of computational models with higher accuracy. Explicit expressions for the interval hypothesis testing-based Bayes factor are derived for the transformed data in the context of univariate and multivariate cases. Bayesian confidence measure is presented based on the Bayes factor metric. A generalized procedure is proposed to implement the proposed probabilistic methodology for model validation of complicated systems. Classic hypothesis testing method is employed to conduct a comparison study. The impact of data normality assumption and decision threshold variation on model assessment accuracy is investigated by using both classical and Bayesian approaches. The proposed methodology and procedure are demonstrated with a univariate stochastic damage accumulation model, a multivariate heat conduction problem and a multivariate dynamic system. [ABSTRACT FROM AUTHOR]

Abstract: This paper develops new insights into quantitative methods for the validation of computational model prediction. Four types of methods are investigated, namely classical and Bayesian hypothesis testing, a reliability-based method, and an area metric-based method. Traditional Bayesian hypothesis testing is extended based on interval hypotheses on distribution parameters and equality hypotheses on probability distributions, in order to validate models with deterministic/stochastic output for given inputs. Formulations and implementation details are outlined for both equality and interval hypotheses. Two types of validation experiments are considered—fully characterized (all the model/experimental inputs are measured and reported as point values) and partially characterized (some of the model/experimental inputs are not measured or are reported as intervals). Bayesian hypothesis testing can minimize the risk in model selection by properly choosing the model acceptance threshold, and its results can be used in model averaging to avoid Type I /II errors. It is shown that Bayesian interval hypothesis testing, the reliability-based method, and the area metric-based method can account for the existence of directional bias, where the mean predictions of a numerical model may be consistently below or above the corresponding experimental observations. It is also found that under some specific conditions, the Bayes factor metric in Bayesian equality hypothesis testing and the reliability-based metric can both be mathematically related to the p-value metric in classical hypothesis testing. Numerical studies are conducted to apply the above validation methods to gas damping prediction for radio frequency (RF) micro-electro-mechanical-system (MEMS) switches. The model of interest is a general polynomial chaos (gPC) surrogate model constructed based on expensive runs of a physics-based simulation model, and validation data are collected from fully characterized experiments. [Copyright &y& Elsevier]

This article presents statistical approaches for model parameter estimation when the input and output measurements are imprecise in nature. Several types of imprecision are discussed: (1) In classical parameter estimation problems, paired input–output data points are used for model calibration. In reality, point data may be available at the input and output levels but they may not be paired, i.e. input and output measurements are independent of each other and there is no correspondence between them. (2) Data available for calibration may be in the form of intervals (range of values) rather than point values. This article investigates both least squares-based and likelihood-based approaches for model calibration in the presence of such imprecise and unpaired data. The likelihood-based approach makes fewer assumptions than the least squares-based approach, especially when aggregating information from multiple interval data. Though the likelihood-based approach is more expensive, it can compute the entire probability distribution of the model parameters, and facilitate further uncertainty propagation after parameter estimation. The proposed methods are illustrated using non-linear structural dynamics experimental data on energy dissipation due to friction in a lap joint, under impact loading. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a Bayesian probabilistic methodology to integrate model-based fatigue damage prognosis (FDP) with online and offline structural health monitoring (SHM) data. The prognosis uses fracture mechanics-based fatigue crack growth modeling, along with quantification of various sources of uncertainty, including natural variability, data uncertainty and model errors. These uncertainty sources are connected using a Bayesian network and a probabilistic sensitivity analysis is performed to assess the uncertainty contributions from these sources. The cycle-by-cycle simulation of fatigue crack growth is expedited via the use of a surrogate modeling technique (Gaussian process model) to replace computationally expensive finite element analysis. Real-time monitoring data of external variable amplitude loading history is used to construct a Bayesian autoregressive integrated moving average (ARIMA) model to predict and update the loading. On-ground crack inspection data is used to quantify the uncertainty in the initial and current size of an existing crack, using the Bayesian approach. Three possible cases of inspection results are considered: (1) crack is not detected; (2) crack is detected but not measured; (3) crack is detected and measured. Different scenarios of data availability (load monitoring data and inspection data) are considered for the prognosis of an individual component in a fleet. A numerical example, surface cracking in a rotorcraft mast under service loading, is implemented to illustrate the proposed methodology. The results of prognosis are validated using Bayesian hypothesis testing. [Copyright &y& Elsevier]

Abstract: This paper develops a methodology to assess the validity of computational models when some quantities may be affected by epistemic uncertainty. Three types of epistemic uncertainty regarding input random variables – interval data, sparse point data, and probability distributions with parameter uncertainty – are considered. When the model inputs are described using sparse point data and/or interval data, a likelihood-based methodology is used to represent these variables as probability distributions. Two approaches – a parametric approach and a non-parametric approach – are pursued for this purpose. While the parametric approach leads to a family of distributions due to distribution parameter uncertainty, the principles of conditional probability and total probability can be used to integrate the family of distributions into a single distribution. The non-parametric approach directly yields a single probability distribution. The probabilistic model predictions are compared against experimental observations, which may again be point data or interval data. A generalized likelihood function is constructed for Bayesian updating, and the posterior distribution of the model output is estimated. The Bayes factor metric is extended to assess the validity of the model under both aleatory and epistemic uncertainty and to estimate the confidence in the model prediction. The proposed method is illustrated using a numerical example. [Copyright &y& Elsevier]

Abstract: This paper presents a new Bayesian nonlinear structural equation modeling approach to hierarchical model assessment of dynamic systems, considering uncertainty in both predicted and measured time series data. A generalized structural equation modeling with nonlinear latent variables is presented to model two sets of relationships in multivariate hierarchical model assessment, namely, the computational model to system-level data, and low-level data to system-level data. A hierarchical Bayesian network with Markov Chain Monte Carlo simulation and Gibbs sampling is developed to represent the two relationships and estimate the influencing factors between them. A Bayesian interval hypothesis testing-based method is employed to quantify the confidence in the predictive model at various levels. The effect of low-level data on the model assessment at the system level is identified by Bayesian inference and factor analysis. The proposed methodology is implemented for hierarchical model validation of three dynamic system problems. [Copyright &y& Elsevier]

This paper presents a Bayesian-hypothesis-testing-based methodology for model validation and confidence extrapolation under uncertainty, using limited test data. An explicit expression of the Bayes factor is derived for the interval hypothesis testing. The interval method is compared with the Bayesian point null hypothesis testing approach. The Bayesian network with Markov Chain Monte Carlo simulation and Gibbs sampling is explored for extrapolating the inference from the validated domain at the component level to the untested domain at the system level. The effect of the number of experiments on the confidence in the model validation decision is investigated. The probabilities of Type I and Type II errors in decision-making during the model validation and confidence extrapolation are quantified. The proposed methodologies are applied to a structural mechanics problem. Numerical results demonstrate that the Bayesian methodology provides a quantitative approach to facilitate rational decisions in model validation and confidence extrapolation under uncertainty. [ABSTRACT FROM AUTHOR]

Abstract: A building block approach to model validation may proceed through various levels, such as material to component to subsystem to system, comparing model predictions with experimental observations at each level. Usually, experimental data becomes scarce as one proceeds from lower to higher levels. This paper presents a structural equation modeling approach to make use of the lower-level data for higher-level model validation under uncertainty, integrating several components: lower-level data, higher-level data, computational model, and latent variables. The method proposed in this paper uses latent variables to model two sets of relationships, namely, the computational model to system-level data, and lower-level data to system-level data. A Bayesian network with Markov chain Monte Carlo simulation is applied to represent the two relationships and to estimate the influencing factors between them. Bayesian hypothesis testing is employed to quantify the confidence in the predictive model at the system level, and the role of lower-level data in the model validation assessment at the system level. The proposed methodology is implemented for hierarchical assessment of three validation problems, using discrete observations and time-series data. [Copyright &y& Elsevier]

This paper presents a Bayesian hypothesis testing-based probabilistic assessment method for nonparametric damage detection of building structures, considering the uncertainties in both experimental results and model prediction. A dynamic fuzzy wavelet neural network method is employed as a nonparametric system identification model to predict the structural responses for damage evaluation. A Bayes factor evaluation metric is derived based on Bayes’ theorem and Gaussian distribution assumption of the difference between the experimental data and model prediction. The metric provides quantitative measure for assessing the accuracy of system identification and the state of global health of structures. The probability density function of the Bayes factor is constructed using the statistics of the difference of response quantities and Monte Carlo simulation technique to address the uncertainties in both experimental data and model prediction. The methodology is investigated with five damage scenarios of a four-story benchmark building. Numerical results demonstrate that the proposed methodology provides an effective approach for quantifying the damage confidence in the structural condition assessment. [ABSTRACT FROM AUTHOR]

Abstract: This paper investigates various statistical approaches for the validation of computational models when both model prediction and experimental observation have uncertainties, and proposes two new methods for this purpose. The first method utilizes hypothesis testing to accept or reject a model at a desired significance level. Interval-based hypothesis testing is found to be more practically useful for model validation than the commonly used point null hypothesis testing. Both classical and Bayesian approaches are investigated. The second and more direct method formulates model validation as a limit state-based reliability estimation problem. Both simulation-based and analytical methods are presented to compute the model reliability for single or multiple comparisons of the model output and observed data. The proposed methods are illustrated and compared using numerical examples. [Copyright &y& Elsevier]

Abstract: In this paper an energy-based Bayesian wavelet method is presented for validation assessment of multivariate predictive models under uncertainty, using time-series data collected from a dynamic system. Time history data are decomposed into multiple time–frequency resolutions using a discrete wavelet packet transform method. As a signal feature, wavelet packet component energy is computed in terms of the decomposed coefficients. The effectiveness of the selected feature is assessed using both cross-correlation and cross-coherence metrics. A generalized likelihood ratio is derived as a quantitative validation metric based on Bayes’ theorem and Gaussian distribution assumption of errors of the wavelet packet component energy between validation data and model prediction. The multivariate model is then assessed based on the Bayesian point and interval hypothesis testing approaches. The probability density function of the likelihood ratio is constructed using the statistics of multiple response quantities and Monte Carlo simulation. The proposed methodology is implemented in the validation of a structural dynamics problem, using multivariate time-series data sets. Sensitivity analysis is also performed to investigate the effect of parameter selection on the model validation decision. [Copyright &y& Elsevier]

Abstract: The importance of modeling and simulation in the scientific community has drawn interest towards methods for assessing the accuracy and uncertainty associated with such models. This paper addresses the validation and calibration of computer simulations using the thermal challenge problem developed at Sandia National Laboratories for illustration. The objectives of the challenge problem are to use hypothetical experimental data to validate a given model, and then to use the model to make predictions in an untested domain. With regards to assessing the accuracy of the given model (validation), we illustrate the use of Hotelling’s T 2 statistic for multivariate significance testing, with emphasis on the formulation and interpretation of such an analysis for validation assessment. In order to use the model for prediction, we next employ the Bayesian calibration method introduced by Kennedy and O’Hagan. Towards this end, we discuss how inherent variability can be reconciled with “lack-of-knowledge” and other uncertainties, and we illustrate a procedure that allows probability distribution characterization uncertainty to be included in the overall uncertainty analysis of the Bayesian calibration process. [Copyright &y& Elsevier]

Abstract: This paper develops a Bayesian methodology for assessing the confidence in model prediction by comparing the model output with experimental data when both are stochastic. The prior distribution of the response is first computed, which is then updated based on experimental observation using Bayesian analysis to compute a validation metric. A model error estimation methodology is then developed to include model form error, discretization error, stochastic analysis error (UQ error), input data error and output measurement error. Sensitivity of the validation metric to various error components and model parameters is discussed. A numerical example is presented to illustrate the proposed methodology. [Copyright &y& Elsevier]

This paper proposes a methodology based on Bayesian statistics to assess the validity of reliability computational models when full-scale testing is not possible. Sub-module validation results are used to derive a validation measure for the overall reliability estimate. Bayes networks are used for the propagation and updating of validation information from the sub-modules to the overall model prediction. The methodology includes uncertainty in the experimental measurement, and the posterior and prior distributions of the model output are used to compute a validation metric based on Bayesian hypothesis testing. Validation of a reliability prediction model for an engine blade under high-cycle fatigue is illustrated using the proposed methodology. [Copyright &y& Elsevier]

This paper develops a methodology to assess the reliability computation model validity using the concept of Bayesian hypothesis testing, by comparing the model prediction and experimental observation, when there is only one computational model available to evaluate system behavior. Time-independent and time-dependent problems are investigated, with consideration of both cases: with and without statistical uncertainty in the model. The case of time-independent failure probability prediction with no statistical uncertainty is a straightforward application of Bayesian hypothesis testing. However, for the life prediction (time-dependent reliability) problem, a new methodology is developed in this paper to make the same Bayesian hypothesis testing concept applicable. With the existence of statistical uncertainty in the model, in addition to the application of a predictor estimator of the Bayes factor, the uncertainty in the Bayes factor is explicitly quantified through treating it as a random variable and calculating the probability that it exceeds a specified value. The developed method provides a rational criterion to decision-makers for the acceptance or rejection of the computational model. [Copyright &y& Elsevier]

This paper presents a sensor placement optimization framework for the calibration of spatially varying model parameters. To account for the randomness of the calibration parameters over space and across specimens, the spatially varying parameter is represented as a random field. Based on this representation, Bayesian calibration of spatially varying parameter is investigated. To reduce the required computational effort during Bayesian calibration, the original computer simulation model is substituted with Kriging surrogate models based on the singular value decomposition (SVD) of the model response and the Karhunen–Loeve expansion (KLE) of the spatially varying parameters. A sensor placement optimization problem is then formulated based on the Bayesian calibration to maximize the expected information gain measured by the expected Kullback–Leibler (K–L) divergence. The optimization problem needs to evaluate the expected K–L divergence repeatedly which requires repeated calibration of the spatially varying parameter, and this significantly increases the computational effort of solving the optimization problem. To overcome this challenge, an approximation for the posterior distribution is employed within the optimization problem to facilitate the identification of the optimal sensor locations using the simulated annealing algorithm. A heat transfer problem with spatially varying thermal conductivity is used to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper presents an intelligent computational methodology for loose-bolt detection in thermal protection panels, considering uncertainties in sensed data. The proposed methodology is based on the integration of a dynamic artificial neural network, wavelet signal analysis, and Bayesian probabilistic assessment. A dynamic fuzzy wavelet neural-network model is employed to perform the multiple-input/multiple-output nonparametric system identification of the panel using time-series data obtained from the panel under a healthy condition. The trained model is used to predict dynamic responses of the structural system under unknown conditions. Both predicted and sensed-time-history data are decomposed into multiple time-frequency resolutions using a discrete wavelet-packet transform method. The wavelet-packet component energy is computed in terms of the decomposed coefficients and used as a signal feature to detect loose bolts. The effectiveness of the selected features is assessed using both crosscorrelation and cross-coherence metrics. The multivariate comparison in damage detection is handled by an interval-based Bayesian hypothesis-testing approach. The methodology is implemented to detect one loose bolt of a prototype thermal protection system panel with four mechanically bolted joints using experimental data collected at the U.S. Air Force Research Laboratory from seven different sensor configurations. [ABSTRACT FROM AUTHOR]

Model calibration analysis is concerned with the estimation of unobservable modeling parameters using observations of system response. When the model being calibrated is an expensive computer simulation, special techniques such as surrogate modeling and Bayesian inference are often fruitful. In this paper, we show how the flexibility of the Bayesian calibration approach can be exploited to account for a wide variety of uncertainty sources in the calibration process. We propose a straightforward approach for simultaneously handling Gaussian and non-Gaussian errors, as well as a framework for studying the effects of prescribed uncertainty distributions for model inputs that are not treated as calibration parameters. Further, we discuss how Gaussian process surrogate models can be used effectively when simulator response may be a function of time and/or space (multivariate output). The proposed methods are illustrated through the calibration of a simulation of thermally decomposing foam. [ABSTRACT FROM AUTHOR]

In the Kennedy and O'Hagan framework for Bayesian calibration of physics models, selection of an appropriate prior form for the model discrepancy function is a challenging issue due to the lack of physics knowledge regarding model inadequacy. Aiming to address the uncertainty arising from the selection of a particular prior, this paper first conducts a study on possible formulations of the model discrepancy function. A first-order Taylor series expansion-based method is developed to investigate the potential redundancy caused by adding a discrepancy function to the original physics model. Further, we propose a three-step (calibration, validation, and combination) approach in order to inform the decision on the construction of model discrepancy priors. In the validation step, a reliability-based metric is used to evaluate posterior model predictions in the validation domain. The validation metric serves as a quantitative measure of how well the discrepancy formulation captures the missing physics in the model. In the combination step, the posterior distributions of model parameters and discrepancy corresponding to different priors are combined into a single distribution based on the probabilistic weights derived from the validation step. The combined distribution acknowledges the uncertainty in the prior formulation of model discrepancy function. [ABSTRACT FROM AUTHOR]

The poor explainability of deep learning models has hindered their adoption in safety and quality-critical applications. This paper focuses on image classification models and aims to enhance the explainability of deep learning models through the development of an uncertainty quantification-based framework. The proposed methodology consists of three major steps. In the first step, we adopt dropout-based Bayesian neural network to characterize the structure and parameter uncertainty inherent in deep learning models, propagate and represent such uncertainties to the model prediction as a distribution. Next, we employ entropy as a quantitative indicator to measure the uncertainty in model prediction, and develop an Empirical Cumulative Distribution Function (ECDF)-based approach to determine an appropriate threshold value for the purpose of deciding when to accept or reject the model prediction. Secondly, in the cases with high model prediction uncertainty, we combine the prediction difference analysis (PDA) approach with dropout-based Bayesian neural network to quantify the uncertainty in pixel-wise feature importance, and identify the locations in the input image that highly correlate with the model prediction uncertainty. In the third step, we develop a robustness-based design optimization formulation to enhance the relevance between input features and model prediction, and leverage a differential evolution approach to optimize the pixels in the input image with high uncertainty in feature importance. Experimental studies in MNIST and CIFAR-10 image classifications are included to demonstrate the effectiveness of the proposed approach in increasing the explainability of deep learning models. [ABSTRACT FROM AUTHOR]

This paper develops analytical methods for test resource allocation that aid in reducing the uncertainty in the system model prediction for multilevel and multidisciplinary systems. The various component, subsystem, and system-level model predictions; the corresponding inputs and calibration parameters; test data; and model and measurement errors are connected efficiently using a Bayesian network. This provides a unified framework for uncertainty analysis where test data can be integrated along with computational simulations. The Bayesian network is used in an inverse problem where the model parameters of multiple subsystems are calibrated simultaneously. This leads to a decrease in the variance of the model parameters, and hence, in the variance of the overall system performance prediction. An optimization-based procedure is then used for test resource allocation using the Bayesian network, and those tests that can effectively reduce the uncertainty in the system model prediction are identified. The proposed methods are extended to three types of aerospace systems-testing applications: structural dynamics (multilevel), thermally induced vibration/flutter (multidisciplinary), and simplified space telescope mirror (multilevel, multidisciplinary). [ABSTRACT FROM AUTHOR]

Abstract: Accurate characterization and prediction of loading, while properly accounting for uncertainty, are essential for probabilistic fatigue damage prognosis. Three different techniques – rainflow counting, the Markov chain method, and autoregressive moving average (ARMA) modeling – are investigated for stochastic characterization and reconstruction of the fatigue load history. The ARMA method is extended in this paper by introducing random coefficients and probabilistic weights, to account for the uncertainty in the selection of the model, inherent variability in loading, and uncertainty due to sparse data. A continuous model updating approach based on real-time monitoring data is developed and applied to all the three techniques mentioned above. The relation between prediction accuracy and updating interval is evaluated quantitatively. A quantitative model validation approach using Bayesian hypothesis testing is proposed to assess the confidence in load prediction from all the three methods. [Copyright &y& Elsevier]

Abstract: This paper presents a methodology for uncertainty quantification and model validation in fatigue crack growth analysis. Several models – finite element model, crack growth model, surrogate model, etc. – are connected through a Bayes network that aids in model calibration, uncertainty quantification, and model validation. Three types of uncertainty are included in both uncertainty quantification and model validation: (1) natural variability in loading and material properties; (2) data uncertainty due to measurement errors, sparse data, and different inspection results (crack not detected, crack detected but size not measured, and crack detected with size measurement); and (3) modeling uncertainty and errors during crack growth analysis, numerical approximations, and finite element discretization. Global sensitivity analysis is used to quantify the contribution of each source of uncertainty to the overall prediction uncertainty and to identify the important parameters that need to be calibrated. Bayesian hypothesis testing is used for model validation and the Bayes factor metric is used to quantify the confidence in the model prediction. The proposed methodology is illustrated using a numerical example of surface cracking in a cylindrical component. [Copyright &y& Elsevier]

Abstract: This paper presents several efficient statistical inference techniques to calibrate the equivalent initial flaw size (EIFS) of fatigue cracks for mechanical components with complicated geometry and multi-axial, variable amplitude loading. Finite element analysis is used to address the complicated geometry and calculate the stress intensity factors. Multi-modal stress intensity factors due to multi-axial loading are combined to calculate an equivalent stress intensity factor using a characteristic plane approach. During cycle-by-cycle integration of the crack growth law, a Gaussian process surrogate model is used to replace the expensive finite element analysis, resulting in rapid computation. Experimental data (crack size after a particular number of loading cycles) and statistical methods are used to calibrate the EIFS. The methods of least squares and maximum likelihood method are extended to evaluate the entire probability distribution of EIFS. Bayesian techniques are also implemented for this purpose. A fast numerical integration technique is developed as an efficient alternative to the expensive Markov Chain Monte Carlo sampling approach in the Bayesian analysis. An application problem of cracking in a cylindrical structure is used to illustrate the proposed methods. [Copyright &y& Elsevier]

A probabilistic Physics of Failure-based framework for fatigue life prediction of aircraft gas turbine discs operating under uncertainty is developed. The framework incorporates the overall uncertainties appearing in a structural integrity assessment. A comprehensive uncertainty quantification (UQ) procedure is presented to quantify multiple types of uncertainty using multiplicative and additive UQ methods. In addition, the factors that contribute the most to the resulting output uncertainty are investigated and identified for uncertainty reduction in decision-making. A high prediction accuracy of the proposed framework is validated through a comparison of model predictions to the experimental results of GH4133 superalloy and full-scale tests of aero engine high-pressure turbine discs. [ABSTRACT FROM AUTHOR]