Search

A systematic approach to defining margin in a manner that incorporates statistical information and accommodates data uncertainty but does not require assumptions about specific forms of the tails of distributions is developed. A margin that is insensitive to the character of the tails of the relevant distributions (tail insensitive margin, TIM) is defined. This is complemented by the calculation of probability of failure (PoF) where the load distribution is augmented by a quantity equal to the TIM. This approach avoids some of the perplexing results common to traditional reliability theory where, on the basis of very small amounts of data, one is led to extraordinary claims of infinitesimal probability of failure. Additionally, this approach permits a more meaningful separation of statistical and engineering issues.

Here, performance assessment of complex systems is ideally accomplished through system-level testing, but because they are expensive, such tests are seldom performed. On the other hand, for economic reasons, data from tests on individual components that are parts of complex systems are more readily available. The lack of system-level data leads to a need to build computational models of systems and use them for performance prediction in lieu of experiments. Because their complexity, models are sometimes built in a hierarchical manner, starting with simple components, progressing to collections of components, and finally, to the full system. Quantification of uncertainty in the predicted response of a system model is required in order to establish confidence in the representation of actual system behavior. This paper proposes a framework for the complex, but very practical problem of quantification of uncertainty in system-level model predictions. It is based on Bayes networks and uses the available data at multiple levels of complexity (i.e., components, subsystem, etc.). Because epistemic sources of uncertainty were shown to be secondary, in this application, aleatoric only uncertainty is included in the present uncertainty quantification. An example showing application of the techniques to uncertainty quantification of measures of response of amore » real, complex aerospace system is included.« less

This article specifies a virtual structural dynamic subsystem and some systems, as well as mathematical models that approximately simulate them. The purpose is to define a setting for model validations conceived by participants in the Sandia National Laboratories Validation Workshop. Some broad guidelines for the model validations are set, as well as a regulatory requirement to be satisfied by the target system. Participants are directed to present techniques for subsystem model validation, to analytically predict whether or not the regulatory requirement will be met, and to estimate the likelihood of accuracy of the prediction.

The six papers in this special issue that develop solutions to the Structural Dynamics Challenge Problem are summarized herein. The goal is to emphasize different tools and approaches applied to various parts of the structural dynamics problem. Specifically the following issues are considered: (1) Development of a mathematical framework for uncertainty quantification of a substructure. (2) Calibration of a mathematical model for the substructure. (3) Validation of the substructure mathematical model. (4) Use of the mathematical model of the substructure, in conjunction with another structure, to make a prediction of the probability that a regulatory limit is surpassed, and discussion of the uncertainty of the prediction. Different methodologies are presented and specific results vary, however, conclusions regarding satisfaction of the regulatory requirement match.

This special issue presents the results of the Sandia organized Model Validation Challenge Workshop, held May 2006. The workshop brought together researchers from different fields to present various approaches to model validation, and focused on the methodological elements of model validation rather than on model building. Three problems were defined in the disciplines of structural statics, structural dynamics, and heat transfer, all with a uniform structure. The workshop was specifically designed to investigate the relative merits of different approaches to hierarchal model validation through application to these problems. This paper describes a hierarchal approach in the challenge problems, presents the uniform conceptual framework that was used for the challenge problem definitions, and provides an overview of the organization of this special issue.

It is now commonplace for engineers to build mathematical models of the systems they are designing, building, or testing. And, it is nearly universally accepted that phenomenological models of physical systems must be validated prior to use for prediction in consequential scenarios. Yet, there are certain situations in which testing only or no testing and no modeling may be economically viable alternatives to modeling and its associated testing. This paper develops an economic framework within which benefit–cost can be evaluated for modeling and model validation relative to other options. The development is presented in terms of a challenge problem. We provide a numerical example that quantifies when modeling, calibration, and validation yield higher benefit–cost than a testing only or no modeling and no testing option.

In this discussion paper, we explore different ways to assess the value of verification and validation (V&V) of engineering models. We first present a literature review on the value of V&V and then use value chains and decision trees to show how value can be assessed from a decision maker's perspective. In this context, the value is what the decision maker is willing to pay for V&V analysis with the understanding that the V&V results are uncertain. The 2014 Sandia V&V Challenge Workshop is used to illustrate these ideas.

Interest in the analysis of random vibrations of mechanical systems started to grow about a half century ago in response to the need for a theory that could accurately predict structural response to jet engine noise and missile launch-induced environments. However, the work that enabled development of the theory of random vibrations started about a half century earlier. This paper discusses contributions to the theory of random vibrations from the time of Einstein to the time of an MIT workshop that was organized by Crandall in 1958.

There is an increasing reliance in the engineering community on the use of mathematical models to characterize physical system behavior. This is happening even though mathematical models rarely simulate real system behavior perfectly. Due to this reliance, we require objective, well-founded mathematical techniques for model validation. This paper develops a formal approach to the validation of mathematical models of structural dynamics systems. It uses a probabilistic/statistical approach to the characterization of an important measure of behavior of dynamic systems subjected to random excitations, and seeks to validate a mathematical model in a statistical sense. An example is presented.

In recent years, great progress has been made in the construction and solution of large finite element models of complex structural dynamic systems. For example, structural models with millions of degrees of freedom are being built and used to approximate responses of structural systems. Further, great progress is being made in stochastic system analysis. Techniques for the construction of stochastic system models have been developed and solution techniques proposed. However, the two areas have not been combined, on a large scale, because stochastic finite element approaches appear very intrusive in their pure form. That is, substantial modifications of deterministic finite element codes are required to accommodate stochastic analysis. In view of this, a technique that uses the techniques of stochastic finite elements in a non-intrusive manner is required. This research provides one such approach. Specifically, the problem is divided into three parts: (1) model structural dynamic excitations using traditional approaches, and model physical system randomness using techniques of stochastic finite elements, namely, the Karhunen-Loeve expansion and polynomial chaos; (2) generate stochastic structural realizations and realizations of the random excitation using a Monte Carlo approach, and analyze structural responses with parallel computation in a suitable, large-scale finite element code; and (3) analyze structural dynamic responses using the techniques of stochastic finite elements, namely, the Karhunen-Loeve expansion and polynomial chaos. This paper supplies the details of the analytical approach. A numerical example is presented.

Sandia National Laboratories has been studying calendar and pulse discharge life of prototype high-power lithium-ion cells as part of the Advanced Technology Development (ATD) Program. One of the goals of ATD is to establish validated accelerated life test protocols for lithium-ion cells in the hybrid electric vehicle application. In order to accomplish this, aging experiments have been conducted on 18650-size cells containing a chemistry representative of these high-power designs. Loss of power and capacity are accompanied by increasing interfacial impedance at the cathode. These relationships are consistent within a given state-of-charge (SOC) over the range of storage temperatures and times. Inductive models have been used to construct detailed descriptions of the relationships between power fade and aging time and to relate power fade, capacity loss and impedance rise. These models can interpolate among the different experimental conditions and can also describe the error surface when fitting life prediction models to the data.

Sandia National Laboratories has conducted a sequence of studies on the performance of lithium ion and other types of electrochemical cells using inductive models. The objectives of some of these investigations are: (1) to develop procedures to rapidly determine performance degradation rates while these cells undergo life tests; (2) to model cell voltage and capacity in order to simulate cell output under variable load and temperature conditions; (3) to model rechargeable battery degradation under conditions of cyclic charge/discharge, and many others. Among the uses for the models are: (1) to enable efficient predictions of battery life; (2) to characterize system behavior. Inductive models seek to characterize system behavior using experimentally or analytically obtained data in an efficient and robust framework that does not require phenomenological development. There are certain advantages to this. Among these advantages is the ability to avoid making measurements of hard to determine physical parameters or having to understand cell processes sufficiently to write mathematical functions describing their behavior. We have used artificial neural networks (ANNs) for inductive modeling, along with ancillary mathematical tools to improve their accuracy. This paper summarizes efforts to use inductive tools for cell and battery modeling. Examples of numerical results are presented.

Probabilistic neural nets have been applied in the detection of structural damage. These networks, which rely upon approximating the multivariate density of the training data, have been shown to be effective in some applications. However, quantification of decision errors, which must ultimately be used to rank their effectiveness, has received little attention. In this paper, a two-dimensional probabilistic pattern classifier (PPC) based on an L 2 detector is studied. Each dimension is modelled as a Gaussian random variable in order to directly study the effects introduced by a commonly used density estimator. The region of acceptance is examined for different parameters of the kernel density estimator. The detector behaviour of the original PPC is compared to theory. It is demonstrated that performance is affected by errors introduced by a kernel constructed using a finite set of data, and also from theoretical limitations inherent in the test statistic. A modification of the test statistic is suggested to improve the sensitivity for structural damage detection.

We investigated the reliability of a rechargeable battery acting as the energy storage component in a photovoltaic power supply system. A model system was constructed for this that includes the solar resource, the photovoltaic power supply system, the rechargeable battery and a load. The solar resource and the system load are modelled as stochastic processes. The photovoltaic system and the rechargeable battery are modelled deterministically, and an artificial neural network is incorporated into the model of the rechargeable battery to simulate damage that occurs during deep discharge cycles. The equations governing system behaviour are solved simultaneously in the Monte Carlo framework, and a first passage problem is solved to assess system reliability.

Model verification and validation (V&V) is an enabling methodology for the development of numerical models that can be used to make predictions with quantified accuracy and confidence. Model V&V procedures are needed to reduce the time, cost and risk associated with component and full-scale testing of engineered systems. Quantifying the confidence and predictive accuracy of model calculations provides the decision-maker with the information necessary for making high-consequence decisions. One of the key steps in a model validation project is to compare model predictions and experimental data given uncertainties in both. This step is critical because it defines how the model will be challenged against the physical data relative to the intended use of the model. This paper briefly reviews the main concepts involved in model V&V, introduces a simple measure termed the z-metric, and compares the z-metric with another measure called the area metric.

A systematic approach to defining margin in a manner that incorporates statistical information and accommodates data uncertainty, but does not require assumptions about specific forms of the tails of distributions is developed. This approach extends to calculations underlying validation assessment and quantitatively conservative predictions.

The objective of this paper is to present a framework for modeling and simulation of parameter vector random processes. While the approach described in the paper can be applied generally, we are interested in problems where there is a limited set of random process data (e.g., FRFs or IRFs) for which the dominant sources of randomness (i.e., the dominant random parameters) are not known a-priori. From this limited set of random process data we aim to create a framework for estimating the underlying distribution to enable simulation and analysis of the family of systems from which the original data set came. The approach recommended for modeling random parameter vectors for physical systems is the Karhunen-Loeve expansion. Our innovation is to use additional knowledge (constraints) about the system that spawned the random process in a screening that classifies the synthetic realizations as plausible or implausible and yields a more representative data set.

Metrics for determining the stability- and performance-robustness of a given controller to variations in open-loop dynamics are examined. Of particular interest are the probability distributions that these metrics have given some amount of uncertainty in the plant. The analysis herein characterizes these distributions using frequency response data collected experimentally for the open-loop plant. In doing so, it avoids using a model for the plant, thereby reducing errors and uncertainty inherent to the modeling of a plant. For a proposed controller, a Monte-Carlo method is used to evaluate the closed-loop performance and stability metrics for a large ensemble of test data and thus create an estimate of their distributions. Since ensembles of test data are typically quite small, a Karhunen-Loeve expansion based method is used to synthetically expand the set of FRFs.

Modeling and simulation to aid in decision making, such as qualification of high consequence systems, is becoming more prevalent. With this increased use of models, there arises a need to formally assess the accuracy of the model underlying these decisions. This process is referred to as model validation. Furthermore an understanding of the uncertainty in the model responses is necessary if the model is used to predict the response of internal system components for establishing their performance. These responses, presumably, cannot be directly measured and/or multiple pieces of hardware are not available to quantify their inherent variability and thus the value of the model is heightened. Additionally, requirements on component assessment that require quantification of margins, in the presence of uncertainty, provide yet another motivation to use modeling and simulation. In this paper, we describe a model validation process to assess the accuracy of a system level model and perform an assessment of the model relative to existing data.

Practically all mechanical structures behave nonlinearly, to some extent, yet most laboratory models for mechanical structures are linear, and most models used for prediction and design are constructed in the linear nite element (FE) framework. This work rst assesses the nonlinearity of the mechanical structure by comparing the structure response to a best-t limear model response obtained from experimental data. Assessment metrics are developed for determining if the system is strongly or weakly nonlinear during this comparison. These same metrics can then be used to validate the use of a linear model.

Linear mathematical models frequently provide good approximations to the input-output relations for real systems. However, ensembles of systems that are nominally identical cannot, usually, be adequately represented with a single model because real systems are stochastic. The randomness in real systems must be modeled if the randomness bears on critical behaviors of the system. The behavior of linear systems can be represented in parametric or non-parametric form; the latter framework is used, here. Among the frameworks available for characterization of system behavior, we choose the frequency response function (FRF). We choose to work with the FRF because many system attributes can be interpreted by inspection of the FRF, and it can be used directly for control design. This paper improves a previously developed Karhunen-Loeve expansion (KLE) representation for linear system behavior based on FRF data. The improvement yields a compact representation of the uncertainty inherent in an ensemble of systems and avoids the introduction of unwanted features in the system representation. This non-parametric, compact representation of the distribution of linear systems can then be used to characterize the performance and stability of a given feedback control law, as well as for control law design.

Numerous investigations have studied the potential for chaotic vibrations of nonlinear systems. It has been shown for many simple nonlinear systems, that when they are excited severely enough, or with the appropriate parametric combinations, that they will execute chaotic vibrations. The present investigation considers the potential for the occurrence of chaos in a practical nonlinear system -- the isolated accelerometer. A simple, first order model is proposed for the isolated accelerometer, and it is shown that chaos can occur in the isolated accelerometer. A preliminary investigation into the bearing that this chaos potential has on the measurement of shock response is summarized. 7 refs.

Linear systems are good approximations of the input-output relationship for many real systems. In practice, ensembles of nominally identical systems can seldom be adequately represented by a single linear system. Some accommodation needs to be made for representing inherent ensemble randomness. One approach would be to use a parameterized model of the system, with the random nature captured in discrete parameters of the parameterized model. Another approach is to use data measured from each system in the ensemble to represent the random nature of the ensemble in a non-parametric form. This is the approach described in this paper. There are several different types of data that could be collected from each system in the ensemble, each type capable of capturing the input-output behavior of the linear system: input-output time domain data, perhaps with specific excitation sequences [1,2]; Markov parameters [2,3]; and Frequency Response Functions (FRFs) [2] to name a few. We choose to work with FRF data because many system attributes can be easily interpreted by inspection of the FRF [4]. FRF data can also be used directly for control design [5,6,7]. This paper develops a Karhunen-Loeve expansion [8] representation for linear system behavior based on FRF data to develop a compact representation of the uncertainty inherent in an ensemble of systems. This non-parametric, compact, representation of the distribution of linear systems can then be used to characterize the performance and stability of a given feedback control law, as well as for control law design [5,6,7].

Sandia National Laboratories currently utilizes two laser tracking systems to provide time-space-position-information (TSPI) and high speed digital imaging of test units under flight. These laser trackers have been in operation for decades under the premise of theoretical accuracies based on system design and operator estimates. Advances in optical imaging and atmospheric tracking technology have enabled opportunities to provide more precise six degree of freedom measurements from these trackers. Applying these technologies to the laser trackers requires quantified understanding of their current errors and uncertainty. It was well understood that an assortment of variables contributed to laser tracker uncertainty but the magnitude of these contributions was not quantified and documented. A series of experiments was performed at Sandia National Laboratories large centrifuge complex to quantify TSPI uncertainties of Sandia National Laboratories laser tracker III. The centrifuge was used to provide repeatable and economical test unit trajectories of a test-unit to use for TSPI comparison and uncertainty analysis. On a centrifuge, testunits undergo a known trajectory continuously with a known angular velocity. Each revolution may represent an independent test, which may be repeated many times over for magnitudes of data practical for statistical analysis. Previously these tests were performed at Sandia's rocket sled track facility but were found to be costly with challenges in the measurement ground truth TSPI. The centrifuge along with on-board measurement equipment was used to provide known ground truth position of test units. This paper discusses the experimental design and techniques used to arrive at measures of laser tracker error and uncertainty.

Verification and validation (V&V) are processes designed to assure, with quantifiable confidence, that the computational models we use, solve the problems they are meant to solve, and that the models adequately represent the structures they are meant to model. The American Society of Mechanical Engineers (ASME) Standards Committee on Verification and Validation in Computational Solid Mechanics (PTC 60/V&V 10) has approved a Guide for performing V&V [1]. The Guide is brief, and does not provide specific methods for the performance of model V&V, therefore, subcommittees have been formed to develop documents specifying methods for accomplishing the various activities of V&V. One of these subcommittees was formed to develop the “Uncertainty Quantification Supplement to the ASME V&V 10-2006 Guide.” The talks in this special session describe some of the activities associated with uncertainty quantification in V&V. This paper presents an example used to illuminate some of the tools and measures developed in the other papers in this sequence. The example describes experimental results and model predictions.

Epistemic uncertainty, characterizing lack-of-knowledge, is often prevalent in engineering applications. However, the methods we have for analyzing and propagating epistemic uncertainty are not as nearly widely used or well-understood as methods to propagate aleatory uncertainty (e.g. inherent variability characterized by probability distributions). In this paper, we examine three methods used in propagating epistemic uncertainties: interval analysis, Dempster-Shafer evidence theory, and second-order probability. We demonstrate examples of their use on a problem in structural dynamics, specifically in the assessment of margins. In terms of new approaches, we examine the use of surrogate methods in epistemic analysis, both surrogate-based optimization in interval analysis and use of polynomial chaos expansions to provide upper and lower bounding approximations. Although there are pitfalls associated with surrogates, they can be powerful and efficient in the quantification of epistemic uncertainty.

We present a proportional hazards model (PHM) that establishes a framework suitable for performing reliability estimates and risk prognostics on complex multi-component systems which are transferred at arbitrary times among a discrete set of non-stationary stochastic environments. Such a scenario is not at all uncommon for portable and mobile systems. It is assumed that survival data, possibly interval censored, is available at several “typical” environments. This collection of empirical survival data forms the foundation upon which the basic effects of selected covariates are incorporated via the proportional hazards model. Proportional hazards models are well known in medical statistics, and can provide a variety of data-driven risk models which effectively capture the effects of the covariates. The paper describes three modifications we have found most suitable for this class of systems: development of suitable survival estimators that function well under realistic censoring scenarios, our modifications to the PHM which accommodate time-varying stochastic covariates, and implementation of said model in a non-linear network context which is itself model-free. Our baseline hazard is a parameterized reliability model developed from the empirical reliability estimates. Development of the risk score for arbitrary covariates arising from movement among different random environments is through interaction of the non-linear network and training data obtained from a Markov chain simulation based on stochastic environmental responses generated from Karhunen-Loeve models.Copyright © 2005 by ASME

The constitutive behavior of mechanical joints is largely responsible for the energy dissipation and vibration damping in weapons systems. For reasons arising from the dramatically different length scales associated with those dissipative mechanisms and the length scales characteristic of the overall structure, this physics cannot be captured adequately through direct simulation of the contact mechanics within a structural dynamics analysis. The only practical method for accommodating the nonlinear nature of joint mechanisms within structural dynamic analysis is through constitutive models employing degrees of freedom natural to the scale of structural dynamics. This document discusses a road-map for developing such constitutive models.

We frequently develop mathematical models of system behavior and sometimes use test data to help identify the parameters of the mathematical model. However, no general purpose technique exists for formally, statistically judging the quality of a model. The paper suggests a formal statistical procedure for the validation of mathematical models of systems when data taken during operation of the system are available. The statistical validation procedure is based on the bootstrap, and it seeks to build a framework where a statistical test of hypothesis can be run to determine whether or not a mathematical model is an acceptable model of a system with regard to user specified measures of system behavior. A numerical example is presented to demonstrate the application of the technique.

Mathematical models of physical systems are used, among other purposes, to improve our understanding of the behavior of physical systems, predict physical system responses, and control the responses of systems. Phenomenological models are frequently used to simulate system behavior, but an alternative is available-the artificial neural network (ANN). We demonstrate that a particular ANN, the connectionist normalized linear spline (CNLS) network, can be used to simulate mechanical system behavior accurately and efficiently. The system simulations are performed in the non recurrent and recurrent frameworks. The paper provides brief discussions on the operation of ANNs in general and the CNLS net in particular, the operation of two different types of mechanical systems, and approaches to the solution of some special problems that occur in connection with ANN simulation of physical system response. Numerical examples are presented.

The objective of this project is to develop a working model of a renewable energy generation/rechargeable battery storage system so these components can be included in calculations to estimate the reliability of the United States electricity grid system under different outage scenarios. This modeling capability will allow the reliability benefit of renewable generation to be quantified as well as providing a tool to optimize the design of these systems. Our work has focused on a specific system comprised of a photovoltaic generator and a lead-acid battery. We present in this report the various methods that were investigated for predicting the solar resource, the development of a robust damage model for the lead-acid battery and the translation of the existing MATLAB code to C and its subsequent integration into a power- flow calculation.

This paper describes the application of directed genetic programming to explore the functional form of non-linear restoring forces that arise due to micro-slip friction. Data are obtained from an experiment to investigate the friction forces in a simple shear interface subjected to a vibration environment. A candidate functional form is proposed to model this friction, and a genetic search is then performed which seeks optimal parameters for the function. The resulting parameters provide the best fit of the proposed functional to the experimental data.