Engineered Systems Applications

Risk, Failure & Vulnerability

Probabilistic Risk Assessment (PRA), Failure and Vulnerability Analysis

Risk, Failure & Vulnerability

Engineering risk and failure analysis focuses on predicting the probability of those (presumably rare) failures in an engineered system that can lead to severe damage to the system, injury, loss of life, and/or perhaps damage to the surrounding environment. Vulnerability analysis focuses on identifying (and reducing) the vulnerability of engineered systems to both natural (e.g., weather-related) and man-made (e.g., sabotage, terrorism) disruptions.

These analyses are typically used to inform decisions about required levels of redundancy and other design features, and to evaluate system safety and risk. In these kinds of studies, the output of the analysis is typically the probability of a particular high consequence outcome (e.g., catastrophic failure of the system), and identification of those events or components most likely to lead to that outcome. Based on this information, specific actions can be identified (e.g., design changes or modification of operating procedures) to reduce the risk.

Planetary Orbiter Model

By combining the flexibility of a general-purpose and highly-graphical probabilistic simulation framework with specialized features to support reliability analysis, GoldSim allows you to create quantitative and transparent risk, failure and vulnerability analysis models to allow you to ask "what if" questions regarding various designs and make defensible risk management decisions. GoldSim is flexible and powerful enough to allow you to create a “total system” model that represents the interactions, interdependencies and feedbacks between the various system components (including humans). Without such a model, it may not be possible to identify potential failure mechanisms, fatal flaws or system incompatibilities.

In particular, the GoldSim Reliability Module provides powerful features to support engineering risk and reliability analysis. The Reliability Module can be used to compute the probability of specific consequences (e.g., catastrophic failure of the system). GoldSim catalogs and analyzes failure scenarios, which allows for key sources of unreliability and risk to be identified.



White Papers

Technical Papers

  • Comparative Analysis of Static and Dynamic Probabilistic Risk Assessment

    Journal Article published in the Reliability and Maintainability Symposium (RAMS) – 2015 Annual

    Mattenberger, C. NASA Ames Res. Center, Moffett Field, CA, USA; Mathias, D.L.; Go, S.

    This study compares and contrasts three different approaches for the probabilistic safety assessment of crewed spacecraft: traditional static fault tree; fault tree hybrid, and dynamic Monte Carlo simulation (using GoldSim).

    Read Abstract

  • Engineering Risk Assessment of a Dynamic Space Propulsion System Benchmark Problem

    Reliability Engineering & System Safety – July 2015

    Donovan L. Mathias, Christopher J. Mattenberger, and Susie Go (NASA Ames Research Center)

    The Engineering Risk Assessment (ERA) team at NASA Ames Research Center develops dynamic models with linked physics-of-failure analyses to produce quantitative risk assessments of space exploration missions. This paper applies the ERA approach to the 2014 Probabilistic Safety Assessment and Management conference Space Propulsion System Benchmark Problem, which investigates dynamic system risk for a deep space ion propulsion system over three missions with time-varying thruster requirements and operations schedules. The dynamic missions are simulated using commercial software to generate integrated loss-of-mission (LOM) probability results via Monte Carlo sampling. The simulation model successfully captured all dynamics aspects of the benchmark missions, and convergence studies are presented to illustrate the sensitivity of integrated LOM results to the number of Monte Carlo trials. In addition, to evaluate the relative importance of dynamic modeling, the Ames Reliability Tool (ART) was used to build a series of quasi-dynamic, deterministic models that incorporated varying levels of the problem's dynamics. The ART model did a reasonable job of matching the simulation results for the simpler mission case, while auxiliary dynamic models were required to adequately capture risk-driver rankings for the more dynamic cases. This study highlights how state-of-the-art techniques can adapt to a range of dynamic problems.


  • Simulation Assisted Risk Assessment Applied to Launch Vehicle Conceptual Design

    Reliability and Maintainability Symposium – January 2008

    Donovan L. Mathias, Susie Go, Ken Gee, and Scott Lawrence, NASA Ames Research Center

    This paper describes the application of simulation-based risk assessment to the analysis of abort during the ascent phase of a space exploration mission.


  • Engineering Risk Assessment of Space Thruster Challenge Problem

    Proceedings, Probabilistic Safety Assessment and Management PSAM 12, Honolulu, HI – June 2014

    Donovan L. Mathias, Susie Go, NASA Ames Research Center and Christopher J. Mattenberger, Science and Technology Corp.

    Quantitative risk assessments of space exploration missions were developed by the Engineering Risk Assessment (ERA) team at NASA Ames Research Center, which uses GoldSim's discrete and continuous-time reliability elements. The model applies the ERA approach to the baseline and extended versions of the PSAM Space Thruster Challenge Problem, which investigates mission risk for a deep space ion propulsion system with time-varying thruster requirements and operations schedules. This study highlighted that state-of-the-art techniques can adequately adapt to a range of dynamic problems.


  • An Integrated Reliability and Physics-based Risk Modeling Approach for Assessing Human Spaceflight Systems

    Proceedings, Probabilistic Safety Assessment and Management PSAM 12, Honolulu, HI – June 2014

    Susie Go, Donovan L. Mathias, Scott Lawrence, Ken Gee, NASA Ames Research Center and Christopher J. Mattenberger, Science and Technology Corp

    This paper presents an integrated reliability and physics-based risk modeling approach for assessing human spaceflight systems. The approach is demonstrated using an example, end-to-end risk assessment of a generic crewed space transportation system during a reference mission to the International Space Station. The behavior of the system is modeled using analysis techniques from multiple disciplines in order to properly capture the dynamic time- and state- dependent consequences of failures encountered in different mission phases. This approach facilitates risk-informed design by providing more realistic representation of system failures and interactions; identifying key risk-driving sensitivities, dependencies, and assumptions; and tracking multiple figures of merit within a single, responsive assessment framework that can readily incorporate evolving design information throughout system development.


  • Development of a Dynamic Simulation Approach to Mission Risk and Reliability Analysis

    American Nuclear Society International Topical Meeting on Probabilistic Safety Analysis, PSA 05 – January, 2005

    Ian Miller and Andrew Burns

    This paper describes a NASA-funded project to develop a reliability analysis module for the GoldSim simulation software capable of modeling highly dynamic systems over the duration of the mission, taking into account variation in input parameters and the evolution of the system. To illustrate the approach, two NASA examples that have previously been evaluated using classical PRA approaches were developed using the simulation-based approach. Issues surrounding the translation of the classical PRA models into a simulation-based approach are discussed, and areas where the simulation-based approach provided additional insights into the system behavior are highlighted.


  • A System Model for Geologic Sequestration of Carbon Dioxide

    Article in Environmental Science and Technology, Volume 43, Number 3, pgs. 565-570 – December 2008

    Philip Stauffer, Hari Viswanathan, Rajesh Pawar and George Guthrie, Los Alamos National Laboratory

    This article describes the CO2-PENS model developed to simulate capture, transport and injection in different geological reservoirs.


    Read Paper

  • Vulnerability Assessment to Support Integrated Water Resources Management of Metropolitan Water Supply Systems

    Journal of Water Resources Planning and Management, DOI: 10.1061/(ASCE)WR.1943-5452.0000738. © 2016 American Society of Civil Engineers – November 2016

    Erfan Goharian, Steven J. Burian, Jason Lillywhite, and Ryan Hile

    The combined actions of natural and human factors change the timing and availability of water resources and, correspondingly, water demand in metropolitan areas. This leads to an imbalance between supply and demand resulting in increased vulnerability of water supply systems. Accordingly, methods for systematic analysis and multifactor assessment are needed to estimate the vulnerability of individual components in an integrated water supply system. This paper introduces a new approach to comprehensively assess vulnerability by integrating water resource system characteristics with factors representing exposure, sensitivity, severity, potential severity, social vulnerability, and adaptive capacity. The effectiveness and advantages of the proposed approach are checked using an investigation of the water supply system of Salt Lake City (SLC), Utah. First, an integrated water resource model was developed using GoldSim to allocate water from different sources in SLC among designated demand points. The model contains individual simulation modules with representative interconnections among the natural hydroclimate system, built water infrastructure, and institutional decision making. The results of the analysis illustrate that basing vulnerability on a sole factor may lead to insufficient understanding and, hence, inefficient management of the system. The new vulnerability index and assessment approach was able to identify the most vulnerable water sources in the SLC integrated water supply system. In conclusion, use of a more comprehensive approach to simulate the system behavior and estimate vulnerability provides more guidance for decision makers to detect vulnerable components of the system and ameliorate decision making.


  • A probabilistic framework for comparison of dam breach parameters and outflow hydrograph generated by different empirical prediction methods

    Journal of Environmental Modelling and Software, Vol. 86, Pgs. 248–263, 1364-8152/© 2016 Elsevier Ltd. – December 2016

    Ebrahim Ahmadisharaf, Alfred J. Kalyanapu, Brantley A. Thames, Jason Lillywhite

    This study presents a probabilistic framework to simulate a dam breach and evaluates the impact of using four empirical dam breach prediction methods on breach parameters (i.e., geometry and timing) and outflow hydrograph attributes (i.e., time to peak, hydrograph duration and peak). Mean values and percentiles of breach parameters and outflow hydrograph attributes are compared for hypothetical overtopping failure of Burnett Dam in the state of North Carolina, USA. Furthermore, utilizing the probabilistic framework of GoldSim, the least and most uncertain methods alongside those giving the most critical value are identified for these parameters. The multivariate analysis also indicates that lone use of breach parameters is not necessarily sufficient to characterize outflow hydrograph attributes. However, timing characteristic of the breach is generally a more important driver than its geometric features.


  • Incorporating Potential Severity into Vulnerability Assessment of Water Supply Systems under Climate Change Conditions

    Journal of Water Resources Planning and Management, 2015DOI: 10.1061/ (ASCE)WR.1943-5452.0000579. © 2015 American Society of Civil Engineers. – November 2015

    Erfan Goharian, S.M.ASCE; Steven J. Burian, M.ASCE; Courtenay Strong with Univ. of Utah; Tim Bardsley with Western Water Assessment

    In response to climate change, vulnerability assessment of water resources systems is typically performed based on quantifying the severity of the failure. This paper introduces an approach to assess vulnerability that incorporates a set of new factors. The method is demonstrated with a case study of a reservoir system in Salt Lake City using an integrated modeling framework composed of a hydrologic model and a systems model driven by temperature and precipitation data for a 30-year historical (1981–2010) period. The climate of the selected future (2036–2065) simulation periods were represented by five combinations of warm or hot, wet or dry, and central tendency projections derived from the World Climate Research Programme's (WCRP's) Coupled Model Intercomparison Project Phase 5. The results of the analysis illustrate that basing vulnerability on severity alone may lead to an incorrect quantification of the system vulnerability. In this study, a typical vulnerability metric (severity) incorrectly provides low magnitudes under the projected future warm-wet climate condition. The proposed new metric correctly indicates the vulnerability to be high because it accounts for additional factors. To further explore the new factors, a sensitivity analysis (SA) was performed to show the impact and importance of the factors on the vulnerability of the system under different climate conditions. The new metric provides a comprehensive representation of system vulnerability under climate change scenarios, which can help decision makers and stakeholders evaluate system operation and infrastructure changes for climate adaptation.


  • Using GoldSim for Joint Probability Assessment of Closure Times on Linear Infrastructure

    Visions to Realities - Stormwater Queensland Conference proceedings, Noosa – June 2014

    E. Symons, C. Gimber, Kellogg Brown and Root Pty Ltd

    Flooding of major regional roads and rail corridors severely disrupts transport operations including the export of mined minerals from central and north Queensland which contribute heavily to the Australian economy. It is important for proponents developing new infrastructure and operators of existing infrastructure to understand annual closure times resulting from flooding. Long linkages of road or rail that cross a number of catchment basins and a large number of drainage lines can be difficult to assess due to spatial variation, moving storms and concurrent storms. The objective of this paper is to create a simple methodology, using joint probability, to quantitatively assess the closure time along linear infrastructure. For this paper, GoldSim was used to represent the road or rail system.


  • Probabilistic Simulation of Large Dam Operations for Water Supply and Flood Control

    Australian National Committee on Large Dams (ANCOLD) Conference – 2014

    A flood operations model was built using GoldSim to facilitate 3 different studies to help better understand and fine tune operations of a large dam used for water supply and flood control. GoldSim was used to probabilistically simulate thousands of flood scenarios, which made it possible to compare different operating schemes under many possible conditions. The approach was found to be valuable to understand the capacity of the dams to mitigate floods while protecting water supplies. The studies identified shortcomings in the conventional design event approach to flood estimation. A broader range of stochastic floods provided an advantage to better assess flood mitigation performance and extreme floods, which is important for dam safety. Papers describing these studies were presented at the Australian National Committee on Large Dams (ANCOLD) conference in 2014



  • Probabilistic Analysis to Evaluate the Effects of Dam Breach Methodologies on Downstream Flood Hazard

    World Environment and Water Resources Congress Conference Proceeding Paper – June 2014

    Alfred Kalyanapu, Ebrahim Ahmadisharaf, Brantley A. Thames, Tennessee Tech University and Jason Lillywhite with GoldSim Technology Group

    The objective of this study is to perform a probabilistic dam breach analysis and evaluate impacts of using four different dam breach methods on downstream flood hazards in Swannanoa River watershed.

    Read Abstract

  • Risk Assessment for Unbound Granular Material Performance in Rural Queensland Pavements

    Master's Thesis – 2006

    Meera Creagh, University of Queensland

    This thesis describes the use of GoldSim to evaluate different material choices for road-building projects.

    Read Abstract

  • Predicting Risks in the Earth Sciences: Volcanological Examples

    Los Alamos Science, Number 29, pgs. 56-69 – 2005

    Greg Valentine, Los Alamos National Laboratory

    This article describes the process of volcanological risk assessment, including describing how this is modeled, using GoldSim, within the Yucca Mountain Total System Performance Assessment.


  • Comparison of Uncertainty and Sensitivity Analyses Methods Under Different Noise Levels

    Presentation, PSAM12: Probabilistic Safety Assessment & Management Conference – June 2014

    David Esh and Christopher Grossman, US Nuclear Regulatory Commission

    Uncertainty and sensitivity analyses are an integral part of probabilistic assessment methods used to evaluate the safety of a variety of different systems. In many cases the systems are complex, information is sparse, and resources are limited. Models are used to represent and analyze the systems. To incorporate uncertainty, the developed models are commonly probabilistic. Uncertainty and sensitivity analyses are used to focus iterative model development activities, facilitate regulatory review of the model, and enhance interpretation of the model results. A large variety of uncertainty and sensitivity analyses techniques have been developed as modeling has advanced and become more prevalent. This paper compares the practical performance of six different uncertainty and sensitivity analyses techniques over ten different test functions under different noise levels. In addition, insights from two real-world examples are developed.


  • Uncertainty Analysis for Unprotected Loss-of-Heat-Sink, Loss-of-Flow, and Transient-Overpower Events in Sodium-Cooled Fast Reactors

    International Conference on Fast Reactors and Related Fuel Cycles (FR 2009), Kyoto, Japan – December 2009

    Morris, E. E. and Nutt, W. M., Argonne National Laboratory

    While the traditional approach to reactor safety analyses remain deterministic, this paper considers a stochastic approach for explicitly including uncertainty in safety parameters by applying Monte Carlo sampling coupled with established deterministic reactor safety analysis tools.


Learn More