Monte Carlo Simulation

|

What is the Monte Carlo method?

In general terms, the Monte Carlo method (or Monte Carlo simulation) can be used to describe any technique that approximates solutions to quantitative problems through statistical sampling. As used here, 'Monte Carlo simulation' is more specifically used to describe a  method for propagating (translating) uncertainties in model inputs into uncertainties in model outputs (results). Hence, it is a type of simulation that explicitly and quantitatively represents uncertainties. Monte Carlo simulation relies on the process of explicitly representing uncertainties by specifying inputs as probability distributions.  If the inputs describing a system are uncertain, the prediction of future performance is necessarily uncertain. That is, the result of any analysis based on inputs represented by probability distributions is itself a probability distribution.

Whereas the result of a single simulation of an uncertain system is a qualified statement ("if we build the dam, the salmon population could go extinct"), the result of a probabilistic (Monte Carlo) simulation is a quantified probability ("if we build the dam, there is a 20% chance that the salmon population will go extinct"). Such a result (in this case, quantifying the risk of extinction) is typically much more useful to decision-makers who utilize the simulation results.

In order to compute the probability distribution of predicted performance, it is necessary to propagate (translate) the input uncertainties into uncertainties in the results. A variety of methods exist for propagating uncertainty.  Monte Carlo simulation is perhaps the most common technique for propagating the uncertainty in the various aspects of a system to the predicted performance.

In Monte Carlo simulation, the entire system is simulated a large number (e.g., 1000) of times. Each simulation is equally likely, referred to as a realization of the system. For each realization, all of the uncertain parameters are sampled (i.e., a single random value is selected from the specified distribution describing each parameter). The system is then simulated through time (given the particular set of input parameters) such that the performance of the system can be computed. This results is a large number of separate and independent results, each representing a possible “future” for the system (i.e., one possible path the system may follow through time). The results of the independent system realizations are assembled into probability distributions of possible outcomes. As a result, the outputs are not single values, but probability distributions.

 A Simple Example: Rolling Dice

As a simple example of a Monte Carlo simulation, consider calculating the probability of a particular sum of the throw of two dice (with each die having values one through six). In this particular case, there are 36 combinations of dice rolls:

Dice
Source: http://hyperphysics.phy-astr.gsu.edu/hbase/math/dice.html

Based on this, you can manually compute the probability of a particular outcome.  For example, there are six different ways that the dice could sum to seven.  Hence, the probability of rolling seven is equal to 6 divided by 36 = 0.167.

Instead of computing the probability in this way, however, we could instead throw the dice a hundred times and record how many times each outcome occurs.  If the dice totaled seven 18 times (out of 100 rolls), we would conclude that the probability of rolling seven is approximately 0.18 (18%). Obviously, the more times we rolled the dice, the less approximate our result would be. Better than rolling dice a hundred times, we can easily use a computer to simulate rolling the dice 10,000 times (or more). Because we know the probability of a particular outcome for one die (1 in 6 for all six numbers), this is simple.  The output of 10,000 realizations (using GoldSim software):

Two Dice

 How Accurate are the Results?

The accuracy of a Monte Carlo simulation is a function of the number of realizations. That is, the confidence bounds on the results can be readily computed based on the number of realizations.  The two examples below show the 5% and 95% confidence bounds on the value for each outcome (i.e., there is a 90% chance the the true value lies between the bounds): 

100 Realizations       10,000 Realizations

History of the Monte Carlo Method

Monte Carlo simulation was named after the city in Monaco (famous for its casino) where games of chance (e.g., roulette) involve repetitive events with known probabilities.  Although there were a number of isolated and undeveloped applications of Monte Carlo simulation principles at earlier dates, modern application of Monte Carlo methods date from the 1940s during work on the atomic bomb. Mathematician Stanislaw Ulam is credited with recognizing how computers could make Monte Carlo simulation of complex systems feasible:

The first thoughts and attempts I made to practice [the Monte Carlo Method] were suggested by a question which occurred to me in 1946 as I was convalescing from an illness and playing solitaires. The question was what are the chances that a Canfield solitaire laid out with 52 cards will come out successfully? After spending a lot of time trying to estimate them by pure combinatorial calculations, I wondered whether a more practical method than “abstract thinking” might not be to lay it out say one hundred times and simply observe and count the number of successful plays. This was already possible to envisage with the beginning of the new era of fast computers, and I immediately thought of problems of neutron diffusion and other questions of mathematical physics, and more generally how to change processes described by certain differential equations into an equivalent form interpretable as a succession of random operations. Later … [in 1946, I] described the idea to John von Neumann, and we began to plan actual calculations.

Eckhardt, Roger (1987). Stan Ulam, John von Neumann, and the Monte Carlo method, Los Alamos Science, Special Issue (15), 131-137


 

Making Better Decisions In An Uncertain World

Introduction