An Application of BlockSims Log of Simulations
The need for understanding the risk of certain
assumptions or predictions has
resulted in the use of
variability metrics in multiple disciplines, from the quality
control field to financial applications. The most widely used
metric to express variability is standard deviation (or variance, the square of the standard deviation). Given the
expected value of a metric and a standard deviation, one can obtain the metric of interest at any confidence level (for more
information on confidence bounds see [1]), sometimes expressed
in terms of the risk level α.
However, this assumes that the metric in question follows a
normal distribution (the well-known, symmetric, bell-shaped curve
[2]). In reality, many measures do not exhibit such behavior.
This article will present a simple example of an asymmetric
estimate using system availability and
BlockSim 7. The
simulation log feature of BlockSim 7 will then be used for
further analysis of the mean availability via a non-parametric
approach followed by a parametric approach in order to obtain
bounds.
Example
Assume a system with two main
failure modes with the properties presented in Table 1. We wish to
estimate the mean availability for a mission time of 8760 hours.
After running 1000 simulations with the settings shown in
Figure 1, the simulation results at the system level are shown
in Figure 2.
Table 1
Failure Mode Properties
|
Failure Mode |
Failure Distribution |
Parameter 1 |
Parameter 2 |
MTTR |
|
Failure Mode 1 |
Weibull |
β
= 1.5 |
η
= 1,000 hrs |
240
hrs |
|
Failure Mode 2 |
Lognormal |
log-μ
= 10 |
log-σ
= 5 |
Non-Repairable |
Figure 1: Simulation Settings
Figure 2: Simulation Results,
System Overview
The simulation results in
BlockSim show a variety of results both at the system and block
level (see [3] for more information on
simulation results). In
this example, the metric of interest is the mean availability,
defined as follows:
For this example, the mean
availability is equal to 0.529. In addition, the standard
deviation of the mean availability is reported as 0.351.
Calculating
Bounds Based on Assumption of a Normal Distribution with a Logit
Transformation
A
common assumption at this point is that the metric in question
follows a normal distribution. The lower 1-sided, 90% confidence
bound can then be obtained:
However, because of the large
standard deviation, it is possible to obtain a lower bound
smaller than 0 and an upper bound larger than 1 (this would also
be the case if
 were close to 0 or 1). A better approximation here
can be obtained by applying the logit transformation
(logit(p)=log[p/(1-p)]). This will ensure that the endpoints always
fall between 0 and 1. We can then obtain the lower 1-sided, 90%
confidence bound as follows [4]:
where:
In this example, AL
= 0.156 or 15.6%.
However, because of the nature
of the failure modes and their properties, we expect that the
mean availability will be highly asymmetrical and will display a
mixed behavior. After all, if failure mode 2 occurs, the system
is down for the remainder of the mission. We therefore choose to
analyze the mean availability in more detail. This will require
that we obtain the availability for each of the individual
simulations so that additional statistical analyses can be
performed.
Obtaining Individual
Simulation Results in BlockSim
Obtaining the mean availability
for each of 1000 simulations could be a very time-consuming
task. BlockSim 7 allows saving specified results for individual
simulations in a tab-delimited text (*.txt) file. It is possible
to save
results for specified blocks, crews and spare part pools in addition
to individual results for the entire system by activating the
Save Log of Simulations
feature as shown in Figure 3. We can then enter an Increment
Seed by value, which specifies the amount by which the
seed will be increased for each new simulation. For example, if
the seed selected in the General Tab is 1 and the Increment Seed
by input is set to 1, the selected results will be reported for a
single simulation for seed 1, then for seed 2 and so on.
Figure 3: Settings to Save a Log of Simulations
We can also specify the results
of interest by clicking Select Results. For this example,
we select the Mean Availability, as shown in Figure 4.
Figure 4: Select Results Window
The last step is to specify the tab-delimited text (*.txt) file
name and location where results will be saved, as shown
previously in Figure 3. We are
now ready to click Simulate. Note that during standard
simulations BlockSim stores the most recent simulation results
with the diagram as a summary of results averaged across all
simulations. The log, on the other hand, saves individual
simulation results in a text file external to the software and
will not affect any results that may have been saved with the
diagram. Once the simulations have been completed, the
user has the option to attach the text file to the current
project. Figure 5 shows the file, opened in Notepad.
Figure 5: Log of Simulations
for the Mean Availability
It will now be easy to perform
further statistical analysis on these results using another application,
such as Weibull++.
Obtaining Bounds Using a
Non-Parametric Approach
We can use Weibull++ to analyze the data shown above and construct the histogram shown in
Figure 6. In
this plot, availability values are grouped in intervals of 0.1. The
x-value shown is the number of times a mean availability within
a range was observed divided by the size of the interval. For
example, if we would like to obtain the percentage of times a mean
availability from 0 to 0.1 was observed, we would take the size
of the corresponding column (2.570) and multiply it by the
interval size (0.1) to obtain 0.2570 or 25.7%.
Figure 6: Histogram of the Mean Availability
Figure 6 shows two distinct
peaks, which results from the fact that the system displays two
markedly different behaviors (i.e. failure modes).
Attempting to use a mean and a standard deviation in this
example (i.e. assuming a normal distribution), or even
applying a transformation that will capture some skewness in the
population, would largely misrepresent the actual
behavior of the system.
The plot presented in Figure 6 is a non-parametric
approach; that is, no model is assumed for the mean
availability. In order to find the lower 1-sided, 90% confidence
bound for such an approach, we can sort the mean availabilities in increasing order
and pick the 100th value (out of the 1000) as our estimate.
For this example, this value is 0.00313 or 0.3%.
Non-parametric methods do not
assume a model and hence are "safer" than parametric approaches,
in which we run risks such as assuming the wrong model. However,
non-parametric methods are limited in the results that they can
provide. For example, no interpolations or extrapolations are
possible.
Calculating
Bounds Using a Parametric Approach
Should we decide to take a parametric approach instead, we might
use a model that can represent the mixed behaviors of this
particular system. Using Weibull++, we estimate the
parameters for a 3-population, mixed Weibull distribution. The
probability plot is shown in Figure 7. (Note that the choice of
the number of populations in this example is based solely on the
fit, which is usually not the case when using the mixed Weibull
distribution.)
Figure 7: Probability Plot for Mean Availability
We can then obtain 0.3% as the
lower 1-sided, 90% confidence bound of the mean availability by
either reading the Mean Availability value for a probability of
10% from the plot in Figure 7 or by using the Weibull++ Quick Calculation Pad
(QCP).
In using any of these three
methods, be aware that they account for both the simulation
error and the intrinsic variability of the system. However they
do not account for the variability due to the data used to
calculate the parameters. In other words, we are treating the
distribution parameters as constants even though there is
uncertainty around those parameters due to the method or source
used to calculate them.
Conclusion
In this article, we presented a
simple example of an asymmetric estimate using the mean
availability of a system with two failure modes. While the
characteristics of the failure modes in this example were
exaggerated for
illustration purposes, it is not uncommon to see varying degrees
of this behavior in actual analysis. The log of simulations feature of BlockSim 7 was then
used for further analysis of the mean availability via a
non-parametric approach followed by a parametric approach in
order to obtain bounds.
References
[1]
http://www.weibull.com/LifeDataWeb/confidence_bounds.htm
[2]
http://www.weibull.com/LifeDataWeb/the_normal_distribution.htm
[3]
http://www.weibull.com/SystemRelWeb/general_simulation_results.htm
[4] W. Q. Meeker and L. A. Escobar, Statistical Methods for
Reliability Data, New York: John Wiley & Sons, 1998.
|
