Using SimuMatic in RGA
[Editor's Note: This article has been updated since its original publication to reflect a more recent version of the software interface.]
Simulation is a very powerful tool that can assist
with developing test plans or generating bootstrap confidence
intervals. This article will describe the function of SimuMatic
in RGA and
provide an example of how SimuMatic can be used to
determine the sample size that is needed to demonstrate
a reliability goal at a specified confidence level.
SimuMatic
RGA's SimuMatic utility uses Monte Carlo simulation to generate a specified
number of data sets based on user-specified inputs and then
automatically calculates the beta and lambda parameters
of the Crow-AMSAA model for each of the generated sets.
Using this type of an analysis, one can estimate the expected
confidence bounds on the demonstrated MTBF or failure intensity
at the end of the test, and one can determine whether the
planned test time or sample size will suffice
to achieve a goal MTBF at the end of a test.
The data types that can be generated using SimuMatic
are described next. With each data type, there is a choice
between a time-terminated test, where the termination time
needs to be specified, or a failure-terminated test, where
the number of failures needs to be specified.
- Failure Times: The data sets are generated
assuming a single system.
- Multiple Systems – Concurrent: The number of systems needs to be specified.
- Grouped Failure Times: The data sets are
generated assuming a single system. Constant or user-defined intervals need to be specified for the grouping
of the data.
- Repairable Systems: The number
of systems needs to be specified.
Example
A reliability engineer is developing a reliability growth
test plan for a new system. The target MTBF that she needs
to demonstrate at the end of the test is 800 hours with
a 90% confidence. She has been allocated 2,000 hours of
test time and wants to determine how many prototypes will
need to be tested in order to
demonstrate the MTBF goal.
From historical growth tests on previous design iterations
of the system, she estimates that the beta and lambda parameters
of the Crow-AMSAA model are expected to be 0.55 and 0.21,
respectively.
The reliability engineer decided to use the SimuMatic
tool in RGA in order to design the reliability
growth test. Figure 1 shows the Main tab of SimuMatic’s setup window.
Figure 1: Main tab of
the SimuMatic setup
window
The estimated Crow-AMSAA parameters were entered and
the data type that was chosen was Multiple Systems – Concurrent,
since the test will include more than one system. The test
was set to end at 2,000 hours, which is the current time
constraint. 10 systems were initially selected to be used
in the test. Finally, 1,000 simulations were chosen to be
run.
Figure 2 shows the remaining tabs of the SimuMatic
window, where the engineer chose to calculate the upper
1-sided 90% confidence bound on the time required to
reach the target MTBF of 800 hours (entered in the
window as the lower 1-sided 10% confidence bound). She also chose to calculate the demonstrated
MTBF at the end of the test.
Figure 2:
Remaining tabs of the SimuMatic setup window
Using these settings, SimuMatic generated 1,000
data sets and automatically calculated beta and lambda
for each set. Figure 3 shows a portion of the calculated
parameters and results of the analysis arranged in an ascending
order based on the Percentage column. This column indicates
the percentage of values that are less than or equal to
each corresponding value. The lower 1-sided 90% confidence
bound on the demonstrated MTBF at the end of the test can
be found by looking at the value under the DMTBF column
that corresponds to the 10th percentile (the upper 1-sided
90% bound would be the value that corresponds to the
90th
percentile). As it can be seen from the highlighted row,
the demonstrated MTBF at the end of the test is 668.123 hours.
Figure 3: Lower 1-sided 90% bound
on the demonstrated MTBF
Given that the target MTBF has not been met at the end
of the test, the next step is to determine the required
time to reach that goal. The required time is displayed
in the “Target DMTBF” column in the SimuMatic results. Figure
4 shows the highlighted row that gives the upper 1-sided
90% confidence bound on the time required to reach the target
MTBF of 800 hours. Note that the upper bound
was used in this case because it corresponds to the worst case scenario
of time required to reach the target.
Figure 4:
Upper 1-sided 90% bound
on time to reach the target MTBF
As it can be seen, the required time to reach the target
MTBF is 28,292 hours. The same value can be confirmed
by looking at the Instantaneous MTBF vs. Time plot as shown
in Figure 5.
Figure 5: Instantaneous MTBF vs. Time
plot
Given that each system in the test accumulates 2,000
hours of testing, the current plan of 10 systems will lead
to a total accumulated time of 20,000 hours. Therefore,
approximately 10,000 hours of additional test time are required
in order to accumulate 28,292 test hours and reach the target
MTBF. In other words, an additional 5 systems (each tested
for 2,000 hours) will be needed to accumulate enough
test time.
Figure 6 shows the results of another simulation, where
the number of systems in the test was set to 15 instead
of 10. The lower 1-sided 90% confidence bound on the demonstrated
MTBF at the end of the test can be found by looking at the
value under the DMTBF column that corresponds to the
10th percentile. That value is
837.3306 hours.
Figure 6: Lower 1-sided 90% bound on the demonstrated MTBF with 15 systems in
the test
As these results show, if 15 systems are included in
the test and each one is tested for 2,000 hours, then the
goal MTBF of 800 hours with a 90% confidence should be met.
Conclusion
This article illustrated how Monte Carlo simulation and the SimuMatic
tool in RGA can be used to determine whether a
given test plan will demonstrate an MTBF goal at a
specified confidence level. If a plan won't demonstrate
the goal, the article illustrated how SimuMatic can be used to modify
the plan so that it will. As with any test design, the inputs to the design
(the beta and lambda parameters of the Crow-AMSAA in this
case) will have a great effect on the calculated results
and oftentimes need to be assumed. Nevertheless, SimuMatic can be very
useful for obtaining an estimate of required test time or
sample size.
|
