Confidence Bounds in Reliability Growth Planning
[Editor's Note: This article has been updated since its original publication to reflect a more recent version of the software interface.]
You can use the reliability growth planning tool in RGA to determine whether a system’s mean time between failures (MTBF) goal will be met by the end of the test given a set of input parameters such as the initial MTBF, the management strategy and the design margin. One very useful feature of this growth planning tool is the calculation of confidence bounds on the planned MTBF values of each test phase. Once testing begins, the actual demonstrated MTBF of each test phase can be compared with these calculated bounds to determine whether the reliability growth is progressing according to the plan and whether there is a risk trying to achieve the MTBF goal in time. If the demonstrated MTBF of a phase is found to be lower than the calculated confidence bound, that would indicate that the plan is not being followed and the likelihood of achieving the goal MTBF at the end of the test is decreased. If this is the case, testing efforts would need to be adjusted such as changing the management strategy (implementing fixes for more observed failure modes) or ultimately increasing the allocated test time. However, given that planning occurs before testing begins and failures are observed, the question that one can ask is how can RGA calculate confidence bounds in the planning model without the presence of any actual data? This article describes the process of calculating these confidence bounds.
Consider a reliability growth test plan that consists of six phases. The goal MTBF is 300 hours while the total cumulative test time is 60,000 hours. Figure 1 shows the inputs and results of the test plan.
Figure 1 - Growth planning folio
(For more information about using the inputs and results of the growth planning tool, refer to the "Reliability Growth Planning" article in HotWire Issue 119.)
As shown in Figure 2, the results are also summarized in the growth planning report.
Figure 2 - Growth planning report
As it can be seen, the plan estimates that the goal MTBF can be reached at ~58,100 hours of test time. The "Planned Growth" values listed at the bottom of the report indicate the expected starting MTBF of each phase. These are the values that the confidence bounds will be applied to in order to track the progress once testing begins.
In order calculate those confidence bounds, we first need to generate a data set. This data set will be the expected number of failures at each phase, given the phase duration and planned MTBF. The following table summarizes the duration and planned MTBF of each phase for this example.
|Phase||Duration (hours)||Planned MTBF|
The expected number of failures (Ni) of each phase can be calculated as:
Therefore, the expected number of failures for phase 1 is:
The expected number of failures for the other five phases can be calculated in a similar manner. The results are:
N2 = 46
N3 = 38
N4 = 36
N5 = 35
N6 = 51
This data set can be analyzed in RGA using the Grouped Failure Times data type and applying the Crow-AMSAA model. Figure 3 shows the grouped data folio and the calculated parameters. Note that the durations from the table are converted to cumulative test times in the standard folio (e.g., the Interval End Time for phase 2 is 15000 because it lasts for 10,000 hours after phase 1 ends at 5,000 hours).
Figure 3 - Grouped data analysis
Now this model can be used to estimate the confidence bounds on the planned MTBF values of each phase. To do that, we first use the Quick Calculation Pad to calculate the time that is associated with each planned MTBF. For example, as shown in Figure 4, if you look at phase 1, the time at which the instantaneous MTBF = 130.65 is 1,409.7685:
Figure 4 - Time given instantaneous MTBF
We can then use this time to calculate the lower 90% 1-sided confidence bound, as shown in Figure 5.
Figure 5 - Lower 1-sided confidence bound
As it can be seen, the lower 1-sided 90% confidence bound for the phase 1 planned MTBF is 114.7830. These values can also be displayed in the growth planning plot. The lower bound for the phase 2 planned MTBF is shown in Figure 6.
Figure 6 - Growth planning plot
As discussed above, these confidence bounds can be used to track the progress of the growth test and assure that the goal can be met. For example, Figure 7 shows a plot of the planned values and actual test data.
Figure 7 - Planned values and actual test data
As it can be seen, the demonstrated MTBF in phase 2 (190.9034) was lower than lower bound of the planned MTBF for that phase (194.7511), which indicates that the reliability growth at this point in testing was not as high as planned. However, after additional effort, the demonstrated MTBF of the other phases was within the bounds until the final goal of 300 hours was reached.
In this article, we presented the steps for calculating the confidence bounds for each phase’s planned MTBF in the growth planning model. We also illustrated how those bounds can be useful once testing begins to determine whether the reliability growth is progressing according to the plan and whether the final MTBF goal is likely to be met.