Reliability HotWire: eMagazine for the Reliability Professional
Reliability HotWire

Issue 36, February 2004

Hot Topics

Developmental Testing with Corrective Actions Implemented During the Test and Delayed Fixes

[Editor's Note: This article has been updated since its original publication to reflect a more recent version of the software interface.]

Traditional reliability growth tracking and reliability growth projection models address reliability growth based on failure modes that appeared during the test. With the tracking model, all corrective actions are implemented during the test (test-fix-test). For the projection model, all corrective actions are delayed until the end of the test (test-find-test). However, for developmental testing programs, it is more common to implement some corrective actions during the test and some delayed fixes at the end of the test (test-fix-find-test). In addition, reliability growth assessments for the test-fix-find-test strategy are not provided with the widely used traditional models. This article presents the use of an extended reliability growth model for analyzing test-fix-find-test data in RGA and is based on the paper "An Extended Reliability Growth Model For Managing And Accessing Corrective Actions" presented by Dr. Larry Crow at the 2004 RAMS. [Click here to download the paper (*.pdf, 238 KB)]


Consider a product that undergoes developmental testing for 400 hours. During testing, the observed failure modes are identified and:

  • some are corrected during the test (BC modes),
  • some will be corrected after the end of the test phase (BD modes),
  • some will be left (uncorrected) in the system (A modes).

The data set is given next:

Failure Time Mode
0.7 BC1
3.7 BC1
13.2 BC1
15 BD1
17.6 BC2
25.3 BD2
47.5 BD3
54 BD4
54.5 BC3
56.4 BD5
63.6 A
72.2 BD5
99.2 BC4
99.6 BD6
100.3 BD7
102.5 A
112 BD8
112.2 BC5
120.9 BD2
121.9 BC6
125.5 BD9
133.4 BD10
151 BC7
163 BC8
164.7 BD9
174.5 BC9
177.4 BD10
191.6 BC10
Failure Time Mode
192.7 BD11
213 A
244.8 A
249 BD12
250.8 A
260.1 BD1
263.5 BD8
273.1 A
274.7 BD6
282.8 BC11
285 BD13
304 BD9
315.4 BD4
317.1 A
320.6 A
324.5 BD12
324.9 BD10
342 BD5
350.2 BD3
355.2 BC12
364.6 BD10
364.9 A
366.3 BD2
373 BD8
379.4 BD14
389 BD15
394.9 A
395.2 BD16

In addition, an effectiveness factor based on engineering assessment has been assigned for the BD failure modes (delayed fixes), as given in the next table. The effectiveness factor is the expected fractional decrease in failure intensity of a failure mode after the implementation of a corrective action.

BD Mode Effectiveness Factor
1 0.7
2 0.7
3 0.8
4 0.8
5 0.9
6 0.9
7 0.5
8 0.9
9 0.9
10 0.7
11 0.7
12 0.6
13 0.6
14 0.7
15 0.7
16 0.5

The average effectiveness factor is equal to 0.7250. The procedure to enter and analyze this data set in RGA is described next.

Initializing RGA

The first step is to create a folio configured for the proper data type. This can be done via the Data Sheet Setup window, as shown in Figure 1.

Figure 1: RGA Data Type Expert

Figure 1: RGA Data Type Expert

As you can see from Figure 1, two procedures are available: Developmental and Fielded (located at the bottom of the list of data types). For this example, Developmental has been selected along with Failure Times under the Time-to-Failure Data options.

Once the folio has been created with the appropriate data sheet, select Crow Extended as the model from the control panel, as shown in Figure 2.

Figure 2: Selecting the Crow Extended model

Figure 2: Selecting the Crow Extended model

Once Crow Extended has been selected, two additional columns will be inserted into the spreadsheet: the Classification column and the Mode column. These columns are necessary so that all applicable data can be entered. The entered data can be seen in Figure 3, along with the specified effectiveness factors.

Figure 3: Entered reliability growth data

Figure 3: Entered reliability growth data

Results and Discussion

The parameters are estimated and the results are shown in Figure 4.

Figure 4: Estimated parameters using Crow Extended model

Figure 4: Estimated parameters using Crow Extended model

At the end of the test (400 hours), the demonstrated MTBF is 7.7070 hours. The demonstrated MTBF represents the achieved MTBF at the end of the test as a result of the corrective actions during the test (BC modes). If the 16 delayed corrective actions are implemented (BD modes), the projected MTBF is 11.029 hours. In addition, if testing continues with the current management strategy (i.e., modes corrected vs. modes not corrected) and with the current effectiveness of each corrective action, then the maximum attainable MTBF is 14.4921 hours. This is called the growth potential MTBF. The following plot illustrates these results.

Figure 5: Growth Potential MTBF plot

Figure 5: Growth Potential MTBF plot

From this plot, you can verify whether or not you will meet your required MTBF goal based on the selected management strategy. If you do not meet your goal, then you can go back and adjust your management strategy. This gives you quite a bit of flexibility while conducting the analysis. The selected management strategy can be summarized using the Failure Mode Strategy plot, as shown next.

Figure 6: Failure Mode Strategy plot

Figure 6: Failure Mode Strategy plot

This plot shows that 13% of the system's initial failure intensity was removed during the test (seen BC modes), 9% has been left in the system (A modes) and 65% has not yet been observed (unseen BC and BD modes). In addition, 9% will be removed after the delayed corrective actions are implemented while 3% of the BD modes will remain. Therefore, the total percentage of the system's failure intensity that will exist after the corrective actions have been implemented is equal to the sum of the failure intensities due to Type A, Type BC - Unseen, Type BD - Unseen and Type BD - Remained modes. Therefore, 77% of the system's failure intensity will remain.

Figure 7 displays the MTBF of each individual failure mode. For the modes with delayed fixes implemented (BD modes), the plot displays the MTBF before and after the delayed corrective action. This plot allows you to identify the failure modes with the lowest MTBF, which are the failure modes that cause the majority of the system failures.

Individual Mode MTBF plot

Figure 7: Individual Mode MTBF plot


Additional results and plots are also available in RGA, as well as automated reports. For more information on RGA, including a complete list of features and capabilities, please visit

ReliaSoft Corporation

Copyright 2004-2014 ReliaSoft Corporation, ALL RIGHTS RESERVED