Developmental Testing with Corrective Actions Implemented During the Test and Delayed Fixes

[Editor's Note: This article has been updated since its original publication to reflect a more recent version of the software interface.]

Traditional reliability growth tracking and reliability growth projection models address reliability growth based on failure modes that appeared during the test. With the tracking model, all corrective actions are implemented during the test (test-fix-test). For the projection model, all corrective actions are delayed until the end of the test (test-find-test). However, for developmental testing programs, it is more common to implement some corrective actions during the test and some delayed fixes at the end of the test (test-fix-find-test). In addition, reliability growth assessments for the test-fix-find-test strategy are not provided with the widely used traditional models. This article presents the use of an extended reliability growth model for analyzing test-fix-find-test data in RGA and is based on the paper "An Extended Reliability Growth Model For Managing And Accessing Corrective Actions" presented by Dr. Larry Crow at the 2004 RAMS. [Click here to download the paper (*.pdf, 238 KB)]

Example

Consider a product that undergoes developmental testing for 400 hours. During testing, the observed failure modes are identified and:

some are corrected during the test (BC modes),
some will be corrected after the end of the test phase (BD modes),
some will be left (uncorrected) in the system (A modes).

The data set is given next:

Failure Time	Mode
0.7	BC1
3.7	BC1
13.2	BC1
15	BD1
17.6	BC2
25.3	BD2
47.5	BD3
54	BD4
54.5	BC3
56.4	BD5
63.6	A
72.2	BD5
99.2	BC4
99.6	BD6
100.3	BD7
102.5	A
112	BD8
112.2	BC5
120.9	BD2
121.9	BC6
125.5	BD9
133.4	BD10
151	BC7
163	BC8
164.7	BD9
174.5	BC9
177.4	BD10
191.6	BC10

Failure Time	Mode
192.7	BD11
213	A
244.8	A
249	BD12
250.8	A
260.1	BD1
263.5	BD8
273.1	A
274.7	BD6
282.8	BC11
285	BD13
304	BD9
315.4	BD4
317.1	A
320.6	A
324.5	BD12
324.9	BD10
342	BD5
350.2	BD3
355.2	BC12
364.6	BD10
364.9	A
366.3	BD2
373	BD8
379.4	BD14
389	BD15
394.9	A
395.2	BD16

In addition, an effectiveness factor based on engineering assessment has been assigned for the BD failure modes (delayed fixes), as given in the next table. The effectiveness factor is the expected fractional decrease in failure intensity of a failure mode after the implementation of a corrective action.

BD Mode	Effectiveness Factor
1	0.7
2	0.7
3	0.8
4	0.8
5	0.9
6	0.9
7	0.5
8	0.9
9	0.9
10	0.7
11	0.7
12	0.6
13	0.6
14	0.7
15	0.7
16	0.5

The average effectiveness factor is equal to 0.7250. The procedure to enter and analyze this data set in RGA is described next.

Initializing RGA

The first step is to create a folio configured for the proper data type. This can be done via the Data Sheet Setup window, as shown in Figure 1.

Figure 1: RGA Data Type Expert

Figure 1: RGA Data Type Expert

As you can see from Figure 1, two procedures are available: Developmental and Fielded (located at the bottom of the list of data types). For this example, Developmental has been selected along with Failure Times under the Time-to-Failure Data options.

Once the folio has been created with the appropriate data sheet, select Crow Extended as the model from the control panel, as shown in Figure 2.

Figure 2: Selecting the Crow Extended model

Figure 2: Selecting the Crow Extended model

Once Crow Extended has been selected, two additional columns will be inserted into the spreadsheet: the Classification column and the Mode column. These columns are necessary so that all applicable data can be entered. The entered data can be seen in Figure 3, along with the specified effectiveness factors.

Figure 3: Entered reliability growth data

Figure 3: Entered reliability growth data

Results and Discussion

The parameters are estimated and the results are shown in Figure 4.

Figure 4: Estimated parameters using Crow Extended model

Figure 4: Estimated parameters using Crow Extended model

At the end of the test (400 hours), the demonstrated MTBF is 7.7070 hours. The demonstrated MTBF represents the achieved MTBF at the end of the test as a result of the corrective actions during the test (BC modes). If the 16 delayed corrective actions are implemented (BD modes), the projected MTBF is 11.029 hours. In addition, if testing continues with the current management strategy (i.e., modes corrected vs. modes not corrected) and with the current effectiveness of each corrective action, then the maximum attainable MTBF is 14.4921 hours. This is called the growth potential MTBF. The following plot illustrates these results.

Figure 5: Growth Potential MTBF plot

Figure 5: Growth Potential MTBF plot

From this plot, you can verify whether or not you will meet your required MTBF goal based on the selected management strategy. If you do not meet your goal, then you can go back and adjust your management strategy. This gives you quite a bit of flexibility while conducting the analysis. The selected management strategy can be summarized using the Failure Mode Strategy plot, as shown next.

Figure 6: Failure Mode Strategy plot

Figure 6: Failure Mode Strategy plot

This plot shows that 13% of the system's initial failure intensity was removed during the test (seen BC modes), 9% has been left in the system (A modes) and 65% has not yet been observed (unseen BC and BD modes). In addition, 9% will be removed after the delayed corrective actions are implemented while 3% of the BD modes will remain. Therefore, the total percentage of the system's failure intensity that will exist after the corrective actions have been implemented is equal to the sum of the failure intensities due to Type A, Type BC - Unseen, Type BD - Unseen and Type BD - Remained modes. Therefore, 77% of the system's failure intensity will remain.

Figure 7 displays the MTBF of each individual failure mode. For the modes with delayed fixes implemented (BD modes), the plot displays the MTBF before and after the delayed corrective action. This plot allows you to identify the failure modes with the lowest MTBF, which are the failure modes that cause the majority of the system failures.

Individual Mode MTBF plot

Figure 7: Individual Mode MTBF plot

Conclusion

Additional results and plots are also available in RGA, as well as automated reports. For more information on RGA, including a complete list of features and capabilities, please visit http://RGA.ReliaSoft.com.