Issue 36, February 2004

Developmental Testing with Corrective Actions Implemented During the Test and Delayed Fixes

Traditional reliability growth tracking and reliability growth projection models address reliability growth based on failure modes that appeared during the test. With the tracking model, all corrective actions are implemented during the test (test-fix-test). For the projection model, all corrective actions are delayed until the end of the test (test-find-test). However, for developmental testing programs, it is more common to implement some corrective actions during the test and some delayed fixes at the end of the test (test-fix-find-test). In addition, reliability growth assessments for the test-fix-find-test strategy are not provided with the widely used traditional models. This article presents the use of an extended reliability growth model for analyzing test-fix-find-test data in RGA 6 and is based on the paper "An Extended Reliability Growth Model For Managing And Accessing Corrective Actions" presented by Dr. Larry Crow at the 2004 RAMS. [Click here to download the paper (*.pdf, 238 KB)]

Example

Consider a product that undergoes developmental testing for 400 hours. During testing, the observed failure modes are identified and:

• some are corrected during the test (BC modes),
• some will be corrected after the end of the test phase (BD modes),
• some will be left (uncorrected) in the system (A modes).

The data set is given next:

In addition, an effectiveness factor based on engineering assessment has been assigned for the BD failure modes (delayed fixes), as given in the next table. The effectiveness factor is the expected fractional decrease in failure intensity of a failure mode after the implementation of a corrective action.

The average effectiveness factor is equal to 0.7250. The procedure to enter and analyze this data set in RGA 6 is described next.

Initializing RGA 6

The first step is to create the proper Data Entry Spreadsheet. This can be done via the Data Type Expert, as shown in Figure 1.

Figure 1: RGA 6 Data Type Expert

As you can see from Figure 1, two procedures are available: Developmental Testing and Fielded Systems (located at the bottom of the list of data types). For this example, Developmental Testing has been selected along with Failure Times under the Time-to-Failure Data options. Notice that the models that will be available during the analysis are displayed on the right side of the window. For the selected data type, the Crow-AMSAA (NHPP), Crow Extended and Duane models will be available.

Once the Data Folio has been created with the appropriate Data Entry Spreadsheet, select Crow Extended as the model from the Data Folio Control Panel, as shown in Figure 2.

Figure 2: Selecting the Crow Extended model

Once Crow Extended has been selected, two additional columns will be inserted into the spreadsheet: the Classification column and the Mode column. These columns are necessary so that all applicable data can be entered. The entered data can be seen in Figure 3, along with the specified effectiveness factors. (Please note that the following Data Entry Spreadsheet displays only 36 rows of data. The Data Entry Spreadsheet for this example contains 56 rows of data.)

Figure 3: Entered reliability growth data

Results and Discussion

The parameters are estimated and the results are shown in Figure 4.

Figure 4: Estimated parameters using Crow Extended model

At the end of the test (400 hours), the demonstrated MTBF is 7.71 hours. The demonstrated MTBF represents the achieved MTBF at the end of the test as a result of the corrective actions during the test (BC modes). If the 16 delayed corrective actions are implemented (BD modes), the projected MTBF is 10.86 hours. In addition, if testing continues with the current management strategy (i.e. modes corrected vs. modes not corrected) and with the current effectiveness of each corrective action, then the maximum attainable MTBF is 19.20 hours. This is called the growth potential MTBF. The following plot illustrates these results.

Figure 5: Growth Potential MTBF plot

From this plot, you can verify whether or not you will meet your required MTBF goal based on the selected management strategy. If you do not meet your goal, then you can go back and adjust your management strategy. This gives you quite a bit of flexibility while conducting the analysis. The selected management strategy can be summarized using the Failure Mode Strategy plot, as shown next.

Figure 6: Failure Mode Strategy plot

This plot shows that 25% of the systems initial failure intensity was removed during the test (BC modes), 17.86% has been left in the system (A modes) and 22.77% has not yet been observed. In addition, 26.10% will be removed after the delayed corrective actions are implemented while 8.27% of the BD modes will remain. Therefore, the total percentage of the system's failure intensity that will exist after the corrective actions have been implemented is equal to the sum of the failure intensities due to Type A, Type BD Unseen and Type BD Remain modes. Therefore, 48.9% of the system's failure intensity will remain. In other words, 51.1% of the system's failure intensity is being removed based on the current management strategy.

Figure 7 displays the MTBF of each individual failure mode. For the modes with delayed fixes implemented (BD modes), the plot displays the MTBF before and after the delayed corrective action. This plot allows you to identify the failure modes with the lowest MTBF, which are the failure modes that cause the majority of the system failures.

Figure 7: Individual Mode MTBF plot

Conclusion

Additional results and plots are also available in RGA 6, as well as automated reports. For more information on RGA 6, including a complete list of features and capabilities, please visit the RGA 6 Web site at http://RGA.ReliaSoft.com.

Copyright 2004 ReliaSoft Corporation, ALL RIGHTS RESERVED