Crow Extended - Continuous Evaluation Model

This chapter includes the following sections:

Introduction

The Crow Extended model is designed for a single test phase. However, in most cases testing for a system will be conducted in multiple phases. The Crow Extended - Continuous Evaluation model is designed for analyzing data across multiple test phases, while considering the data for all phases as one data set. In RGA 7 it is applied when using the Multi-Phase data sheet.

The Crow Extended - Continuous Evaluation (3-parameter) model is an extension of the Crow Extended model, and is designed for the practical testing situation where we need the flexibility to apply corrective actions at the time of failure or at a later time during the test, at the end of the current test or during a subsequent test phase. This three-parameter model is free of the constraint that testing must be stopped and all BD modes must be corrected at the end of the test, as in the Crow Extended model. The failure modes can be corrected during the test or when the testing is stopped for the incorporation of the fixes, or even not corrected at all at the end of the test phase. Based on this flexibility, the end time of testing is also not predefined, and it can be continuously updated with new test data, and this is the reason behind the naming, "continuous evaluation."

Definitions: Crow Extended - Continuous Evaluation Model

Classifications

Under the Crow Extended - Continuous Evaluation model, corrective actions can be fixed at the time of failure or delayed to a later time (before time Τ, at time Τ or after time Τ, where Τ indicates the test's end time). The definition of "delayed" is expanded to include all Type B failure modes corrected after the time of failure. This will include most, if not all, design-related failure modes requiring a root cause failure analysis. Failure modes that are corrected at the time of failure are typically related to human factors, such as manufacturing, operator error, etc.

For the Crow Extended - Continuous Evaluation model, the classifications are defined as follows:

A indicates that a corrective action will not be performed (management chooses not to address these modes for technical, financial or other reasons).
BC is defined as a failure mode that receives a corrective action at the time of failure and before the testing resumes. Typically a type BC failure mode does not require extensive root cause failure analysis and therefore can be corrected quickly. Type BC modes are generally easy to fix and are usually related to issues such as quality, manufacturing, operator, etc.
BD is defined as a failure mode that receives a corrective action at a test time after the first occurrence of the failure mode. Therefore, a fix is considered delayed if it is not implemented at the time of failure. A delayed fix can then occur at a later time during the test, at the end of the test (before the next phase) or during another test phase. Type BD failure modes typically require failure analysis and time to fabricate the corrective action. During the period (0,Τ) into the test, there may be BD modes with corrective actions incorporated into the systems and other BD modes that have been seen but not yet fixed.
Table 10.1 shows a comparison between the definitions of the classifications in the Crow Extended and Crow Extended - Continuous Evaluation models:

Table 10.1 -Comparison of classification definitions

Classification	Crow Extended	Crow Extended - Continuous Evaluation
A	Corrective action will not be performed.	Same as Crow Extended.
BC	Corrective action during the test.	Corrective action as the time of failure.
BD	Corrective action delayed until after the completion of the test.	Corrective action delayed to a test time after the first occurrence of the failure mode.

Reliability growth is achieved by decreasing the failure intensity. The failure intensity for the A failure modes will not change. Therefore, reliability growth can be achieved only by decreasing the BC and BD mode failure intensities. It is also clear that, in general, the only part of the BD mode failure intensity that can be decreased is the part that has been seen during testing. BC failure modes and fixed BD modes (delayed fixes implemented during the test) are corrected during testing and their failure intensities will not change any more at the end of test.

Event Codes

The Crow Extended - Continuous Evaluation model uses a column to indicate the events that occurred during testing. Within RGA, event codes are entered within the Event column of the Multi-Phase data sheets. The possible event codes that can be used in the analysis are:

F: indicates a failure time.
I: denotes that a fix has been implemented for a BD failure mode at the specific time. BD modes that have not received a corrective action by time Τ would not have an associated I event in the data set.
Q: indicates that the failure was due to a quality issue. An example of this might be a failure caused by a bolt not being tightened down properly. You have the option to decide whether or not to include quality issues in the analysis. This option can be specified by checking or clearing the Include Q Events checkbox under Continuous Options on the Analysis tab, as shown in Figure 10.1.
P: indicates that the failure was due to a performance issue. You can determine whether or not to include performance issues in the analysis. This option can be specified by checking or clearing the Include P Events checkbox under Continuous Options on the Analysis tab, as shown in Figure 10.1.
AP: indicates an analysis point. Analysis points can be shown in a multi-phase plot to track overall project progress and can be compared to an idealized growth curve.
PH: indicates the end of a test phase. Test phases can be shown in a multi-phase plot to track overall project progress and can be compared to planned growth phases. RGA 7 allows for a maximum of seven test phases to be defined.
X: indicates that the data point will be excluded from the analysis. An "X" can be placed in front of any existing event code or entered by itself. This row of data will then not be included in the analysis.

rga7_ch10cece_q,p_event_options.wmf

Figure 10.1: Options to include or exclude quality and performance
events from the Crow Extended - Continuous Evaluation analysis

Figure 10.2 shows an example of a Crow Extended - Continuous Evaluation folio with all of the possible event codes. As you can see, each failure is indicated with A, BC or BD in the Classification column. In addition, any text can be used to specify the mode. In this figure, failure codes were used in the Mode column for simplicity, but you could just as easily use "Seal Leak," or whatever designation you deem appropriate for the failure mode.

rga7_ch10cece_folio_example.wmf

Figure 10.2: A Crow Extended - Continuous Evaluation folio with all possible event codes

p Ratio

In the Crow Extended - Continuous evaluation, there is a certain probability of not incorporating a corrective action by time Τ. This is the additional (third parameter), as compared to the Crow Extended model. We define p as:

MATH

It is assumed that the ratio p remains fixed over (0,Τ). This implies that each time a distinct failure mode is seen, the probability that the corrective action is delayed until time Τ or later is p. In other words, each time a new BD mode is seen over (0,Τ) there is a probability p that the corrective action for that mode will not have been incorporated into the system by time Τ. Under the Crow Extended two-parameter model, this probability is always equal to 1 at time Τ.

Effectiveness Factors

As discussed in the Crow Extended section of this on-line reference, it is very important to note that failure modes are rarely totally eliminated by a corrective action. After failure modes have been found and fixed, a certain percentage of the failure intensity will be removed, but a certain percentage of the failure intensity will also remain. For each BD mode, an effectiveness factor (EF) is required to estimate how effective the corrective action will be in eliminating the failure intensity due to the failure mode. The EF is the fractional decrease in a mode's failure intensity after a corrective action has been made and must be a value between 0 and 1. A study on EFs showed that an average EF, d, was about 0.7. However, individual EFs for the failure modes may be larger or smaller than the average. Therefore, typically about 30%, i.e. 100(1-d)%, of the BD mode failure intensity will remain in the system after all of the corrective actions have been implemented.

Similar to the Crow Extended model, each BD mode has an effectiveness factor that represents the decrease in failure intensity for that mode once the corrective action has been incorporated into the system. In addition, a delayed fix can be incorporated any time after the first occurrence of the failure mode. Therefore, delayed fixes can be incorporated before the end of the test phase, at the end of the test phase (just like the Crow Extended delayed fixes) or not incorporated at all at the end of the current test phase but postponed for a subsequent test phase. For calculation purposes, any delayed fixes that are incorporated during the test (those with an I event code) do not need to have an effectiveness factor specified, since the fix is already incorporated in the system. Figure 10.3 shows how effectiveness factors are defined in the Crow Extended - Continuous Evaluation model in RGA 7. The figure also shows that you can specify if the delayed fix was actually implemented at the end of the current test phase, at a later phase or not implemented at all.

rga7_ch10cece_ef.wmf

Figure 10.3: Effectiveness factors defined for each unique BD mode not corrected during the test.
If the fix will be implemented at the end of a test phase, the phase number is indicated.

For a Type BD failure mode not yet corrected but still deferred, the Actual Effectiveness Factor for that mode is zero. The Actual Effectiveness Factor for a deferred Type BD failure mode will stay at zero until the point when the corrective action will be incorporated. At that time, the Actual Effectiveness Factor is changed to equal the Assigned Effectiveness Factor for that mode. At this point, the Nominal Effectiveness Factor (the effectiveness factor assuming fixes are implemented at the end of the specific phase) and the Actual Effectiveness Factor are the same. In other words, if a fix is not incorporated for a BD mode, its actual effectiveness for reducing failure intensity is zero, and the assigned effectiveness factor will be used only for projecting the MTBF (or failure intensity), as it will be discussed in Growth Potential and Projections. Figure 10.4 shows an event report in RGA 7. At the end of the test phase, depending on whether the BD mode was specified as fixed or not fixed, the actual EF is zero (e.g. mode BD3000) or equal to the nominal EF (e.g. mode BD1500).

rga7_ch10cece_i_event_report.wmf

Figure 10.4: Event report showing nominal and actual effectiveness factors

The Average Nominal EF is:

MATH (1)

where M is the total number of open and distinct BD modes at time TJ and dNi is the Nominal Effectiveness Factor as specified for each of the BD mode.

The Average Actual EF is:

MATH (2)

where M is the total number of open and distinct BD modes at time TJ and $d_{A_{i}}$ is the Actual Effectiveness Factor at time Tj for each BD mode.

Growth Potential and Projections

The failure intensity left in the system will depend on the management strategy that determines the classification of the A, BC and BD failure modes. The engineering effort applied to the corrective actions determines the effectiveness factors. In addition, $\$ the failure intensity depends on h(t), which is the rate at which unique BD failure modes are being discovered during testing. The rate of discovery drives the opportunity to take corrective actions based on the seen failure modes and it is an important factor in the overall reliability growth rate. The reliability growth potential is the limiting value of the failure intensity as time Τ increases. This limit is the maximum MTBF that can be attained with the current management strategy. The maximum MTBF will be attained when all BD modes have been observed and fixed.

If all seen BD modes are corrected by time Τ, that is, no deferred corrective actions at time Τ, then the Growth Potential is the maximum attainable based on the Type BD designation of the failure modes, the corresponding assigned effectiveness factors and the remaining A modes in the system. This is called the Nominal Growth Potential.

If some seen BD modes are not corrected at the end of the current test phase then the prevailing growth potential is below the maximum attainable with the Type BD designation of the failure modes and the corresponding assigned effectiveness factors.

The Crow-AMSAA (NHPP) model is used to estimate the current demonstrated MTBF or MTBFD. The demonstrated MTBF does not take into account any type of projected improvements.

The corresponding current demonstrated failure intensity is:

or:

The nominal growth potential factor is:

MATH (3)

where:

M is the total number of distinct unfixed BD modes at time Tj.
dNi is the assigned (nominal) EF for the ith unfixed BD mode at time Tj.
Ni is the total number of failures over (0, Tj) for the distinct unfixed BD mode i.

The nominal growth potential factor signifies the failure intensity of the M modes after corrective actions have been implemented for them, using the nominal values for the effectiveness factors.

Similarly, the actual growth potential factor is:

MATH (4)

where dAi is the actual EF for the ith unfixed BD mode at time Tj

The actual growth potential factor signifies the failure intensity of the M modes after corrective actions have been implemented for them, using the actual values for the effectiveness factors.

Based on the definition of BD modes for the Crow Extended - Continuous Evaluation model, the estimate of p at time Tj is calculated as follows:

MATH (5)

The unfixed BD mode failure intensity at time Tj is:

MATH (6)

Similar to the Crow Extended model, the discovery function at time Τ for the Crow Extended - Continuous Evaluation model is calculated using all the first occurrences of the all the BD modes, both fixed and unfixed. h(t) is the unseen BD mode failure intensity and is also the rate at which new unique BD modes are being discovered.

MATH (7)

where:

is the unbiased estimated of β for the Crow-AMSAA (NHPP) model based on the first occurrence of M distinct BD modes.
is the unbiased estimated of λ for the Crow-AMSAA (NHPP) model based on the first occurrence of M distinct BD modes.