Reliability Basics

# Understanding Updated Life Data Results

It is very common for reliability engineers to update their test results with new data as tests progress, or to augment their test results with fielded system information. If the analyst is not careful in interpreting the results, he or she can easily draw wrong conclusions about the reliability of the product. In this article, we present a scenario where updated life data analysis results can seem confusing to the analyst. The purpose of the article is to highlight the impact that new data can have in life data analysis results and to show one way to handle such a situation in Weibull++.

## The Scenario

An aerospace manufacturer is looking at the reliability of a new component with an intended mission duration of 2,100 hours. The reliability engineer has internal test data for the beta version and has also collected reliability information from the company’s beta customers after 2,500 hours of usage, in which no failures were seen.

The reliability engineer, Lisa, decides to use a 2-parameter Weibull distribution to analyze the data. Since there are relatively few failure data points and a lot of suspended data (30 units from customer beta sites), she decides to use Maximum Likelihood Estimation (MLE) as the statistical analysis method for the data. MLE uses the actual suspension times, not just their relative position in terms of where they occurred in the data set, as rank regression would do. The life data analysis results are shown in Figure 1.

Figure 1: Folio with Internal Test Data and Suspensions from Beta Testing After 2,500 Hours of Usage

Lisa then uses the Quick Calculation Pad (QCP) to obtain the reliability estimate at the mission duration of 2,100 hours, as shown in Figure 2.

Figure 2: Reliability Estimate at 2,100 Hours with Suspended Data at 2,500 Hours

Based on the original data set, the reliability at 2,100 hours is calculated to be 97.65%. She creates a report distributing this information to her organization.

Later in the process, Lisa is ready to update the report with new data from the beta sites. She collects data from the beta sites and sees that there were still no failures in the field, with the 30 beta units having reached 7,000 cumulative hours of test time. She is expecting an increase in the calculated reliability, since there was more accumulated time and no failures. She wants to keep the analysis consistent so she again calculates the life data using a 2-parameter Weibull distribution using MLE. Figure 3 shows the updated life data analysis results with the 30 beta site units still operating at 7,000 hours.

Figure 3: Folio with Internal Test Data and Suspensions from Beta Testing After 7,000 Hours of Usage

She then uses the QCP to obtain the updated estimate of the reliability at the mission duration of 2,100 hours, as shown in Figure 4.

Figure 4: Reliability Estimate at 2,100 Hours with Suspended Data at 7,000 Hours

To her surprise, the new reliability estimate is now 95.61%, which is lower than the previous estimate of 97.65%. How is this possible? The beta site units did not exhibit any failures and had more accumulated hours. The reliability should have gone up.

## Explanation

This is a typical problem of looking at data in isolation and not understanding the impact of the selection of the model. Using a 2-parameter Weibull distribution, the model parameters are recalculated both in terms of the beta parameter, which indicates the slope of the Weibull probability plot, and the eta parameter, which is the characteristic life of the component (i.e. the estimated time by which 63.2% of the components will have failed).

If you take a closer look at the data, you’ll notice that both the beta and eta parameters are different in the original and updated results. The slope has changed from 5.0151 to 1.5634, and the eta parameter has changed from 4,427 to 15,280. Figure 5 shows a MultiPlot of the two Weibull distributions. Before the point at which the two probability lines cross each other, the updated analysis (shown in blue) yields lower reliabilities. After the lines cross, the updated analysis yields higher reliabilities. Since the mission duration of 2,100 hours is before the two lines meet, the updated analysis resulted in a lower reliability estimate for 2,100 hours.

Figure 5: Weibull Probability Plot of the Original and Updated Analyses

## Solution

There is nothing inherently wrong with the MultiPlot shown in Figure 5. The analyst just needs to understand the impact of re-estimating a probability plot based on the data. The number of parameters that will be used can allow different "degrees of freedom" for the line to change both in terms of slope and intercept.

If Lisa wanted to create an updated report reflecting the progress from using the suspended data from the beta sites, a possible solution would be to superimpose the same slope in both probability plots.

In this case, a good estimate of the actual slope would involve the original test data and good engineering judgment in terms of the expected failure rate behavior. Remember that a high beta can indicate wear-out, while a beta close to 1 indicates more random failure types, such as insufficient design margins, external events, etc. A beta less than 1 indicates early life failures, such as those caused by misassembly, manufacturing issues or damage during shipping or storage.

The original internal test data set, excluding the 30 suspended units from the customer beta sites, is calculated using a 2-parameter Weibull distribution and Rank Regression on X (RRX), since the data are now complete (exact times to failure) and it is a small data set. For comparison purposes, the data are also analyzed using MLE, and Figure 6 shows a MultiPlot of the two methods, where the black line represents the RRX analysis and the blue line represents the MLE analysis.

Figure 6: Weibull Probability Plots Including Internal Test Data Only, Using MLE and RXX

Using engineering judgment about the nature of the expected failure modes, a beta would be chosen to be used for all further analyses. In this case, it would be appropriate to accept the most conservative estimate of beta. Since the mission duration is 2,100 hours, a smaller beta would yield more failures in that region. In this case, the RRX beta of 2.8219 is chosen.

After choosing a beta value, every updated analysis with new suspended data can be done by superimposing that value of beta on the data set. In that case, a 1-parameter Weibull distribution would be used and the value of beta would be provided, as shown in Figure 7 for the data set including the suspensions at 2,500 hours.

Figure 7: Superimposing a Beta Parameter Value onto the Analysis

The same process would be used in the analysis including the customer beta site units suspended at 7,000 hours. Using this process ensures that the slope remains the same in the two data sets, making it easier to draw comparisons, as shown in the MultiPlot in Figure 8.

Figure 8: Weibull Probability Plot of the Original and Updated Analysis with the Same Superimposed Beta

Figures 9 and 10 show the new calculated values of reliability at 2,100 hours. The data set with the 30 customer beta site units suspended at 2,500 hours exhibits a reliability of 91.44% and the data set with the customer beta site units suspended at 7,000 hours exhibits a reliability of 98.96%. The improvement obtained by adding more hours to the suspended units is now more evident.

Figure 9: Reliability Estimate at 2,100 Hours with Suspended Data at 2,500 Hours and Superimposed Beta

Figure 10: Reliability Estimate at 2,100 Hours with Suspended Data at 7,000 Hours and Superimposed Beta

## Conclusion

When updating life data after a previous analysis, it is important to understand the impact of the choice of the model and the statistical analysis method upon the results. A good practice is to look at probability plots in order to understand the total behavior of the model. When the analysis and further updates are performed without examining the overall behavior of the statistical models, wrong conclusions can be drawn.