Issue 88, June 2008

Design of Experiments (DOE) is one of the important tools in Design for Six Sigma (DFSS) and Design for Reliability (DFR). However, there are difficulties with applying the traditional DOE analysis methods, such as ANOVA or linear regression, for data from life tests. In this article, we will briefly explain why the traditional DOE analysis methods cannot be directly applied to life data, and then provide an example of how to use ReliaSoft's DOE++ to design a life test and analyze the data.

Difficulties of Using Traditional DOE Analysis Methods
Traditional DOE analysis methods, particularly the ANOVA technique and standard linear regression, assume that the responses are normally distributed. However, lifetimes usually follow a skewed distribution, such as a Weibull, exponential, lognormal or gamma distribution. Therefore, it is not appropriate to use the F ratio of the sum of squares, which is based on the normal distribution assumption, in ANOVA for life data.

Another difficulty arises due to data type. In life tests, the data obtained may be either complete, censored or interval. However, there is no way to calculate the sum of squares for censored and interval data. The sum of squares makes sense only for complete response data. Further, the standard regression techniques applicable to the complete response data in traditional DOE can no longer be used if there are censored and interval data.

Therefore, in order to apply the traditional DOE methods to life data, people usually treat suspensions as failures and use the middle points of the interval data as the failure times. This "adjustment" is not correct and may lead to wrong conclusions. Even after this treatment, the data still may be skewed. So, a transformation is often applied to the "adjusted" data and then ANOVA and linear regression are used for the "transformed" data.

However, even with the adjustment and transformation, problems may still exist [1]. Standard linear regression assumes the following about the data used in the model:

• They are normally distributed.
• They have constant variances independent of the response value.

Thus, a single transformation is asked to produce the above two requirements simultaneously, which usually does not occur.

So it is best to reconsider using ANOVA and standard linear regression on life data. The Reliability DOE module in the DOE++ software provides an alternative, and the following sections provide an example of using this technique (Example 1), contrasted with the use of traditional DOE methods for the same data set (Example 2).

Example 1:
An engineer wants to improve the reliability of an electric motor. After brainstorming, he thinks of five factors that might affect the motor life. He needs to design an experiment to identify the significant factors and then adjust the value of these factors to improve the reliability. Because of the cost of the experiment, a full factorial design is not practical, so he decides to conduct a two-level fractional factorial design to provide some useful conclusions.

1) Design the Experiment
Together with the design engineers, he determines the properties for the five factors:

Figure 1: Factor Property Table

It is reasonable to assume that the 3-way or higher interactions among factors are negligible, so the fractional generator is set to E = ABCD. A 16-run design is generated as shown below.

Figure 2: 16 Run Design Matrix in Coded Values and Standard Order

In Figure 2 above, the code "-1" means a factor is at its low level and "1" means the factor is at its high level. In DOE++, you also can select Actual Values to display the actual settings of these factors.

The experiment is not conducted according to the standard order of the rows; rather, it is conducted by run order. Run order is a randomized order intended to eliminate the effects of some noise factors. Also, the technicians in this case feel that it is not convenient to read the coded values of the factor settings, so the engineer sorts the rows by Run Order and selected Actual Values for the Display Factor, obtaining the following design matrix.

Figure 3: 16 Run Design Matrix in Actual Values and Run Order

2) Conduct Experiment and Collect Data
The engineer prints out the design sheet and gives it to a technician. The technician has only 5 days to test all the motors before her vacation. At the end of the 5 days, the technician provides the following data to the engineer.

Figure 4: Experiment Results

The failure times are in units of hours. Notice that there is a suspension (highlighted in Figure 4 above) at the end of the 5 day test.

3) Analyze the Data
After receiving the data, the engineer wonders which lifetime distribution he should use. He tries both the lognormal and the Weibull distribution. The results are given below.

Figure 5 (a): Results for Lognormal Distribution

Figure 5 (b): Results for Weibull Distribution

In order to understand the results in Figure 5, let's discuss the math a little bit. For a detailed discussion, please refer to [1, 2]. The probability density function of the Weibull distribution is:

In reliability DOE, the scale parameter, , is assumed to be a function of the effects, which is expressed as:

Usually, the high order interactions such as 3-way and 4-way interactions can be ignored in the above equation. Once the effects have been selected for the model, maximum likelihood estimation is used to estimate the parameters of each distribution. From the log-likelihood values highlighted in Figure 5, we can see that the Weibull distribution provides larger likelihood values. Thus, the engineer decides to use the Weibull distribution. To further confirm his choice, he also examines the residual plots for both distributions, which are shown in Figure 6.

Figure 6 (a): Residual vs. Run Order Plot for Lognormal Distribution
[Click to Enlarge]

Figure 6 (b): Residual vs. Run Order Plot for Weibull Distribution
[Click to Enlarge]

The red dashed lines are the thresholds at the given significance level of 0.1 for detecting abnormal residuals. Apparently, the Weibull distribution has a better residual plot, since all of the residuals fall between the high and the low thresholds.

4) Interpret the Results
From the results in Figure 5(b), it can be seen that effects A, B, E, AE and CD are significant at the significance level of 0.1, as their p values are less than 0.1. In contrast to traditional ANOVA, where the p values are calculated based on the F ratio test, in reliability DOE the p values are calculated based on the likelihood ratio test. For more detail, please refer to references [1, 2].

Factors that are statistically significant are colored red in the results. Although the CD interaction is shown as significant, neither of the main effects, C or D, is significant. So it is questionable whether the 2-way interaction for CD is actually significant. A possible reason why CD is shown as being significant in the results is because it is confounded with other effects that were not in the model. So by checking the generator E=ABCD, the engineer realizes that CD is confounded with ABE, a 3-way interaction. For this interaction, all the main effects, A, B and E, are significant. Therefore, it is reasonable to assume that the interaction of ABE, instead of CD, causes the significance of the confounded effects. So, he decides to only keep A, B, E, AE and ABE in the model and gets the following results.

Figure 7: Result of the Final Model
[Click to Enlarge]

5) Draw Conclusions
From this analysis, the engineer finds that factors C and D are not important to the motor life. In other words, their settings have limited effects. Meanwhile, A, B and E are important. To increase the life of the motor, A, B and E should be set to the optimum values. Since this example is simple, we can easily see from the Coefficient column in Figure 7 that A and E at 1 (high level) and B at -1 (low level) are the best settings. But sometimes, if there are interaction terms in the model and the best settings for the main effects are not the best settings for the interactions, one needs to use the optimization tool to find the optimum solution. By clicking the Optimization icon in DOE++, the following plot is obtained.

Figure 8: Optimum Solution for Motor Life
[Click to Enlarge]

The red dashed lines indicate the best settings for each factor. The blue dashed line is the predicted response [()] under the best settings. Those values are also displayed as X and Y values in the plot.

6) Make Prediction
Using the final model in Figure 7, the scale parameter can be predicted:

So . The distribution is:

In DOE++, one can get the predicted and its two-sided confidence bounds by clicking the Prediction icon to open the prediction utility. Figure 9 shows the prediction results using a 90% confidence level.

Figure 9: Predicting Distribution Parameter

7) Confirmation Experiments
If time, cost and other resources allow, it is always recommended to conduct a follow-up experiment to confirm the best settings. However, in this example, because of resource constraints, no follow-up experiments are performed.

In Example 1, we illustrated how to use reliability DOE for life tests and data analysis. One additional question is: What are the results if the traditional DOE methods are used for the data in Example 1? Example 2 will give the answer.

Example 2
Another engineer gets a copy of the data from the technician after her vacation. In order to do the analysis using traditional DOE, he treats the suspension as a failure.

1) Transformation
The engineer knows that the life data is skewed, so he uses a logarithmic transformation before performing the analysis. In DOE++, the transformation can be applied by clicking the Transformation icon and selecting Natural Log.

Figure 10: Applying a Logarithmic Transformation

2) Analysis
First, all the effects up to 2-way interactions are chosen. With all the 2-way effects, the model is saturated and no residuals are calculated for this 16-run experiment. Therefore, no ANOVA table is provided. The engineer uses a probability plot to examine the effects. The normal probability of effect is shown below.

Figure 11: Effect Probability Plot
[Click to Enlarge]

From this plot, none of the effects is significant.

The analysis could be stopped here with this conclusion. However, the engineer wants to go further. He decides to drop effects AC, DE and BD which are on the probability line with values close to 0. He believes it to be safe to pool these effects into error.

The results of the reduced model are shown in Figures 12 and 13:

Figure 12: Results after Initial Pooling
[Click to Enlarge]

Figure 13: Pareto Chart of Results after Initial Pooling
[Click to Enlarge]

From the results in Figures 12 and 13, the engineer concludes that effects A and AE are significant. At this point, he hears that someone has already performed the analysis using the reliability DOE technique without "adjusting" and "transforming" the original data. He also worries that his adjustment to the original data could result in incorrect conclusions, so he stops the analysis.

Conclusion
In this article, we discussed how to use DOE++ to do life test and data analysis. The advantages of using reliability DOE can be easily seen by comparing the analyses in Examples 1 and 2. Using reliability DOE, a better lifetime distribution can be chosen, while in traditional DOE only the lognormal distribution can be used by employing the logarithm transformation. The significant effects can be identified correctly in reliability DOE, while in traditional DOE, erroneous conclusions may be drawn by "adjusting" suspensions and interval data. It is also worth noting that for the data set used in this article, there is only one suspension. If there were more suspensions and interval data, the advantages of using reliability DOE would be even more clear.

References
[1] C. F. Jeff Wu, Michael Hamada, Experiments: Planning, Analysis, and Parameter Design Optimization, John Wiley & Sons, New York, 2000.
[2] ReliaSoft Corporation, Experiment Design and Analysis Reference, ReliaSoft Publishing, Tucson, AZ, 2008.
 

Copyright © 2008 ReliaSoft Corporation, ALL RIGHTS RESERVED