Reliability HotWire

Issue 99, May 2009

Hot Topics
Hands On: Reliability DOE with Covariates

In this article we present a step-by-step practical application of a reliability DOE using ReliaSoft's DOE++ software. We also introduce the concept of covariates, which are uncontrolled factors that can influence the response. Introducing covariates in experiment design can provide a more accurate analysis and interpretation of the results.

The Story
Joe works as a reliability engineer for a product development lab that produces power tools. During the last quarter, they performed a gap analysis on their reliability program plan. One of the results of the gap analysis was the need to adopt a "Design-for-Reliability" strategy, instead of just relying on testing to expose and design out failure modes. During the last product cycle, the strategy of relying solely on testing proved both costly and inefficient.

The new product cycle for their handheld, pneumatically powered nail gun tool has started, and Joe has a significantly smaller test budget than in the past. Moreover, the development schedule is more aggressive, since the company expects the competition to launch a new model within the next year.

Joe decides to use design of experiments in order to maximize product life (measured in cycles to failure). He considers conducting a reliability DOE using a five-factor, two level design. His control factors are operating pressure, piston material, piston to cylinder length ratio, drive blade diameter and valve O-ring material. When he is ready to design the experiment, he realizes that there is another factor that will influence the response. The housing cases that the test units will be assembled with have been finished with two different steel hardening processes. Based on the recommendations of the mechanical engineering team, the steel housing hardening process was changed from process annealing to full annealing, with a heating temperature from 400°C in the initial annealing to 800°C in the full annealing. The process change resulted in a much more robust housing case. Joe is worried that this uncontrolled variable will influence his results and mask the effect of his controlled variables. Based on the lot number, he knows the hardening process for each sample. He decides to use the hardening process for the housing of the power tool as a covariate in his experiment.

What are Covariates?
Covariates in DOE are variables that influence the response but are assumed not to interact with any of the other factors. Examples of covariates can be ambient conditions, suppliers, raw materials, operators, batches, setups, lots, shifts, etc.

Covariance analysis utilizes the relationship between the response variable and one or more variables for which observations are available in order to reduce the error term variability and make the study a more powerful one for comparing treatment effects [1].

A covariate, also called a concomitant variable, should not be affected by the treatments. If it is, then covariance analysis will fail to show some of the effects that the treatments have on the response variable. Using covariates in a model is an alternative approach to blocking.

Designing the Reliability Experiment
Joe uses DOE++ to design and analyze the experiment. Since he is dealing with reliability data, he selects Reliability DOE in step 1 of the Design Wizard, as shown in Figure 1.

Figure 1: Selecting Reliability DOE in the Design Wizard

His experiment will be conducted at two levels for each of the factors, so he chooses a two-level full factorial reliability design in step 2, as shown in Figure 2.

Figure 2: Selecting a Two Level Full Factorial Reliability Design

Based on requests from another group, Joe is expecting to remove some of the test units before failure. Therefore, he selects the option for the data to contain suspensions, as shown in Figure 3. He also selects 6 for the number of factors, which is the combination of five control factors and one covariate.

Figure 3: Specifying Data Options and Number of Factors

He then clicks the Factor Properties button and enters the factor names, units, type (either quantitative or qualitative) and respective low and high levels, as shown in Figure 4.

Figure 4: Defining Factor Properties

He clicks OK to close the Factor Properties window and then, in step 4 of the Design Wizard, he clicks the View in a separate window link to make a final check of his design, as shown in Figure 5.

Figure 5: Reviewing the Design

He closes the design review window and clicks Finish to create the design. In the Control Panel, he clicks the Response Properties icon.

In the Response Properties window, he enters the name and units for the response, as shown in Figure 6.

Figure 6: Defining Response Properties

Joe chooses to use a Weibull distribution for the analysis, since he knows from previous product data that the Weibull distribution can accurately describe the life of the nail gun products. He could also use lognormal, since it is usually a good fit for this type of product. He does not consider exponential, since a constant failure rate does not apply.

Based on discussions with the product team, Joe decides to limit the design to investigate only up to two-way interactions, since three-way and higher order interactions are considered to be insignificant for the specific application. To do this, he clicks the Select Effects icon.

He selects 2-Way Interaction in the Limit by Order field. Furthermore, since his factor F (Housing case heat treatment) is a covariate, he includes only the main effect of F on the response and clears all the two-way interactions of F with other factors. Figure 7 shows these settings (note that to select or clear more interactions, the user would have to scroll down).

Figure 7: Selecting Effects and Interactions to Include in the Analysis

Joe is now ready to run the experiment. He prepares the test area and requests the 64 test units with the specific combinations (treatments) from the prototyping department. A week later, all the nail guns are delivered and testing begins.

Two of the units are suspended before failure, because another department has requested samples for pneumatic integration testing.

Analyzing the Results
After the test is complete, Joe enters the response (i.e. the number of cycles to failure or suspension) for each of the treatments in DOE++, as shown in Figure 8.

Figure 8: The Complete Reliability DOE for the Nail Gun

Joe clicks the Calculate icon.

Figure 9 shows the results. The operating pressure, the driver blade diameter, the valve O-ring material and the interaction between the operating pressure and the O-ring material are significant factors. Also, the housing case heat treatment covariate is shown to be significant. Joe knows that he made a good choice by including this covariate in his analysis, since it could have masked the effects of other factors.

Figure 9: Analysis Results Including Covariate

For comparison purposes, Figure 10 shows the analysis results without including a covariate. Notice that the driver blade diameter, which was a significant factor in the original analysis, now shows up as insignificant. The exclusion of the covariate has resulted in the masking of the effects of the control factors in the response, which leads to inaccurate and misleading conclusions.

Figure 10: Analysis Results Excluding Covariate

Reducing the Model
Once Joe has identified the significant factors and interactions, he proceeds with creating a reduced model by excluding any non-significant factors and interactions.

He returns to the Select Effects window and clicks the Select Significant Effects button to select only those effects that have been determined to be significant, as shown in Figure 11, then clicks OK.

Figure 11: Selecting Significant Effects to Create a Reduced Model

He then clicks Calculate again to get the results of the reduced model, which are shown in Figure 12.

Figure 12: Reduced Model Results

Joe has now identified the significant factors that affect product life, and based on the experiment he can predict product life for different design combinations. He plans to use these predictions for the upcoming design review, where he can provide the team with specific reliability prediction information concerning alternative design choices.

He clicks the Prediction icon.

In the Prediction window, he enters two of the possible alternative scenarios that have been discussed. He then obtains characteristic life (eta) predictions, together with a 90% two-sided confidence interval, as shown in Figure 13.

Figure 13: Predicting Characteristic Life

Note that DOE++ enables reliability DOEs with complete and censored data by using maximum likelihood estimation instead of regression analysis. It also does not require the traditional DOE assumption that response values at any treatment level follow the normal distribution. In the case of a reliability DOE, lifetime distributions that are typically good models for most products, such as the Weibull, lognormal and exponential, are used. [2]

In this article we presented a step-by-step example of designing and analyzing a reliability DOE with a covariate. As this example demonstrates, using covariates in experiment design and analysis provides more clarity in the results and reduces the chance of significant factors being masked.

[1] Kutner, Michael H., Nachtsheim, Christopher J., Neter, John, and Li, William, Applied Linear Statistical Models, New York: McGraw-Hill/Irwin, 2005.
[2] ReliaSoft Corporation, Experiment Design and Analysis Reference, Tucson, AZ: ReliaSoft Publishing, 2008.

Copyright © 2009 ReliaSoft Corporation, ALL RIGHTS RESERVED