|
Hands On: Reliability DOE with
Covariates In this
article we present a step-by-step practical application of a
reliability DOE using ReliaSoft's
DOE++ software. We
also introduce the concept of covariates, which are uncontrolled
factors that can influence the response. Introducing covariates
in experiment design can provide a more accurate analysis and
interpretation of the results.
The Story
Joe works as a reliability engineer for a product development
lab that produces power tools. During the last quarter, they
performed a gap analysis on their reliability program plan. One
of the results of the gap analysis was the need to adopt a
"Design-for-Reliability" strategy, instead of just relying on
testing to expose and design out failure modes. During the last
product cycle, the strategy of relying solely on testing proved
both costly and inefficient.
The new product cycle for their
handheld, pneumatically powered nail gun tool has started, and
Joe has a significantly smaller test budget than in the past.
Moreover, the development schedule is more aggressive, since the
company expects the competition to launch a new model within the
next year.
Joe decides to use design of
experiments in order to maximize product life (measured in
cycles to failure). He considers conducting a reliability DOE
using a five-factor, two level design. His control factors are
operating pressure, piston material, piston to cylinder length
ratio, drive blade diameter and valve O-ring material. When he
is ready to design the experiment, he realizes that there is
another factor that will influence the response. The housing
cases that the test units will be assembled with have been
finished with two different steel hardening processes. Based on
the recommendations of the mechanical engineering team, the
steel housing hardening process was changed from process
annealing to full annealing, with a heating temperature from
400°C in the initial annealing to 800°C in the full annealing.
The process change resulted in a much more robust housing case.
Joe is worried that this uncontrolled variable will influence
his results and mask the effect of his controlled variables.
Based on the lot number, he knows the hardening process for each
sample. He decides to use the hardening process for the housing
of the power tool as a covariate in his experiment.
What are Covariates?
Covariates in DOE are variables that influence the response but
are assumed not to interact with any of the other factors.
Examples of covariates can be ambient conditions, suppliers, raw
materials, operators, batches, setups, lots, shifts, etc.
Covariance analysis utilizes
the relationship between the response variable and one or more
variables for which observations are available in order to
reduce the error term variability and make the study a more
powerful one for comparing treatment effects [1].
A covariate, also called a
concomitant variable, should not be affected by the treatments.
If it is, then covariance analysis will fail to show some of the
effects that the treatments have on the response variable. Using
covariates in a model is an alternative approach to blocking.
Designing the Reliability
Experiment
Joe uses DOE++ to design and analyze the experiment.
Since he is dealing with reliability data, he selects
Reliability DOE in step 1 of the Design Wizard, as shown in
Figure 1.
Figure 1: Selecting Reliability DOE in the Design Wizard
His experiment will be
conducted at two levels for each of the factors, so he chooses a
two-level full factorial reliability design in step 2, as shown
in Figure 2.
Figure 2: Selecting a Two Level Full Factorial Reliability
Design
Based on requests from another
group, Joe is expecting to remove some of the test units before
failure. Therefore, he selects the option for the data to
contain suspensions, as shown in Figure 3. He also selects 6
for the number of factors, which is the combination of five
control factors and one covariate.
Figure 3: Specifying Data Options and Number of Factors
He then clicks the Factor
Properties button and enters the factor names, units, type
(either quantitative or qualitative) and respective low and high
levels, as shown in Figure 4.
Figure 4: Defining Factor Properties
He clicks OK
to close the Factor Properties window and then, in step 4 of the
Design Wizard, he clicks the View in a separate window
link to make a final check of his design, as shown in Figure 5.
Figure 5: Reviewing the Design
He closes the
design review window and clicks Finish to create the
design. In the Control Panel, he clicks the Response
Properties
icon.
In the Response
Properties window, he enters the name and units for the
response, as shown in Figure 6.
Figure 6: Defining Response Properties
Joe chooses to use a Weibull
distribution for the analysis, since he knows from previous
product data that the Weibull distribution can accurately
describe the life of the nail gun products. He could also use
lognormal, since it is usually a good fit for this type of
product. He does not consider exponential, since a constant
failure rate does not apply.
Based on discussions with the
product team, Joe decides to limit the design to investigate
only up to two-way interactions, since three-way and higher
order interactions are considered to be insignificant for the
specific application. To do this, he clicks the Select
Effects icon.
He selects 2-Way Interaction
in the Limit by Order
field. Furthermore, since his factor F (Housing case heat
treatment) is a covariate, he includes only the main effect of F
on the response and clears all the two-way interactions of F
with other factors. Figure 7 shows these settings (note that to
select or clear more interactions, the user would have to scroll
down).
Figure 7: Selecting Effects and Interactions to Include in
the Analysis
Joe is now ready to run the
experiment. He prepares the test area and requests the 64 test
units with the specific combinations (treatments) from the
prototyping department. A week later, all the nail guns are
delivered and testing begins.
Two of the units are suspended
before failure, because another department has requested samples
for pneumatic integration testing.
Analyzing the Results
After the test is complete, Joe enters the response (i.e.
the number of cycles to failure or suspension) for each of the
treatments in DOE++, as shown in Figure 8.
 
Figure 8: The Complete Reliability DOE for the Nail Gun
Joe clicks the Calculate
icon.
Figure 9 shows the results. The
operating pressure, the driver blade diameter, the valve O-ring
material and the interaction between the operating pressure and
the O-ring material are significant factors. Also, the housing
case heat treatment covariate is shown to be significant. Joe
knows that he made a good choice by including this covariate in
his analysis, since it could have masked the effects of other
factors.
Figure 9: Analysis Results Including Covariate
For comparison
purposes, Figure 10 shows the analysis results without including
a covariate. Notice that the driver blade diameter, which was a
significant factor in the original analysis, now shows up as
insignificant. The exclusion of the covariate has resulted in
the masking of the effects of the control factors in the
response, which leads to inaccurate and misleading conclusions.
Figure 10: Analysis Results Excluding Covariate
Reducing the Model
Once Joe has identified the significant factors and
interactions, he proceeds with creating a reduced model by
excluding any non-significant factors and interactions.
He returns to the Select
Effects window and clicks the Select Significant Effects
button to select only those effects that have been determined to
be significant, as shown in Figure 11, then clicks OK.
Figure 11: Selecting Significant Effects to Create a Reduced
Model
He then clicks Calculate
again to get the results of the reduced model, which are shown
in Figure 12.
Figure 12: Reduced Model Results
Prediction
Joe has now identified the significant factors that affect
product life, and based on the experiment he can predict product
life for different design combinations. He plans to use these
predictions for the upcoming design review, where he can provide
the team with specific reliability prediction information
concerning alternative design choices.
He clicks the Prediction
icon.
In the Prediction window, he
enters two of the possible alternative scenarios that have been
discussed. He then obtains characteristic life (eta)
predictions, together with a 90% two-sided confidence interval,
as shown in Figure 13.
Figure 13: Predicting Characteristic Life
Note that DOE++
enables reliability DOEs with complete and censored data by
using maximum likelihood estimation instead of regression
analysis. It also does not require the traditional DOE
assumption that response values at any treatment level follow
the normal distribution. In the case of a reliability DOE,
lifetime distributions that are typically good models for most
products, such as the Weibull, lognormal and exponential, are
used. [2]
Conclusion
In this article we presented a step-by-step example of designing
and analyzing a reliability DOE with a covariate. As this
example demonstrates, using covariates in experiment design and
analysis provides more clarity in the results and reduces the
chance of significant factors being masked.
References
[1] Kutner, Michael H., Nachtsheim, Christopher
J., Neter, John, and Li, William, Applied Linear Statistical
Models, New York: McGraw-Hill/Irwin, 2005.
[2] ReliaSoft Corporation, Experiment Design
and Analysis Reference, Tucson, AZ: ReliaSoft Publishing,
2008.
|
