|
Using Multiple Reliability
Tools to Augment Limited Data
In last months
Hot Topics article, Joe the reliability engineer was forced
to make a decision regarding the B10 life of two products with very
limited data. Having to make an important decision based on only
a few data points is common when suppliers have expensive
hardware and/or limited access to testing equipment. In this
case, additional research is often necessary. Other reliability
tools, such as comparing test data to field data, relating
failure modes seen during the test to a failure modes and
effects analysis (FMEA) and even using engineering judgment may
be employed to substantiate the conclusions reached with limited
test data. This article will discuss some follow-up questions
that Joe might ask in order to strengthen his analysis.
To download the complete *.rso7 file
for this example, right-click here to
save the file to your computer. You must have Weibull++
installed in order to be able to open this file. Free
demonstration copies of the software are available for download
from
http://Download.ReliaSoft.com.
Background
For health and safety reasons, federal regulators had banned a
substance that was found in Product A. This ban required that
all manufacturers discontinue the use of Product A in three
months' time. After two months of frantically searching for a
suitable replacement, the engineers at Joes company found a
candidate, Product B. After discussions with management, Joe was
given 5 samples of Product B, which were made with rapid
prototyping (soft) tools. He rushed these samples to an
independent laboratory for testing. In the meantime, he found
the results from the 10 tests on Product A that had been ordered
by his predecessor. When the results from the tests on
Product B arrived, Joe quickly ran the analysis that was
described in
last month's issue
of Reliability HotWire. He found that the B10 life of
Product A had a median estimate of 311 hours, while the B10 life
of Product B
had a median estimate of 453 hours. However, because
Joe had so few samples, when he added 90% 2-sided confidence
bounds to the estimates, there was overlap between the estimates
(135 hours to 719 hours for Product A and 218 to 947 hours for
Product B). Therefore, Joe was unable to tell if there was a difference
between the two products at the 90% confidence level. This left
Joe feeling uneasy, but his company needed a decision in less
than 2 weeks regarding whether
Product B was a suitable substitute for Product A. So Joe decided to try to find more information about the
two products to help him make a decision.
Actions Taken
Joe starts by making some phone calls. First, he talks to his
contact, Connie, at the laboratory that had tested the products. He
asks her about the test procedure and what types of failure modes
were observed during the test of Product B. She tells him that
four of the failures of Product B were fatigue failures, which
were confirmed by the striations on the fracture surfaces of the
specimens. The other failure (the one that occurred at 402
hours) did not show any evidence of fatigue; rather, that
specimen failed due to a failure of the test apparatus. Joe also
asks Connie if she can remember anything about the testing of
Product A. Although those tests were run several years ago, she
looks up her notes from the tests. She says that six of the
tests on Product A were done four months earlier than the other
tests. She also says that the failure analysis showed that the
six early tests showed both fatigue and impact failures, but she
does not know which failure times were the impact failures. She
says the later four tests lasted much longer than the original
six and exhibited only fatigue failures.
Joe goes to his data file
(right-click here to save Joes
original file to your computer), duplicates his original
analysis
Folio and changes the name of the new Folio to Folio2.
In Folio2, he clicks the Alter Data Type icon.
In the Data Sheet Setup window
that appears, Joe selects the My data set contains
suspensions (right censored data) option and clicks OK.
This adds a column to the Data Sheet so that Joe can specify
that the failure at 402 hours was a suspension, since it failed
due to the failure of the test fixture. Joe then calculates the
Data Sheet and obtains the parameters shown in Figure 1.
Figure 1: Updating Data Point Classification for Product B
Next, Joe calls his friend
Fred, who works for a division of Joe's company that uses Product A in the field. Joe
asks Fred how long he thinks Product A typically lasts, and Fred
says that a "bad" Product A would last 2000 hours, while a
"good" Product A would last 3000 hours. Joe knows from his
Product A laboratory test data set that he has seen quite a few
failures at times significantly less than 2000 hours, so he asks
Fred if he has ever experienced a Product A that lasted less
than, say, 1500 hours. Fred cannot remember that happening in all
the years he has been using Product A, but says that he will
look back at the company's failure reporting and corrective action
(FRACAS) system to see what additional data he can find.
Joe is still puzzled by the
early failures seen in the Product A data, so he asks his
colleague Jen to let him look at the FMEA that has been done on
Product A. Fortunately for Joe, his company is very diligent in
ensuring that their FMEAs are living documents, and they have
updated the Product A FMEA frequently. He finds out the
following things from the FMEA:
- The primary failure mode
of Product A in the field is fatigue.
- The test results that he
has for Product A were done on a mixture of specimens. Six
specimens that were made with rapid prototyping tools were
tested first. Based on these tests, a small design change
was implemented. Four additional specimens that were made
with production (hard) tools were tested to validate the
design change.
Joe knows from his experience
on other jobs that products made with hard tools last longer
than those made with soft tools. His engineering judgment tells
him that he can expect an increase in life in Product B of about
two times when the switch to the hard tools is implemented.
Joe goes back to his Product A
data and updates the subset ID column to reflect his new
knowledge about the data set, as shown in Figure 2.
Figure 2: Specifying Product A Subset ID Data
He clicks the Batch Auto Run
icon.
In the Batch Auto Run window,
Joe selects both Product A Soft Tools and Product A Hard
Tools for consideration in the batch process in order to
separate the data into two Data Sheets based on the subset ID
column. The calculated parameters for these two data sets are
shown in Figures 3 and 4.
Figure 3: Parameters for Soft Tool Data of Product A
Figure 4: Parameters for Hard Tool Data of Product A
Just as he finishes separating
the Product A data, Joe receives an e-mail from his friend Fred
that contains a spreadsheet with times-to-failure data for
Product A from the field. This data set contains 119 suspensions
and 10 failures. He copies this data into a new Folio, called
Folio3. Figure 5 shows the calculated parameters for this data set.
Figure 5: Field Data for Product A
First, Joe looks at the
laboratory test results. He adds MultiPlot2, including the data
from the Product A Soft Tools, Product A Hard Tools, and
Product B Data Sheets in Folio2. Figure 6 shows the resulting MultiPlot.
Figure 6: Probability Plots of Laboratory Test Data for
Products A and B
The line on the left represents
the tests done on Product A made using soft tools, the middle
line represents the tests done on Product B made using soft
tools and the line on the right represents the tests done on
Product A made using hard tools. Joe is not surprised to see
that the Product A Soft Tools line has a different slope than
the others, as it includes failures from both impact and
fatigue, while the other two lines have only fatigue failures.
Also, he notes that the Product A Soft Tools line is well to
the left of the Product A Hard Tools line, which is what he
expected given his previous experience with rapid prototype and
production tooling.
Joe changes the plot type to
Contour Plot and selects the 90% confidence level to obtain the
plot shown in Figure 7.
Figure 7: Contour Plots for Laboratory Test Data for Products
A and B
The left contour represents
Product A made using soft tools, the middle contour represents
Product B made using soft tools, and the right contour
represents Product A made using hard tools. It is clear from
this plot that at a 90% confidence level these populations are
different.
Next, Joe adds MultiPlot3,
including Product A Hard Tools from Folio2 and the field data
for Product A from Folio3 in order to see how well the
laboratory test predicted the field data. This plot is shown in
Figure 8. Joe is pleased to see how well the laboratory test
predicted the life of Product A in the field. He quickly checks
the contour plots, shown in Figure 9, to ensure that there is
overlap in the contours at the 90% confidence level. (Note that
since there are many more data points in the field data set, the
resulting contour is smaller than that of the laboratory data
set.) This comparison gives Joe additional confidence that he can
make a decision based on the results of the laboratory tests.
Figure 8: Probability Plot of Laboratory and Field Data for
Product A made with Production Tooling
Figure 9: Contour Plot of Laboratory and Field Data for
Product A made with Production Tooling
Given the following findings,
Joe is confident that he can recommend Product B as a suitable
substitute for Product A:
- Products made using
production tooling last longer than those using soft
tooling. Joes experience says that the difference in life
is at least a factor of 2.
- Product B made using soft
tooling has a longer life than Product A made using soft
tooling, as seen in the probability and contour plots. In
spite of the fact that Product A using soft tooling showed
both impact and fatigue failures, every data point in the
Product B data set is longer than the Product A data set for
products made using soft tooling.
- Product B using soft
tooling exhibits the same failure modes as Product A using
hard tooling and Product A in the field.
- The analysis for Product A using
hard tooling laboratory data predicts the life of Product A
in the field very well.
|
