Reliability HotWire: eMagazine for the Reliability Professional
Reliability HotWire

Issue 60, February 2006

Hot Topics

Using Simulation to Design Reliability Demonstration Tests


Designing reliability demonstration tests is an integral part of any reliability program. These tests are often required to demonstrate customer reliability and confidence requirements. While it is desirable to be able to test a large population of units to failure in order to obtain information on a product's or design's reliability, time and resource constraints sometimes make this impossible. In cases such as these, a test can be run on a specified number of units, or for a specified amount of time, that will demonstrate that the product has met or exceeded a given reliability at a given confidence level. In the final analysis, the actual reliability of the units will of course remain unknown, but the reliability engineer will be able to state that certain specifications have been met. In addition to analytical methods (such as the Parametric Binomial, the Non-Parametric Binomial and the Exponential Chi-Squared methods), a simulation method is also possible.


This article explores the application of simulation and the SimuMatic tool to design a reliability demonstration test. SimuMatic is incorporated in Weibull++ 7. The figures in this article represent the analysis in the Weibull++ 7 utility.


Example 1

Let us assume that a reliability specification states that at T = 10 hr, the reliability must be 99%, or R(T = 10) = 99%, and at T = 20 hr, the reliability must be 90%, or R(T = 20) = 90%, at an 80% lower one-sided confidence level (L1S = 80%).


One way to meet this specification is to design a test that will demonstrate either of these requirements at L1S = 80% with the required parameters (for this example we will use the R(T = 10) = 99% @ L1S = 80% requirement). With SimuMatic, we can specify the underlying distribution, distribution parameters, sample size on test, censoring scheme, required reliability and associated confidence level. From these inputs, SimuMatic will solve via simulation for the time demonstrated at the specified reliability and confidence level (i.e. X  in the R(T = X) = 99% @ L1S = 80% formulation), as well as the expected test duration. If the demonstrated time is greater than the time requirement, this indicates that the test design would accomplish its required objective. Since there are multiple test designs that may accomplish the objective, multiple experiments should be performed until we arrive at an acceptable test design (i.e. number of units and test duration).


We start with a test design using a sample size of 10 with no censoring (i.e. all units to be tested to failure). We perform the analysis using RRX and 10,000 simulated data sets. The outcome is an expected test duration of 217 hr and a demonstrated time of 25 hr. This result is well above the stated requirement of 10 hr (note that in this case, the true value of T at a 50% CL, for R = 99%, is 40 hr, which gives us a ratio of 1.6 between true and demonstrated). Since this would demonstrate the requirement, we can then attempt to reduce the number of units or test time. Suppose that we need to bring the test time down to 100 hr (instead of the expected 217 hr). The test could then be designed using Type II censoring (i.e. any unit that has not failed by 100 hr is right censored) assuring completion by 100 hr. Again, we specify Type II censoring at 100 hr in SimuMatic, and we repeat the simulation with the same parameters as before. The simulation results in this case yield an expected test duration of 100 hr and a demonstrated time of 17 hr at the stated requirements. This result is also above our requirement. The next figure graphically shows the results of this experiment.


This process can then be repeated using different sample sizes and censoring schemes until we arrive at a desirable test plan.

This example explains the concept of using simulation to design reliability tests. The next example is a more detailed example that illustrates, in a step-by-step fashion, how the SimuMatic utility can be used to design a reliability test.


Example 2

A manufacturer wants to perform a demonstration test. The goal is to demonstrate that the one-sided 95% lower bound of the reliability at 100 hr is larger than 0.90. The engineers expect that about 12% of the specimens will fail in the first 500 hours of the test and about 20% of the specimens will fail by the end of 1000 hours. There is a time limit on the demonstration test; the test time is not to exceed 400 hours.


Step 1: Calculate the parameters. Using the Parameter Experimenter in the SimuMatic setup window and assuming a Weibull distribution, the values of
β and η can be calculated as shown next.



When you click Update, the values will appear in the Parameter area of the Main page of the SimuMatic setup window.

The reliability at 100 hr can be calculated as follows:



The value is larger than 0.90, so we can design a test to demonstrate the reliability. (If it were less than the required reliability, the product reliability would have to be improved first.)

In the SimuMatic setup window, set the Number of Points to 10 and set the Number of Data Sets to 1000.


Step 2: On the Analysis page, set the Analysis Method to Maximum Likelihood (MLE) (a preferred choice under the censoring scheme of this analysis) and leave the other fields as the default values.


Step 3: On the Censoring page, select Right censoring after a specific time and enter 400 for the Time value.



If the censoring time is too short, no failures will be generated for some simulation runs. The resulting test plan will be a conservative one. This means that the sample size generated by SimuMatic will be larger than the sample size of the true and best test plan.


Step 4: On the Reliabilities & Times page, enter 100 in the time column. This will record R(100) for each simulation.


Step 5: Click Generate. On the Sorted sheet of the SimuMatic Results Folio, read the value of R(100) that corresponds to 5.00%. If this value is less than 0.90, then go to Step 6.


Step 6: Increase the Number of Points and repeat the above steps until the goal is met.


This analysis estimates that 30 units are required to perform the demonstration test under the time constraints (results could vary depending on seed values used in simulation).

Copyright 2006 ReliaSoft Corporation, ALL RIGHTS RESERVED