<<< Back to Index
This Month's Tip >>>

Sequential Probability Ratio Test for Reliability Demonstration

 

 

The Sequential Probability Ratio Test (SPRT) was developed by Abraham Wald more than a half century ago [1]. It is widely used in quality control in manufacturing and detection of anomalies in medical trials. In this article, we will explain the theory behind this method and illustrate its use in reliability engineering, especially in reliability demonstration test design. An example using the SPRT report template in Weibull++ is provided.

SPRT Theory

SPRT was originally developed as an inspection tool to determine whether a given lot meets the production requirements. Basically, a sequential test is a method by which items are tested in sequence (one after another). The test results are reviewed after each test. Two tests of significance are applied to the data accumulated up to that time.

Concept of SPRT

Let's first use a simple example to explain the principal behind SPRT. Two vendors provide the same component to a company. Although the components from the two companies look exactly the same, their lifetime distributions are different. Components from vendor A have a mean life of μ1 = 15, and components made by vendor B have a mean life of μ2 = 20. An unlabeled box of components was received by the company. We want to determine if the components are from vendor A or from vendor B by conducting a test. The test should meet the following requirements:

Therefore, we need to conduct two statistical hypothesis tests. Since we know μ2 > μ1, the two tests are one-sided tests. The first test is for vendor A:

test 1

The second one is for vendor B:

test 2

These two separate hypothesis tests are shown graphically below:

plot vendor A and B

The top plot is for the first hypothesis test (vendor A). C1 is the critical value at a significance level of α1. If we take some samples and the sample mean is less than C1, then we accept Equation, which is that the components are from vendor A. Otherwise, we accept Equation, that the components are not from vendor A.

The bottom plot is for the second test (vendor B). C2 is the critical value at a significance level of α2. If a sample mean is greater than C2, then we accept Equation that the component is from vendor B; otherwise, we accept Equation, that the component is not from vendor B.

When we take samples for the life test, the resulting sample mean has one of the following values:

With more and more samples, the sample mean will be closer to the true population mean. The test will end with a conclusion either from vendor A or from vendor B. This is the principal behind a sequential test. A sequential probability ratio test is based on this idea.

Calculation of SPRT

Now assume the lifetime t of the component follows an exponential distribution. Let θA = μA for vendor A and θB = μB for vendor B. The probability density function (pdf) of the exponential distribution is:

Equation (1)

For an observed failure time t, if it is from vendor A, then the “probability” of observing it is:

Equation (2)

where Δt is a very small time duration around t.

If the observation is from vendor B, then the “probability” of observing it is:

Equation (3)

If the component is from vendor A, then Eqn. (2) will likely have a larger value than the one given in Eqn. (3), and vice versa.

The logarithm of the ratio of the above two probabilities is given by:

Equation (4)

When there are more samples, the log-likelihood ratio becomes:

Equation (5)

If the ratio is greater than a critical value U, then the chance that the samples are from vendor B is much larger than the chance that the samples are from vendor A. We can conclude that the samples are from vendor B.

If the ratio is less than a critical value L, then the chance that the samples are from vendor A is much larger than the chance that the samples are from vendor B. We can conclude that the samples are from vendor A.

If the ratio is between L and U, then no conclusion can be made. More samples are needed. The decision is made based on the following formula:

Equation (6)

But what are the values for U and L? U and L are determined based on the two significance levels α1 and α2. The significance level is also called a Type I error. For details on Type I and Type II errors, please refer to http://www.weibull.com/hotwire/issue88/relbasics88.htm.

When the ratio is less than L, we accept vendor A:

Equation (7)

When we accept vendor A, the probability of making the right decision (the component is from vendor A) should be greater than 1-α1, as required by the hypothesis test. The probability of making the wrong decision (the component is actually from vendor B) should be less than α2. Here α1 is the Type I error α and α2 is the Type II error β for the hypothesis test for vendor A.

Please note that Type I and Type II errors are related to a given statistical hypothesis test. Since SPRT combines two hypothesis tests together, it is very important to determine which one is the Type I error and which one is the Type II error.

When vendor A is accepted, based on the requirement for the Type I and Type II errors, we have:

Equation (8)

From Eqns. (7) and (8), we set:

Equation (9)

Similarly, when the ratio is larger than U, we accept vendor B:

Equation (10)

When we accept vendor B, the probability of making the right decision (the component is indeed from vendor B) should be greater than 1-α2. The probability of making the wrong decision (the component is actually from vendor A) should be less than α1. Here α2 is the Type I error and α1 is the Type II error for the hypothesis test for vendor B. Therefore, we have:

Equation (11)

From Eqns. (7) and (8), we can set:

Equation (12)

Combining all the above equations, we get the decision formula for SPRT as the follows:

Equation (13)

Which is:

Equation (14)

SPRT for Weibull Distribution

SPRT can be used for any distribution. The likelihood ratio can be calculated based on the assumed distribution. In this section, we will use the Weibull distribution to illustrate how it is used in a reliability requirement test. The probability density function for a Weibull distribution is given by:

Equation (15)

where:

Assume we want to test if a component is from a Weibull population with parameters of b1 and η1, or from a population with parameters of b2 and η22  >  η1). The two likelihood functions are:

Equation (16)

and

Equation (17)

The likelihood ratio is:

Equation (18)

When b1 = b2, the above equation becomes:

Equation (19)

The decision equation for the log-likelihood ratio R is:

Equation (20)

L and U are given in Eqns. (9) and (12). Therefore, Eqn. (20) becomes:

Equation (21)

Example

Let's assume the lifetime of a component is described by a Weibull distribution with the shape parameter b = 1.5. We will use SPRT to determine if the component meets the following reliability requirements:

Our objectives are to:

Solution Using Manual Calculations

  1. Calculate η1 and η2 based on the reliability requirements. The reliability function for a Weibull distribution is given by:

    Equation

    Therefore, η2 equals:

     Equation

    and η1 equals:

    Equation

  2. Enter the calculated values for η1 and η2 into the decision equations from Eqn. (21).

    Equation

    The equation becomes:

    Equation

  3. Calculate equation for each observed failure time. The observed time from each sequential test and the calculated results are shown next. (Note that in the table, negative rejection values were adjusted to 0.)
ID Ti T Acceptance Value Rejection Value Decision
1 629 15,775.24 76,651.04125 0 Continue
2 369 22,863.5 97,964.88014 0 Continue
3 685 40,791.66 119,278.719 0 Continue
4 270 45,228.22 140,592.5579 14,209.44008 Continue
5 682 63,038.74 161,906.3968 35,523.27897 Continue
6 194 65,740.84 183,220.2357 56,837.11786 Continue
7 113 66,942.05 204,534.0746 78,150.95675 Reject

The plot of the data is shown next. The component is rejected at a failure time of 113 hours.

 plot

Solution Using the Weibull SPRT Template in Weibull++

The Synthesis version of Weibull++ includes a report template for calculating the SPRT results using a Weibull distribution and generating a plot of the results. To use the template:

  1. Add a new analysis workbook in an existing project, by choosing Insert > Reports and Plots > Analysis Workbook.
  2. Select the Based on Existing Template check box and then choose Weibull SPRT Template on the Standard tab.

Workbook Wizard

  1. Click OK, then click Yes to create the workbook.
  2. Enter the reliability and risk requirement values, and the observed failure times in the white cells. The results and plot are shown next.

data and plot

The resulting report provides all of the information that you obtained by doing the calculations manually and it automatically creates a plot.

Conclusion

Many published materials on SPRT only provide the simplified final formulas, such as Eqn. (21), for specific distributions for ease of use. In this article, we reviewed the basic theory of SPRT and illustrated its use in reliability engineering. It can be seen that it is a general tool that can be used for any distribution. Once the theory is understood, it is an easy task to develop your own SPRT for your applications.

Reference

[1] A. Wald, Sequential Analysis, John Wiley & Sons, Inc, New York, 1947.

 
ReliaSoft