Sequential Probability Ratio Test for Reliability Demonstration
The Sequential Probability Ratio Test (SPRT) was developed by Abraham Wald more than a half century ago . It is widely used in quality control in manufacturing and detection of anomalies in medical trials. In this article, we will explain the theory behind this method and illustrate its use in reliability engineering, especially in reliability demonstration test design. An example using the SPRT report template in Weibull++ is provided.
SPRT was originally developed as an inspection tool to determine whether a given lot meets the production requirements. Basically, a sequential test is a method by which items are tested in sequence (one after another). The test results are reviewed after each test. Two tests of significance are applied to the data accumulated up to that time.
Concept of SPRT
Let's first use a simple example to explain the principal behind SPRT. Two vendors provide the same component to a company. Although the components from the two companies look exactly the same, their lifetime distributions are different. Components from vendor A have a mean life of μ1 = 15, and components made by vendor B have a mean life of μ2 = 20. An unlabeled box of components was received by the company. We want to determine if the components are from vendor A or from vendor B by conducting a test. The test should meet the following requirements:
- If the component is indeed from vendor A, the chance of making a wrong claim that it is from vendor B should be less than α1 = 0.01.
- If the component is indeed from vendor B, the chance of making a wrong claim that it is from vendor A should be less than α2 = 0.05.
Therefore, we need to conduct two statistical hypothesis tests. Since we know μ2 > μ1, the two tests are one-sided tests. The first test is for vendor A:
The second one is for vendor B:
These two separate hypothesis tests are shown graphically below:
The top plot is for the first hypothesis test (vendor A). C1 is the critical value at a significance level of α1. If we take some samples and the sample mean is less than C1, then we accept , which is that the components are from vendor A. Otherwise, we accept , that the components are not from vendor A.
The bottom plot is for the second test (vendor B). C2 is the critical value at a significance level of α2. If a sample mean is greater than C2, then we accept that the component is from vendor B; otherwise, we accept , that the component is not from vendor B.
When we take samples for the life test, the resulting sample mean has one of the following values:
- Assume a sample with mean was drawn. For the test for vendor A, since it is less than C1, we accept that μ = μ1. For the test for vendor B, since it is less than C2, we accept that μ < μ2. The test is ended and we conclude that the component is from vendor A.
- Assume a sample with mean was drawn. For the test for vendor A, since it is greater than C1, we reject that μ = μ1. For the test for vendor B, since it is greater than C2, we accept that μ = μ2. The test is ended and we conclude that the component is from vendor B.
- Assume a sample with mean was drawn. For the test for vendor A, since it is less than C1, we accept that μ = μ1. For the test for vendor B, since it is greater than C2, we accept that μ = μ2. We conclude the component is from both vendor A and vendor B, which is impossible. Therefore, the test is not ended and more samples are needed.
With more and more samples, the sample mean will be closer to the true population mean. The test will end with a conclusion either from vendor A or from vendor B. This is the principal behind a sequential test. A sequential probability ratio test is based on this idea.
Calculation of SPRT
Now assume the lifetime t of the component follows an exponential distribution. Let θA = μA for vendor A and θB = μB for vendor B. The probability density function (pdf) of the exponential distribution is:
For an observed failure time t, if it is from vendor A, then the “probability” of observing it is:
where Δt is a very small time duration around t.
If the observation is from vendor B, then the “probability” of observing it is:
If the component is from vendor A, then Eqn. (2) will likely have a larger value than the one given in Eqn. (3), and vice versa.
The logarithm of the ratio of the above two probabilities is given by:
When there are more samples, the log-likelihood ratio becomes:
If the ratio is greater than a critical value U, then the chance that the samples are from vendor B is much larger than the chance that the samples are from vendor A. We can conclude that the samples are from vendor B.
If the ratio is less than a critical value L, then the chance that the samples are from vendor A is much larger than the chance that the samples are from vendor B. We can conclude that the samples are from vendor A.
If the ratio is between L and U, then no conclusion can be made. More samples are needed. The decision is made based on the following formula:
But what are the values for U and L? U and L are determined based on the two significance levels α1 and α2. The significance level is also called a Type I error. For details on Type I and Type II errors, please refer to https://www.weibull.com/hotwire/issue88/relbasics88.htm.
When the ratio is less than L, we accept vendor A:
When we accept vendor A, the probability of making the right decision (the component is from vendor A) should be greater than 1-α1, as required by the hypothesis test. The probability of making the wrong decision (the component is actually from vendor B) should be less than α2. Here α1 is the Type I error α and α2 is the Type II error β for the hypothesis test for vendor A.
Please note that Type I and Type II errors are related to a given statistical hypothesis test. Since SPRT combines two hypothesis tests together, it is very important to determine which one is the Type I error and which one is the Type II error.
When vendor A is accepted, based on the requirement for the Type I and Type II errors, we have:
From Eqns. (7) and (8), we set:
Similarly, when the ratio is larger than U, we accept vendor B:
When we accept vendor B, the probability of making the right decision (the component is indeed from vendor B) should be greater than 1-α2. The probability of making the wrong decision (the component is actually from vendor A) should be less than α1. Here α2 is the Type I error and α1 is the Type II error for the hypothesis test for vendor B. Therefore, we have:
From Eqns. (7) and (8), we can set:
Combining all the above equations, we get the decision formula for SPRT as the follows:
SPRT for Weibull Distribution
SPRT can be used for any distribution. The likelihood ratio can be calculated based on the assumed distribution. In this section, we will use the Weibull distribution to illustrate how it is used in a reliability requirement test. The probability density function for a Weibull distribution is given by:
η is the scale parameter.
b is the shape parameter. Note that here we do not use the traditional notation beta for the shape parameter because beta is used for the Type II error in this article.
Assume we want to test if a component is from a Weibull population with parameters of b1 and η1, or from a population with parameters of b2 and η2(η2 > η1). The two likelihood functions are:
The likelihood ratio is:
When b1 = b2, the above equation becomes:
The decision equation for the log-likelihood ratio R is:
L and U are given in Eqns. (9) and (12). Therefore, Eqn. (20) becomes:
Let's assume the lifetime of a component is described by a Weibull distribution with the shape parameter b = 1.5. We will use SPRT to determine if the component meets the following reliability requirements:
- A target reliability of 92% at 200 hours. If the component meets or exceeds the target reliability, the chance of rejecting it (i.e., Type I error or α error) should be less than 0.05. This is comparable to α2 in the previous section.
- A minimum reliability of 82% at 200 hours. If the component’s reliability is 82% or less, the probability of accepting it (i.e., Type II error or β error) should be less than 0.1. This is comparable to α1 in the previous section.
Our objectives are to:
- Calculate the acceptance and rejection line for the SPRT test.
- Determine whether to accept or reject the component based on a series of observed failure times.
Solution Using Manual Calculations
- Calculate η1 and
based on the reliability requirements. The reliability function for a Weibull
distribution is given by:
Therefore, η2 equals:
and η1 equals:
- Enter the calculated values for
into the decision equations from Eqn. (21).
The equation becomes:
- Calculate for each observed failure time. The observed time from each sequential test and the calculated results are shown next. (Note that in the table, negative rejection values were adjusted to 0.)
|ID||Ti||T||Acceptance Value||Rejection Value||Decision|
The plot of the data is shown next. The component is rejected at a failure time of 113 hours.
Solution Using the Weibull SPRT Template in Weibull++
The Synthesis version of Weibull++ includes a report template for calculating the SPRT results using a Weibull distribution and generating a plot of the results. To use the template:
- Add a new analysis workbook in an existing project, by choosing Insert > Reports and Plots > Analysis Workbook.
- Select the Based on Existing Template check box and then choose Weibull SPRT Template on the Standard tab.
- Click OK, then click Yes to create the workbook.
- Enter the reliability and risk requirement values, and the observed failure times in the white cells. The results and plot are shown next.
The resulting report provides all of the information that you obtained by doing the calculations manually and it automatically creates a plot.
Many published materials on SPRT only provide the simplified final formulas, such as Eqn. (21), for specific distributions for ease of use. In this article, we reviewed the basic theory of SPRT and illustrated its use in reliability engineering. It can be seen that it is a general tool that can be used for any distribution. Once the theory is understood, it is an easy task to develop your own SPRT for your applications.
 A. Wald, Sequential Analysis, John Wiley & Sons, Inc, New York, 1947.