Issue 34, December 2003

One of the most confusing concepts to a novice reliability engineer is estimating the precision of an estimate. This is an important concept in the field of reliability engineering, leading to the use of confidence intervals. This subject can be confusing, as confidence is "the probability of a probability." However, the use of confidence is becoming more and more common as more organizations include confidence bounds in their reliability requirements. In this article, we will try to briefly present the concept in relatively simple terms and based on solid common sense.

The Marble Analogy
To illustrate this concept, consider the case in which there are millions of perfectly mixed black and white marbles in a large swimming pool. Our job is to estimate the percentage of black marbles. The only way to be absolutely certain about the exact percentage of marbles in the pool is to accurately count every marble and calculate the percentage. However, this is too time- and resource-intensive to be a viable option, so we need to come up with a way of estimating the percentage of black marbles in the pool. In order to do this, we will take a relatively small sample of marbles from the pool and then count how many black marbles are in the sample.

First, pick out a small sample of marbles and count the black ones. Let's say you picked out ten marbles and counted four black marbles. Based on this, your estimate would be that 40% of the marbles are black.

If you put the ten marbles back in the pool and repeated this example again, you might get six black marbles, changing your estimate to 60% black marbles. Which of the two is correct? Both estimates are correct. As you repeat this experiment over and over again, you might find out that this estimate is usually between X1% and X2% and you can assign a percentage to the number of times your estimate falls between these limits. For example, you notice that 90% of the time this estimate is between X1% and X2%.

If we now repeat the experiment and pick out 1,000 marbles, we might get results for the number of black marbles, such as 545, 570, 530, etc., for each trial. The range of our estimates in this case will be much narrower than before. For example, we observe that 90% of the time, the number of black marbles will now be from Y1% to Y2%, where X1% < Y1% and X2% > Y2%, thus giving us a narrower estimate interval. The same principle is true for confidence intervals; the larger the sample size, the narrower the confidence intervals.

We will now look at how this phenomenon relates to reliability. In general, the reliability engineer's task is to determine the probability of failure, or reliability, of the population of units in question. However, we will never know the exact reliability value of the population unless we are able to obtain and analyze the failure data for every single unit in the population. Since this usually is not a realistic situation, then the task is to estimate the reliability based on a sample, much like estimating the number of black marbles in the pool. If we perform ten different reliability tests for our units and analyze the results, we will obtain slightly different parameters for the distribution each time and, thus, slightly different reliability results. However, by employing confidence bounds, we obtain a range within which these reliability values are likely to occur a certain percentage of the time. This helps us gauge the utility of the data and the accuracy of the resulting estimates. Plus, it is always useful to remember that each parameter is an estimate of the true parameter, one that is unknown to us. This range of plausible values is called a "confidence bound" or "confidence interval."

Two-Sided Confidence Bounds
When we use two-sided confidence bounds (or intervals), we are looking at a closed interval where a certain percentage of the population is likely to lie. That is, we determine the values, or bounds, between which a specified percentage of the population lies. For example, when dealing with 90% two-sided confidence bounds of (X,Y), we are saying that 90% of the population lies between X and Y with 5% less than X and 5% greater than Y.

One-Sided Confidence Bounds
One-sided confidence bounds are essentially an open-ended version of two-sided bounds. A one-sided bound defines the point where a certain percentage of the population is either higher or lower than the defined point. This means that there are two types of one-sided bounds: upper and lower. An upper one-sided bound defines a point that a certain percentage of the population is less than. Conversely, a lower one-sided bound defines a point that a specified percentage of the population is greater than. For example, if X is a 95% upper one-sided bound, this would indicate that 95% of the population is less than X. If X is a 95% lower one-sided bound, this would indicate that 95% of the population is greater than X.

Care must be taken to differentiate between one- and two-sided confidence bounds, as these bounds can take on identical values at different percentage levels. For example, in the figures above, we see bounds on a hypothetical distribution. Assuming this is the same distribution in all of the figures, we see that X marks the spot below which 5% of the distribution's population lies. Similarly, Y represents the point above which 5% of the population resides. Therefore, X and Y represent the 90% two-sided bounds, since 90% of the population lies between the two points. However, X also represents the lower one-sided 95% confidence bound, since 95% of the population lies above that point, and Y represents the upper one-sided 95% confidence bound, since 95% of the population is below Y. It is important to be sure of the type of bounds you are dealing with, particularly as both upper and lower one-sided bounds can be displayed simultaneously in Weibull++ and ALTA.

Copyright 2003 ReliaSoft Corporation, ALL RIGHTS RESERVED