What Are Confidence
One of the most confusing concepts to a novice reliability engineer is estimating the precision of an estimate. This is an important concept in the field of reliability engineering, leading to the use of confidence intervals. This subject can be confusing, as confidence is "the probability of a probability." However, the use of confidence is becoming more and more common as more organizations include confidence bounds in their reliability requirements. In this article, we will try to briefly present the concept in relatively simple terms and based on solid common sense.
The Marble Analogy
First, pick out a small sample of marbles and count the black ones. Let's say you picked out ten marbles and counted four black marbles. Based on this, your estimate would be that 40% of the marbles are black.
If you put the ten marbles back in the pool and repeated this example again, you might get six black marbles, changing your estimate to 60% black marbles. Which of the two is correct? Both estimates are correct. As you repeat this experiment over and over again, you might find out that this estimate is usually between X1% and X2% and you can assign a percentage to the number of times your estimate falls between these limits. For example, you notice that 90% of the time this estimate is between X1% and X2%.
If we now repeat the experiment and pick out 1,000 marbles, we might get results for the number of black marbles, such as 545, 570, 530, etc., for each trial. The range of our estimates in this case will be much narrower than before. For example, we observe that 90% of the time, the number of black marbles will now be from Y1% to Y2%, where X1% < Y1% and X2% > Y2%, thus giving us a narrower estimate interval. The same principle is true for confidence intervals; the larger the sample size, the narrower the confidence intervals.
We will now look at how this phenomenon relates to reliability. In general, the reliability engineer's task is to determine the probability of failure, or reliability, of the population of units in question. However, we will never know the exact reliability value of the population unless we are able to obtain and analyze the failure data for every single unit in the population. Since this usually is not a realistic situation, then the task is to estimate the reliability based on a sample, much like estimating the number of black marbles in the pool. If we perform ten different reliability tests for our units and analyze the results, we will obtain slightly different parameters for the distribution each time and, thus, slightly different reliability results. However, by employing confidence bounds, we obtain a range within which these reliability values are likely to occur a certain percentage of the time. This helps us gauge the utility of the data and the accuracy of the resulting estimates. Plus, it is always useful to remember that each parameter is an estimate of the true parameter, one that is unknown to us. This range of plausible values is called a "confidence bound" or "confidence interval."
Care must be taken to differentiate between one- and two-sided confidence bounds, as these bounds can take on identical values at different percentage levels. For example, in the figures above, we see bounds on a hypothetical distribution. Assuming this is the same distribution in all of the figures, we see that X marks the spot below which 5% of the distribution's population lies. Similarly, Y represents the point above which 5% of the population resides. Therefore, X and Y represent the 90% two-sided bounds, since 90% of the population lies between the two points. However, X also represents the lower one-sided 95% confidence bound, since 95% of the population lies above that point, and Y represents the upper one-sided 95% confidence bound, since 95% of the population is below Y. It is important to be sure of the type of bounds you are dealing with, particularly as both upper and lower one-sided bounds can be displayed simultaneously in Weibull++ and ALTA.
Copyright 2003 ReliaSoft Corporation, ALL RIGHTS RESERVED