Comparison
of Fisher Matrix and Likelihood Ratio Confidence Bound Methods
In Weibull++,
there are several ways to calculate reliability confidence
bounds for different distributions. The Fisher matrix (FM)
method and the likelihood ratio bounds (LRB) method are
both used very often. Both methods are derived from the
fact that the parameters estimated are computed using the
maximum likelihood estimation (MLE) method. However, they
are based on different theories. The MLE estimates are based
on large sample normal theory, and are easy to compute.
However, when there are only a few failures, the large sample
normal theory is not very accurate. Thus, the FM bounds
interval could be very different from the true values.[1]
The LRB method is based on the ChiSquared distribution
assumption. It is generally better than FM bounds when the
sample size is small. In this article, we will compare these
two methods for different sample sizes using the Weibull
distribution.
Fisher Matrix Confidence Bounds
The bounds are calculated using the Fisher information
matrix. The inverse of the Fisher information matrix yields
the variancecovariance matrix, which provides the variance
of the parameters.[2] The bounds
on the parameters are then calculated using the following
equations:
where:

E(G) is the estimate of the mean value
of the parameter
G.

Var(G) is the variance of the parameter
G.

α
= 1 
CL, where
CL is the confidence level.

z_{a}
is the standard normal statistic.
The bounds for functions of the distribution parameters
(e.g. the reliability and the unreliability) also
can be calculated using the above equation. For example,
the variance of the reliability can be estimated from the
variance/covariance matrix of the distribution parameters.
Likelihood Ratio Confidence Bounds
The likelihood ratio bounds method is often preferred
over the Fisher Matrix confidence bound method in situations
where there are smaller sample sizes. For data sets with
a larger number of data points, there is no significant
difference in the results of these two methods. Likelihood
ratio bounds are calculated using the likelihood function
as follows:
where:

L(G) is the likelihood function for the
unknown parameter
G.

is the likelihood function calculated at the estimated
parameter value
G.

α = 1 
CL, where
CL is the confidence level.
 is the ChiSquared statistic
with
k degrees of freedom, where
k is the number of quantities jointly
estimated.
Similarly, the bounds of the reliability also can be
calculated based on the ChiSquared distribution.[3]
Comparison of the FM and LRB Methods
To illustrate the difference, we will compare the confidence
bounds over time with a fixed BX% using three data sets.
We use sample sizes of 5, 50 and 100 with the settings shown
in the following table.
Table 1: Sample Settings
Confidence Bound Method 
FM Method 
LRB Method 
BX 
5 
5 
Confidence Bound used 
1 sided 
1 sided 
Confidence Level 
0.9 
0.9 
Weibull 2P 
Beta=0.5, Eta=20 
Beta=0.5, Eta=20 
First we create a data set with a sample size of 5 data
points using the Monte Carlo tool from Weibull++,
as shown next.
Figure 1: Use Monte Carlo to generate
sample
The generated data set is shown next.
Figure 2: Generated data set for a
sample size of 5
On the Analysis tab, choose Maximum Likelihood (MLE)
for the Analysis Method, and Use Likelihood Ratio
as the Confidence Bound Method. Leave all other options
with the default values and then click the Calculate
button.
Using the QCP tool we can calculate the upper and lower
confidence bounds. For this example, choose Both One
Sided on the Confidence Bounds tab and then go to the
Basic Calculations tab and choose BX Information
and type "5" in the input box to reflect a 5% probability
of failure. Click Calculate. The results are shown
next.
Figure 3: QCP result with LRB method
In the Analysis tab, if you choose Use Fisher Matrix
bounds for the Confidence Bounds Method, you get similar
results for the time confidence bounds at B5. Table 2 illustrates
the results for the two different methods.
Table 2: Comparison of FM and LRB methods
and simulation results for a sample size of 5

FM Method 
LRB Method 
Simulation Results 
Upper Bound 
4.7155 
2.4311 
2.028566221 
Time 
0.3069 
0.3069 
0.144772350 
Lower Bound 
0.02 
0.0063 
0.004437272 
Confidence 
1S @0.9 
1S @0.9 
1S @0.9 
Bound Width 
4.6955 
2.4248 
2.024128949 
Bound Ratio 
235.775 
385.8888889 
457.1651593 
The simulation results in Table 2 were obtained from
SimuMatic with the following settings.
Figure 4: SimuMatic Setup window
Figure 5: SimuMatic Setup for a data
sample size of 5
SimuMatic generated 1,000 data sets for this example.
The beta and eta of the Weibull distribution for each of
the 1,000 data sets were estimated using the MLE method.
From the 1,000 paired beta and eta parameters, 1,000 B5
values were also calculated. To obtain the 1sided 90% confidence
bounds, these 1,000 B5 values first need to be sorted in
ascending order. The 100th value will be the lower bound
and the 900th value will be the upper bound. Since there
are no assumptions in obtaining the simulation bounds, the
results serve as the benchmark for evaluating confidence
bound estimation methods.
In Table 2, bound width and bound ratio are calculated
using the following formulas:
bound width = upper bound – lower bound
bound ratio = upper bound/lower bound
As we can see, the LRB bound width is narrower than the
FM bound width, and the LRB bound ratio is higher than the
FM bound ratio. Using the simulation results obtained with
SimuMatic, the comparison shows that the LRB result is much
closer to the simulation result than the FM result is.
Repeating the above procedures for sample sizes of 50
and 100, we get similar comparisons, which are shown in
Tables 3 and 4.
Table 3: Comparison of FM and LRB methods
and simulation results for a sample size of 50

Fisher Matrix Method 
Likelihood Ratio Method 
Simulation Results 
Upper bound 
0.2407 
0.2217 
0.1518388385 
Time 
0.0923 
0.0923 
0.05481979463 
Lower bound 
0.0354 
0.0321 
0.0184973181 
Confidence 
1S @ 0.9 
1S @0.9 
1S @0.9 
Bound Width 
0.2053 
0.1896 
0.13334152 
Bound Ratio 
6.7994 
6.9065 
8.208694778 
Table 4: Comparison of FM and LRB
methods and simulation results for a sample size of
100

Fisher Matrix Method 
Likelihood Ratio Method 
Simulation Results 
Upper bound 
0.1659 
0.1593 
0.109948402 
Time 
0.086 
0.086 
0.055857738 
Lower bound 
0.0446 
0.0426 
0.024601808 
Confidence 
1S @ 0.9 
1S @0.9 
1S @0.9 
Bound Width 
0.1213 
0.1167 
0.085346594 
Bound Ratio 
3.7197 
3.7394 
4.469118774 
With the increase in sample size, the difference between
the FM and LRB bound widths gets smaller, and both get very
close to the simulation bound width and bound ratios. This
supports our statement in the introduction that the LRB
method usually is more accurate than the FM bound for small
sample sizes, and there is not much difference between the
two methods if sample size is large enough.
References [1] Scott
A. Vander Wiel, William Q. Meeker, "Accuracy of Approximate
Confidence Bounds Using Censored Weibull Regression Data
from Accelerated Life Tests," IEEE Transactions on Reliability,
Vol. 39, No. 3, 1990, August. [2] ReliaSoft Corporation.
"Individual and Joint Parameter Bounds in Weibull++."
Reliability Hotwire Issue 101. 2009.
http://www.weibull.com/hotwire/issue101/relbasics101.htm.
[3] ReliaSoft Corporation, Life Data Analysis Reference,
Tucson, AZ: ReliaSoft Publishing, 2007.
