The idea behind maximum likelihood parameter estimation is to determine the parameters that maximize the probability (likelihood) of the sample data. From a statistical point of view, the method of maximum likelihood is considered to be more robust (with some exceptions) and yields estimators with good statistical properties. In other words, MLE methods are versatile and apply to most models and to different types of data. In addition, they provide efficient methods for quantifying uncertainty through confidence bounds. Although the methodology for maximum likelihood estimation is simple, the implementation is mathematically intense. Using today's computer power, however, mathematical complexity is not a big obstacle. The MLE methodology is presented next.
Background Theory
This section presents the theory that underlies maximum likelihood estimation for complete data. If x is a continuous random variable with pdf:
where 
are k
unknown constant parameters which need to be estimated, conduct an experiment
and obtain N
independent observations, x1,
x2,...,xN. Then
the likelihood function is given by the following product:
(23)
The logarithmic likelihood function is given by:
The maximum likelihood estimators (MLE) of are
obtained by maximizing L
or 
.
By maximizing , which is
much easier to work with than L,
the maximum likelihood estimators (MLE) of
are the simultaneous solutions of k equations such that:
Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE solutions), this is not completely accurate. As it can be seen from the equations above, the MLE method is independent of any kind of ranks or plotting methods. For this reason, many times the MLE solution appears not to track the data on the probability plot. This is perfectly acceptable since the two methods are independent of each other and in no way suggests that the solution is wrong.
Illustrating the MLE Method Using the Exponential Distribution
To estimate 
, for
a sample of n
units (all tested to failure), first obtain the likelihood function:
Take the natural log of both sides:
Obtain 
,
and set it equal to zero:
Solve for or:
Notes on lambda
Note that the value of λ is an estimate because if we obtain another sample from the same population and re-estimate λ, the new value would differ from the one previously calculated.
In plain language, is an estimate of the true value of λ.
How close is the value of our estimate to the true value? To answer this question, one must first determine the distribution of the parameter, in this case λ. This methodology introduces a new term, confidence level, which allows us to specify a range for our estimate with a certain confidence level.
The treatment of confidence intervals is integral to reliability engineering and to all of statistics. (Confidence intervals are presented in Appendix A: A Brief Statistical Background.)
Illustrating the MLE Method Using the Normal Distribution
To obtain the MLE estimates for the mean, 
and
standard deviation, 
for the
normal distribution, start with the pdf
of the normal distribution which is given by:
If T1, T2, ... TN are known times-to-failure (and with no suspensions), then the likelihood function is given by:
Then:
Then taking the partial derivatives of
with respect to each one of the parameters and setting it equal to zero
yields:
(24)
and:
(25)
Solving Eqns. (24) and (25) simultaneously yields:
(26)
and:
(27)
These solutions are only valid for data with no suspensions, i.e. all units are tested to failure. In cases in which suspensions are present, the methodology changes and the problem becomes much more complicated.
Estimator
As mentioned above, the parameters obtained from maximizing the likelihood function are estimators of the true value. It is clear that the sample size determines the accuracy of an estimator. If the sample size equals the whole population, then the estimator is the true value. Estimators have properties such as unbiasedness, sufficiency, consistency and efficiency. Numerous books and papers deal with these properties and this coverage is beyond the scope of this reference. However, we would like to briefly address unbiasedness and consistency.
Unbiased Estimator
An estimator is said to be unbiased if and only if the estimator 
= d(X1,
X2,
..., XN) satisfies the condition
E[]
= 
for all
. Note
that E[X]
denotes the expected value of X and is defined by (for continuous
distributions):
This implies that the true value is not consistently underestimated nor overestimated by an unbiased estimator.
Consistent Estimator
An unbiased estimator that converges more closely to the true value as the sample size increases is called a consistent estimator. In the example above, the standard deviation of the normal distribution was obtained using MLE. This estimator of the true standard deviation is a biased one. It can be shown [ 4] that the consistent estimate of the variance and standard deviation for complete data (for the normal and lognormal distribution) is given by:
Note that for larger values of N, 
tends to 1.
See Also:
Appendix B: Parameter Estimation
Go
to Weibull.com
Go to ReliaSoft.com
©1998-2010. ReliaSoft Corporation. ALL RIGHTS RESERVED.