Profile Testing in RGA 7
During a development program, it is common practice for
systems to be subjected to operational testing in order
to evaluate the performance of the system under conditions
that represent actual use. It is also common for reliability
fixes to be implemented in conjunction with this testing
and to perform reliability growth analysis on the data obtained.
However, when the system must be tested for a variety of
different mission profiles, it can be a challenge to make
sure that the testing is applied in a balanced manner that
will yield data suitable for reliability growth analysis.
In this article, we show how reliability teams can use
RGA 7's new
Mission Profile Folios to incorporate operational mission
profiles as part of their planning to:
- Create and manage an operational test plan that
effectively balances all of the mission profiles that
need to be tested.
- Track the expected vs. actual testing conducted
for all mission profiles and validate that the testing
has been conducted in a manner that will yield data
sets that are appropriate for reliability growth analysis.
If there are any significant variations from the test
plan that could jeopardize the analysis results, RGA
can automatically group the data at specified "convergence
points" so the growth model can be applied appropriately.
Usually, stated mission
profile conditions are used for operational testing. These
mission profile conditions are typically general statements
that guide testing on an average basis. For example, a copier
might be required to print 10,000 pages by time T=15 days
and 20,000 pages by time T=30 days. In addition, the copier
is required to scan 200 documents by time T=15 days, 400
documents by time T=30 days, etc.
Because of practical constraints, these full mission
profile conditions are typically not repeated one after
the other during testing. Instead, the elements that make
up the mission profile conditions are tested under varying
schedules with the intent that, on average, the mission
profile conditions are met. In practice, reliability corrective
actions are generally incorporated into the system as a
result of this type of testing.
In order to have valid reliability growth assessments,
it is required that the operational mission profile be conducted
in a structured manner. Therefore, the testing methodology
described in this article involves convergence and stopping
points during the testing.
A stopping point is when the testing is halted
for the expressed purpose of incorporating delayed corrective
actions. While there may be more than one stopping point
during a particular testing phase, for simplicity, the methodology
with only one stopping point will be described in this example.
However, the methodology can be extended to the case of
more than one stopping point.
A convergence point is a time during the test
when all the operational mission profile tasks either meet
their expected averages or fall within an acceptable range.
At least three convergence points are required for a well-balanced
test and the end of the test must be a convergence point.
The test times between the convergence points do not have
to be the same.
The objective of having the convergence points is to
be able to apply the Crow Extended model directly in such
a way that the projection and other key reliability growth
parameters can be estimated in a valid fashion. To do this,
the grouped data methodology is applied. Note that the methodology
also can be used with the Crow-AMSAA (NHPP) model for a
simpler analysis without the ability to estimate projected
and growth potential reliability.
Let’s explore the concept
of operational mission profiles through a practical application.
A company that manufactures construction equipment is
planning the reliability test of their new backhoe loader.
The loader has four distinct functions that need to be tested:
- Traveling on asphalt roads.
- Loading up to 5,000lb using the front loader.
- Statically rotating 180 degrees in construction
- Digging up to 12ft using the backhoe.
To ensure that the test is balanced, the reliability
team decided to use operational mission profiles for all
four functions to be tested. They decided to require convergence
points at times T=2500, 5000 and 7200 hours.
Based on product specifications, they have specific usage
requirements for each of the four functions. For example,
in 2,500 hours of field use, it is expected that the average
user will have driven the loader for 2,000 miles on asphalt
roads, have used the front loader with 5,000lb loads for
18,750 times, used the backhoe to dig 12ft trenches for
4,990 times and performed 180 degree rotations in construction
zones for 28,125 times. The reliability team gathers all
this information to build an expected usage profile over
the 7200 test hours, as shown in Table 1.
Table 1: Expected usage for each of the 4 operational
|Cumulative test time
||Traveling distance (miles)
||180 degree rotations
In reality, it is almost impossible to stay within the
expected profiles for all of the functions at all times
during the test. Even the most organized and balanced test
will be ruled by special and common causes of variation
that will result in divergence from the expected usage profiles
at times during the test. The idea behind operational mission
profiles is to set convergence points as targets in the
test, where there will be a concerted effort to "catch up"
or "slow down" in order to meet that expected average at
the convergence point. In this example, this methodology
involves monitoring the actual usage profiles for all of
the four functions and using that input as actionable information
in order bring the usage to its expected average at predefined
points. It is essentially a test management methodology
that assures that the overall test is balanced and representative
of how the product will be used in the field.
RGA 7 has a built-in utility that allows the monitoring
of operational mission profiles and the automated grouping
of reliability growth test data into groups based on the
predefined convergence points.
When a new mission profile is added, the first step is
to specify the convergence points for the reliability growth
test, as shown in Figure 1.
Figure 1: Specifying convergence
points in RGA 7
The next step is to create operational mission profiles
for each one of the distinct functions, such as the expected
traveling distance in miles, as shown in Figure 2.
Figure 2: Setting expected
traveling distance profile
After the expected usage has been entered for each one
of the four profiles, the test can start. The reliability
team will be monitoring actual usage for each of the four
profiles, and take appropriate actions in order to meet
the expected usage goals at each of the three convergence
points. At the same time, failure time data can be logged
using a formal reliability growth methodology, in this case
the Crow Extended model.
Figure 3 shows the expected and actual usage profile
for the front loads over the duration of the reliability
growth test. We can see that the loads were lower than expected
at the beginning of the test, so the management action was
to accelerate front load usage to higher rates in order
to meet the first convergence point. During the second period,
from the first convergence point to the second convergence
point, the actual usage starting exceeding the expected
one, so the management action was to completely stop testing
front loads from the time when the expected usage was met
until the next convergence point of 5,000 hours.
Figure 3: Expected vs. actual
usage for front loads throughout the reliability growth
RGA 7 allows the user to plot expected vs. actual
usage for all the profiles simultaneously in order to get
a high level overview of the degree of test balance achieved,
as shown in Figure 4. From the graph we can see that all
profiles met their expected averages at the convergence
points of 2,500, 5,000 and 7,200 hours, so the execution
of this reliability growth test was successful in terms
of managing all the operational mission profiles simultaneously.
Figure 4: Expected vs. actual
usage for all profiles throughout the reliability growth
When failure times data are collected during the test,
as shown in Figure 5, the data can be grouped based on the
convergence points to assure that the overall results reflect
the balanced usage profiles for all four functions.
Figure 5: Failure times data
for the reliability growth test
By clicking the mission profile analysis icon,
we can associate an operational mission profile with this
data set, as shown in Figure 6.
Figure 6: Associating a mission
profile with a data set
Using this utility, the data set is automatically grouped
based on intervals specified by the convergence points in
the associated mission profile. The grouped data and the
final results are shown in Figure 7.
Figure 7: Grouped data and
results based on associated mission profile
Mission profiles are
a powerful way to simulate the expected field usage of a
complicated system with different distinct functions.
RGA 7 allows you to create mission profiles and provides
an easy way to visually track and manage that the actual
usage for each of the profiles meets their expected average
at the predefined convergence points. Then the same convergence
points can be used to automatically group data to assure
that the analysis is based on a balanced test. This ultimately
reduces the test management and execution complexity, while
bringing the reliability growth analysis results closer
to what the actual customer is going to be experiencing
in the field.
Portions reprinted, with
Crow, Larry, H., "A Methodology for
Managing Reliability Growth During Operational Mission Profile
Testing" Reliability and Maintainability Symposium, 2008,
pp: 48-53, 10.1109/RAMS.2008.4925768.© 2008 IEEE.