A Blueprint for Implementing a Comprehensive Reliability Engineering Program
Section 3 of 7: Reliability Testing
Reliability testing is the cornerstone of a reliability engineering program. It provides the most detailed forms of data, because the conditions under which the data are collected can be carefully controlled and monitored. Furthermore, the reliability tests can be designed to uncover particular suspected failure modes and other problems. The type of reliability testing a product undergoes will change along different points of its life cycle, but the overriding goal is to insure that data from all or most of the tests were generated under similar enough conditions so that an "apples-to-apples" comparison can be made of the product's reliability characteristics at different points in the product's life. It is for this reason that consistent and thorough reliability specifications and a standard definition of failure are up-front requirements to implementing reliability testing.
A properly designed series of tests, particularly during the product's earlier design stages, can generate data that would be useful in the implementation of a reliability growth tracking program. This will provide information helpful in making management decisions regarding scheduling, development cost projections and so forth. This information will also be useful in planning the development cycle of future products.
An important requirement for designing useful reliability tests is to have a good idea of how the product is actually going to be used in the field. The tests should be based on a realistic expectation of the customer usage, rather than estimates or "gut feelings" about the way the customer will use the product. Tests based on mere speculation may result in a product that has not been rigorously tested and consequently may run into operational difficulties due to use stress levels being higher than anticipated. On the other hand, tests that are designed with a strong basis of information on how the product will be used will be more realistic and result in an optimized design that will exhibit fewer failures in the field.
Customer usage profiles can be designed to actively gather information on how the customers are actually using an organization's product. This design can range from a simple questionnaire to a sophisticated instrumentation within the product that feeds back detailed information about its operation. An incentive is often useful to get customers to sign on for a usage measurement program, particularly if it is an intrusive process that involves the installation of data collection equipment. Additionally, customers are often eager to participate in these programs in the knowledge that the information that they provide will ultimately result in a more reliable and user-friendly product.
In many cases, the type of testing that a product undergoes will change as the product's design becomes mature and the product moves from the initial design stages to final design release and production. Nevertheless, it is a good practice to continue to collect internally-generated data concerning the product's reliability performance throughout the life cycle of the product. This will strengthen the reliability growth analysis and help provide correlation between internal test results and field data. A brief summary of the various types of reliability tests is presented next.
Development testing occurs during the early phases of the product's life cycle, usually from project inception to product design release. It is vital to be able to characterize the reliability of the product as it progresses through its initial design stages so that the reliability specifications will be met by the time the product is ready for release. With a multitude of design stages and changes that could affect the product's reliability, it is necessary to closely monitor how the product's reliability grows and changes as the product design matures. There are a number of different test types that can be run during this phase of a product's life cycle to provide useful reliability information:
- Component-level Testing: Although component-level testing can continue throughout the development phase of a product, it is most likely to occur very early in the process. This may be due to the unavailability of parts in the early stages of the development program. There may also be special interest in the performance of a specific component if it has been radically redesigned, or if there is a separate or individual reliability specification for that component. In many cases, component-level testing is undertaken to begin characterizing a product's reliability when full system-level test units are unavailable or prohibitively expensive. However, system-level reliability characterization can be achieved through component-level testing. This is possible if sufficient understanding exists to characterize the interaction of the components. If this is the case, the system-level reliability can be modeled based on the configuration of components and the results of component reliability testing.
- System-level Testing: Although the results of component-level tests can be used to characterize the reliability of the entire system, the ideal approach is to test the entire system, particularly if that is how the reliability is specified. That is, if the technical specifications set forth a reliability goal for a specific system or configuration of components, that entire system or configuration should be tested to compare the actual performance with the stated goal. Although early system-level test units may be difficult to obtain, it is advisable to perform reliability tests at the system level as early in the development process as possible. At the very least, comprehensive system-level testing should be performed immediately prior to the product's release for manufacturing, in order to verify design reliability. During such system-level reliability testing, the units under test should be from a homogeneous population and should be devoted solely to the specific reliability test. The results of the reliability test could be skewed or confounded by "piggybacking" other tests along with it, and this practice should be avoided. A properly conducted system-level reliability test will be able to provide valuable engineering information above and beyond the raw reliability data.
- Environmental and Accelerated Testing: It may be necessary in some cases to institute a series of tests in which the system is tested at extreme environmental conditions, or with other stress factors accelerated above the normal levels of use. It may be that the product would not normally fail within the time constraints of the test, and, in order to get meaningful data within a reasonable time, the stress factors must be accelerated. In other cases, it may be necessary to simulate different operating environments based on where the product will be sold or operated. Regardless of the cause, tests like these should be designed, implemented and analyzed with care. Depending on the nature of the accelerating stress factors, it is easy to draw incorrect conclusions from the results of these tests. A good understanding of the proper accelerating stresses and the design limits of the product are necessary to be able to implement a meaningful accelerated reliability test. For example, you would not want to design an accelerated test that would overstress the product and introduce failure modes that would not normally be encountered in the field. Given that there have been a lot of incredible claims about the capability of accelerated testing and the improbably high acceleration factors that can supposedly be produced, care needs to be taken when setting up this type of reliability testing program.
- Shipping Tests: Although shipping tests do not necessarily qualify as reliability tests per se, shipping tests or simulations designed to test the impact on the product of shipping and handling should be a prerequisite to reliability testing. This is because the effects of shipping will often have an impact on the reliability of the product as experienced by the customer. As such, it may be useful to incorporate shipping tests alongside the normal reliability testing. For example, it may be a good idea to put the units of a final design release reliability test through a non-destructive shipping test prior to the actual reliability testing in order to better simulate actual use conditions.
The testing that takes place after a product design has been released for production generally tends to measure the manufacturing process rather than the product, under the assumption that the released product design is final and good. However, this is not necessarily the case, as post-release design changes or feature additions are not uncommon. It is still possible to obtain useful reliability information from manufacturing testing without diluting any of the process-oriented information that these tests are designed to produce.
- Functionality Testing and Burn-In: This type of testing usually falls under the category of operation verification. In these tests, a large proportion, if not all, of the products coming off of the assembly line are put on a very short test in order to verify that they are functioning. In some situations, they may be run for a predetermined "burn-in" time in order to weed out those units that would have early failures in the field. Although it may not be possible to collect detailed reliability information from this type of testing, what is lost in quality is made up for in quantity. With the proper structuring, these tests can provide a fairly good picture of the early-life reliability behavior of the product.
- Extended Post-Production Testing: This type of testing usually gets implemented near the end or shortly after the product design is released to production. It is useful to structure these types of tests to be identical to the final reliability verification tests conducted at the end of the design phase. The purpose of these tests is to assess the effects of the production process on the reliability of the product. In many cases, the test units that undergo reliability testing prior to the onset of actual production are hand-built or carefully adjusted prior to the beginning of the reliability tests. By replicating these tests with actual production units, potential problems in the manufacturing process can be identified before many units are shipped.
- Design/Process Change Verification: This type of testing is similar to the extended post-production testing in that it should closely emulate the reliability verification testing that takes place at the end of the design phase. This type of testing should occur at regular intervals during production, or immediately following a post-release design change or a change in the manufacturing process. These changes can have a potentially large effect on the reliability of the product, and these tests should be adequate, in terms of duration and sample size, to detect such changes.
Copyright © 1992- HBM Prenscia Inc. Document updated January 2016.