Reliability HotWire: eMagazine for the Reliability Professional
Reliability HotWire

Issue 11, January 2002

Hot Topics

Customer Usage Profiling

An important requirement for designing useful reliability tests is to have a good idea of how the product is actually going to be used in the field. The tests should be based on a realistic expectation of the customer usage, rather than estimates or "gut feelings" about the way the customer will use the product. Tests based on mere speculation may result in a product that has not been rigorously tested and consequently may run into operational difficulties due to use stress levels being higher than anticipated. On the other hand, tests that are designed with a strong basis of information on how the product will be used will be more realistic and result in an optimized design that will exhibit fewer failures in the field.

Customer usage profiles can be designed to actively gather information on how the customers are actually using a product. This design can range from a simple questionnaire to sophisticated instrumentation within the product that feeds back detailed information about its operation. An incentive is often useful to get customers to sign on for a usage measurement program, particularly if it is an intrusive process that involves the installation of data collection equipment. Additionally, customers are often eager to participate in these programs in the knowledge that the information that they provide will ultimately result in a more reliable and user-friendly product.

Customer Surveys
One of the most straightforward ways of collecting meaningful customer usage information is to simply ask the customer. Simple customer surveys can provide a wealth of information that can be of real use in dealing with engineering and test design problems. A simple questionnaire can often be sufficient in providing useful information. This information can be used for anecdotal engineering problems and statistical test design.

Customer surveys can range from simple questionnaires to detailed surveys. In most cases, organizations sponsoring a questionnaire or survey can expect a 25% return rate for their questions. A questionnaire can consist of a few simple questions about how and when a product is used. More detailed surveys can provide more complex information that can be correlated and analyzed to produce more detailed results. One major automotive manufacturer sponsors a detailed annual survey similar to the J.D. Power and Associates vehicle dependability survey. The customers are asked detailed questions about specific problems encountered with their vehicles as well as questions about their intentions of purchasing another vehicle from the manufacturer. This information can then be correlated and used to calculate a cost in lost sales due to specific part unreliability. For example, a dollar value can be attached to every drop in reliability percentage for the vehicle transmission. For obvious reasons, the methodology involved in performing these correlations is a closely-held corporate secret.

Often, this customer usage information can be useful in dispelling anecdotal misconceptions about field use that may result in poor product or test designs. For example, a printer manufacturer discovered that a new printer design was having difficulty making transparencies shortly after the printer was turned on. An element of the design engineering team was willing to write this difficulty off, saying "Oh well, nobody is going to make transparencies first thing in the morning, anyway." However, reliability engineers involved with the product were able to produce customer usage information illustrating that a sizable percentage of users were producing transparencies first thing in the morning and right after lunchtime, times when the printer may not be sufficiently warmed up. As a result of this information, the design team was compelled to concentrate more effort on the problem, avoiding a potentially large and costly design flaw.

Simple customer usage information can be used to design tests as well. Information on usage or environment can be analyzed statistically and used to design reliability and validation tests. Frequently, it is desired to design a test to a high-stress or high-percentile-use customer. This may be best illustrated in an example.

A company manufactures a product whose performance is strongly affected by the relative humidity in which it operates. It wishes to design its product to be robust enough to withstand use in a 95th percentile humid environment. It devises and sends out a simple questionnaire to a representative cross-section of its current customers, asking the customers to provide information about the environmental conditions under which the product is used. When the information has been collected from 100 customers (out of the 400 questionnaires that were distributed), the following information on relative humidity was compiled.

Use Environment Relative Humidity  Number of Customers
5% 0
10% 1
15% 1
20% 1
25% 3
30% 2
35% 3
40% 10
45% 13
50% 15
55% 22
60% 10
65% 8
70% 4
75% 3
80% 2
85% 1
90% 1
95% 0

This information can then be analyzed in a manner similar to regular life data in order to determine the distribution of usage humidity levels. In this case, a Weibull distribution was used to characterize the humidity distribution, with a calculated beta (β) value of 3.35 and eta (η) value of 55.5. The following graphic shows a probability plot of the data.

Probability plot of relative humidity usage environment.

The exact value for the 95th percentile humidity level can be calculated using the QCP tool in the Weibull++ software. The 95th percentile humidity level corresponds to the 95% unreliability for an analysis of regular life data. Consequently, a calculation for 5% reliability will return the corresponding 95th percentile relative humidity level.

Weibull++ QCP with 95th percentile relative humidity level (i.e. 5% reliability result)

This shows that the 95th percentile relative humidity level is 77%. Note that for this analysis, the "Time" label in the QCP actually refers to the relative humidity, as this was the variable being analyzed, and not failure/suspension time as in a typical life data analysis.

Direct Measurement Programs
Often, an organization may be interested in more information or types of information than can be supplied through the use of a questionnaire or survey. In these situations, a direct measurement program may be instituted, utilizing equipment that can directly measure how a product is being used. While this is often much more expensive than a survey, it can provide much more detailed information and eliminate any of the potential errors inherent in a written survey.

A large part of the expense in a direct measurement program involves purchasing and installing the measurement equipment. Sophisticated multi-channel recorders can be hooked up to measure and record a number of different inputs of interest. This information can be stored on a computer disk for manual retrieval or connected to a modem/cell phone apparatus capable of automatically dialing up and transmitting data on a regular basis. Obviously, this approach is more intrusive and will result in more inconvenience to the customer than a survey. In these situations, incentives may be provided to customers who participate in the measurement program, in order to compensate them for the inconvenience.

Direct measurement programs are frequently used by organizations that produce complex products that have a sizable number of inputs or use modes, such as the automotive or computer industries. As with simple survey data, the results of measurement program data analysis can be used anecdotally or statistically to help design products and tests.

In an example similar to that of the computer printers, engineers for a major automotive manufacturer used to design durability tests for their four-wheel-drive vehicles under the assumption that the vehicles would never be driven more than 55 miles an hour in 4WD mode. They had no scientific or statistical basis for this assumption; it just sounded logical to them. However, as warranty claims for four-wheel-drive vehicles mounted, the company began to measure how these vehicles were being used. To their surprise, they found that customers were exceeding 55 miles per hour driving in 4WD mode quite frequently, with some speeds topping 80 miles per hour. Consequently, the tests were redesigned, problems with new vehicle designs spotted more quickly, and warranty costs were reduced.

The data from direct measurement programs may be analyzed in a variety of ways, depending on how the information is to be used. In many instances the data can be analyzed in a fashion similar to that of the example given for the analysis of data from customer surveys. However, there tend to be more anomalies in data collected via a direct measurement program, such as data sets with a number of zero-value data points. (For an example of how the Weibull++ software handles this type of data, see this month's Tool Tips.) 

Whether obtained from customer questionnaires, direct measurement programs or another method suitable to a particular product and customer base, the acquisition and analysis of realistic information about customer usage can significantly improve product design and reliability test design activities.

ReliaSoft Corporation

Copyright © 2002 ReliaSoft Corporation, ALL RIGHTS RESERVED