Reliability HotWire

Issue 95, January 2009

Hot Topics

Replicated Runs and Repeated Runs

In the context of Design of Experiments (DOE), replicated runs and repeated runs are both multiple response readings taken at the same factor levels. This article explains the difference between the two and illustrates how ReliaSoft's DOE++ software can be used to deal with the two cases.

In experiments with many factors, each combination of levels of the different factors is called a treatment. For example, imagine that 3 factors, A, B and C, are investigated in an experiment. Factor A has 3 levels, factor B has 2 levels and factor C has 3 levels. The total combinations, or treatments, for the experiment would be 3*2*3 = 18. To run the experiment completely, 18 observations would have to be taken, 1 observation corresponding to each of the 18 treatments.

While conducting an experiment, observations are recorded in a random order so that the effect of nuisance factors may get cancelled out. (Nuisance factors are those factors that are not of interest to the experimenter but may affect the response.) Thus, to run the given 3-factor experiment once, 18 observations would be recorded in a random order. At times an experiment may be run more than once to obtain an estimate of pure error. (Pure error is the error between observations taken at the same treatment.) If the experiment is run completely twice, then 36 observations would be recorded, with each treatment having 2 observations. If all 36 observations are recorded in a random order, then each treatment of the experiment is said to have 2 replicates. If the observations are recorded in such a way that the 2 readings corresponding to each of the 18 treatments are taken at the same time, then each treatment of the experiment is said to have 2 repeated runs.

In the case of repeated runs, only 1 reading is recorded at each treatment of the experiment; this is obtained by taking an average of all the repeated observations at that treatment. The reason for doing this is that since repeated runs are recorded at the same time (without considering randomization of all experimental observations) some of the variation that otherwise would have been captured (had all of the observations been randomized) is considered lost. Therefore, as far as the analysis of experiment data is concerned, observations corresponding to repeated runs are treated as if just 1 observation had been recorded for each treatment of the experiment.

The difference between replicated runs and repeated runs is further explained through the following example.

Assume that a baker wants to investigate the effects of 2 factors, A and B, on the taste of cakes made in his bakery. Each of the 2 factors has 2 levels, with the first level of the factors represented by "Low" and the second level by "High." Therefore, the total number of treatments for this experiment is 4 (2*2 = 4), as shown in Table 1.

Table 1: Treatments for an Experiment with Two Factors at Two Levels
Low Low
High Low
Low High
High High

To run the experiment once completely, the baker will have to bake 4 cakes, 1 corresponding to each of the 4 treatments. Figure 1 shows the cakes corresponding to each of the 4 treatments.

Figure 1: Cakes Corresponding to Each Treatment of Table 1

The baker decides to bake 8 cakes, 2 for each treatment. He tastes the cakes and gives them a rating on a scale from 0 to 10 with 0 indicating the worst tasting cake and 10 indicating the best tasting cake.

If the baker bakes each of the 8 cakes in a random order, as shown in Figure 2, then this represents the case of 2 replicates. The readings obtained by the baker are entered in DOE++ as shown in Figure 3.

Figure 2: Replicated Runs All 8 Cakes Are Baked in a Random Order

Figure 3: Data Entry in DOE++ for the Case of 2 Replicates

However, if the baker bakes the 2 cakes corresponding to each of the treatments together, as shown in Figure 4, then this represents a case of 2 repeats. This case is effectively the same as if the baker had baked 1 large cake for each treatment instead of 2 cakes and tasted the large cake twice to obtain the two readings.

Figure 4: Repeated Runs Cakes Corresponding to the Same Treatment Are Baked at the Same Time

In the case of repeats, the average of the observations at a treatment is used in the experiment, as shown in Figure 5. In DOE++, the average values are automatically calculated by using the option in the Design Wizard to specify that you are using repeated measurements, as shown in Figure 6.

Figure 5: Data Entry in DOE++ for the Case of Repeats

Figure 6: Specification of Repeated Runs in the Design Wizard

This article explained the difference between replicated runs and repeated runs. The choice of replicated or repeated runs can have a significant impact on the analysis of experiment data. For example, the results for the previous cases of replicated runs and repeated runs are shown in Figures 7 and 8 respectively.

Figure 7: Results for the Case of Replicated Runs

Figure 8: Results for the Case of Repeated Runs

Although the set of observations is the same in both the cases, the results of the analysis lead to entirely different conclusions. In the case of replicates, Figure 7 shows that the interaction between the two factors has a significant impact on the taste of the baked cakes. However, for the case of repeats shown in Figure 8, both of the factors and the interaction are found to be insignificant. When you are using multiple observations for all treatments of an experiment, all the runs must be randomized in order for the runs to count as replicated runs.

Copyright 2009 ReliaSoft Corporation, ALL RIGHTS RESERVED