I cannot deny that I have seen my share of inappropriate applications of design of experiments in my professional career.

One of the challenges of implementing Six Sigma, MPCpS, TQM or any improvement initiative in large organizations is that when the momentum increases rapidly, you will find that many teams will be at a point of improving or optimizing their processes at once. The stage of improvement or optimization is the stage where the processes get fixed. This is the stage with the longest cycle time since it requires finding a solution to the root-cause of the problem, and this involves planning, designing, conducting and analyzing experiments.

If you are not well prepared to provide the proper guidance or coaching when many teams and individuals are ready to design and run experiments, you may experience a high level of misapplications.

Many times team members believe they have enough knowledge to plan, design and conduct their own experiments and they may not seek guidance a priori to save time. However, once they find out that their DOE does not bring the expected results, or the error term is so large that nothing appears to be significant, they may seek help from the statistician.

If an experiment has flaws in the design or in its execution, there isn't a sophisticated analytical tool that may extract useful and conclusive information from the experimental data. The design has to be flawless and its execution often requires controlled supervision by a statistician, particularly to apply a contingent plan to salvage the design in case of an accidental miscarriage.

During the deployment of Six Sigma at Motorola, I implemented a very simple and useful remedy to ascertain that experiments were designed properly prior to being conducted. The remedy was named Request for Engineering Experiment, REEX. REEX is a form that requires answers to questions about the problem being solved, the design of the experiment, its execution, disposition of the product and appropriate approval. At the pinnacle of the Six Sigma deployment at Motorola, we were running hundreds of experiments a week.

Experiments require the allocation of product, material, equipment, gauges, machines, and processes, as well as production, engineering, and maintenance personnel, not to mention the alteration of scheduled downtime, which amounts to significant cost expenses. For this reason, experiments need to be managed and controlled to guarantee success and efficiency.

Making sure the experiments are properly designed and conducted makes a significant difference in the success of optimization and in the deployment of Six Sigma, or any other improvement initiative.

Hope this helps.

by MARIO PEREZ-WILSON

"Nested" or "crossed" refers to the relationship between factors.

Imagine that you are running an experiment with four factors: Vineyard, Grape-Type (or Must), Temperature of Fermentation, and Filtering, and that you are measuring a particular response.

For the factor Vineyard, you have chosen to compare two vineyards one in France and one in Chile.

For Grape-Type, imagine that both vineyards have Chardonnay and Sauvignon Blanc grapes planted from the same root vine, and the same amount of must is harvested and used in the experiment.

For Temperature of Fermentation you select two levels, 10 degrees Centigrade and 30 degrees Centigrade. (Let's assume the temperature can be controlled very accurately).

Finally, for Filtration, you have selected two filter types, one from a company call Pall-Bio and the other from Kieselguhr, and both types of filters are shipped to the vineyards.

Let's examine the relationship between the factors in this experiment.

The relationship between Vineyard and Grape-Type is NESTED.
In this example, it should be obvious that the factor "Grape-Type" is nested or uniquely contained within the levels of Vineyard. At each level of Vineyard, we have Chardonnay and Sauvignon Blanc. The Chardonnay in France, will be different than the Chardonnay in Chile, and the Sauvignon Blanc in Chile, will be different from the one in France. Even though they came from the same original root vine, they will be different due to other influential conditions. The name of the levels may be the same, but the grapes (Chardonnay and Sauvignon Blanc) are different at each level of the factor Vineyard. So, the factor Grape-Type is nested within the factor Vineyard. Nested, as the word implies, means contained within.

The relationship between Grape-Type and Fermentation Temperature is CROSSED.
Here, both Chardonnay and Sauvignon Blanc will be fermented at two distinct temperatures, 10 and 30 degrees Centigrade. 10 degrees Centigrade, will be the same in France as it is in Chile. The same goes for 30 degrees Centigrade. So here the relationship between Grape-Type and Fermentation Temperature is that these two factors are crossed. In other words, the levels of Temperature of Fermentation are the same at each level of Grape-Type.

The relationship between Fermentation Temperature and Filtering is CROSSED.
Again, the filtering is done at two levels using the filters from Pall-Bio and the filters from Kieselguhr. Both filters were shipped to each Vineyard, so these two levels are the same. So, in this case, Temperature and Filtering are also crossed factors.

Now, let's say that we decided to replicate the experiment. Replication is always nested.

Hope this is helpful.

by MARIO PEREZ-WILSON

I worked for Motorola, Inc. from 1984 to 1991. My direct responsibility, as head of the department of statistical methods, was to implement and disseminate the use of statistical methods to achieve and sustain Motorola's corporate quality "Five Year Goal" which was: "Achieve Six Sigma Capability by 1992 - in Everything We Do".

When the document "Our Six Sigma Challenge" was distributed on January 15, 1987, it made reference to the plus or minus 1.5-sigma shift, and to the 3.4-ppm defect level. When I sought more factual details to support these statements, the facts were always "anecdotal". There was no data, no hard analysis, no conclusive evidence, and no statistical validation to these assertions.

As the inquiries grew, so did the doubt about the quality goal. Later in 1988, Mikel Harry and Riegle Stewart came to the rescue to add credibility to the statements by publishing an internal document "Six Sigma Mechanical Design Tolerancing". This document again presented no data to support any validation of the 1.5 sigma shift, however, it makes reference to articles written by David H. Evans and A. Bender.

If you follow the trail by reading the articles:

The Cpk formula has two values that vary from sample to sample. These are the mean and the sigma, where the sigma is estimated by the sample standard deviation. The mean also varies, but to a larger degree the sigma may be a more important statistic.

In the Y-Axis, we have sigma (the parameter or population standard deviation) divided by the sample standard deviation. When they are equal, we have 1.0. The graph shows a horizontal line at 1.0.

In the graph, we can see that the confidence limits converge to 1.0 as the sample size increases. This implies that as the sample sizes become larger, the sample standard deviations are better estimates of the sigma (population standard deviation).

Using a confidence limit (CL) at 5% and n=31, from the graph we get 0.785, and at CL=95% and n=31, we get 1.208. This implies that 90 percent of the time, sigma (the parameter being estimated) will be contained by this interval [0.785-1.208]. Let's say you have a characteristic measured in inches and the value of the sample standard deviation was 2.85 inches, then the confidence interval 2.24 (2.85 x 0.785) and 3.44 (2.85 x 1.208) will contain the sigma of the characteristic. In other words, at the 90 percent confidence level, the standard deviation is good to within -21.5% and +20.8%.

Now, at CL=5% and n=101, from the graph we get 0.883, and at CL=95% and n=101, we get 1.115. With a sample size of 101, and with a 90 percent degree of confidence, the standard deviation is good to within -11.7% and +11.5%. By increasing the sample size, the length of the confidence interval is much shorter; it went from 42.3 to 23.2. That is a 45.15% reduction.

In short, when it's economically feasible, you should try to increase the sample size to at least 100 observations. 200 would be even better. The graph in figure 1.0 can help you determine the appropriate sample size and its confidence interval.

by MARIO PEREZ-WILSON