Experimental Strategy: A Blueprint. Designs

By | November 18, 2014

Step 5. What do we do in the laboratory?

“You can’t fix by analysis what you bungled by design.” – By Design (Light, Singer and Willett)

This is the part of the experimental strategy process where most Design of Experiments (DOE) classes start. The student is taught about Analysis of Variance (ANOVA), factorial designs, and other statistical techniques. All important, but often highly confusing and off-putting.

If we boil it down to the real principles

  • We think of experiments as arrays of runs rather than one run at a time.
  • Each phase of the experiment (screening, followup, response surface, optimization, robustness…) is associated with a specific type of array.
  • The underlying structure of these arrays is geometric. They can be thought of as squares or cubes (or sometimes triangles and tetrahedra).


A judicious man uses statistics, not to get knowledge, but to save himself from having ignorance foisted upon him. –Thomas Carlyle

All we want to do at this stage is find out if a given factor has any effect, and, if not, eliminate it from further consideration. To do this we should only need two runs for each factor – at high and low levels (if quantitative), present or absent, or item A vs. Item B (if qualitative). So for three factors we should only need six runs.

FactorialSo what are we missing? One key item is replication. Every conclusion we are making depends on every single observation being correct! A second is  “apples to apples” comparisons. These pairs of runs are independent of each other – so it becomes difficult to see if the effect of one factor is greater than the other. These objections are easily met in a simple geometric experiment – the 8 run cubical array. This design actually contains the three 2-run experiments mentioned above. BUT – it has the bonus that each 2-way comparison is made four times, and all the factors are examined in a single setting. As the number of factors increases, the dimension of the array also increases – but you do not have to use anything more complicated than a cube to think about the problem. Tesseracts are unnecessary!


Response Surfaces

Raffiniert ist der Herrgott, aber boshaft ist er nicht. (God is subtle, but He is not malicious”) – Einstein

The underlying mental model of screening designs is the straight line. It’s useful for eliminating the unimportant, but far from enough to get us to the quality of products and processes, we want and need. To do this, we need to be able to think of both interactions and curvature. Curvature is straightforward; a property can increase and level off; decrease and level off, or go through a max or minimum.

In experimental design, an interaction is a change in the effect of one factor on a response that is caused by a change in a second factor. This concept is not self-explanatory and is often confused or overlooked.

NoInteraction Untitled
No Interaction Strong interaction

Interactions can usually be found with a well-planned two-level design. Determination of curvature requires that there be three (or more) levels for each factor, which can rapidly ascend the heights of complexity, as well as the required number of experimental runs. Computer-aided design rapidly becomes necessary.

The examples given here are only the tip of the iceberg of an immense body of knowledge. There are plenty of excellent texts available. I use Statistics for Experimenters (Box, Hunter and Hunter) and Design and Analysis of Experiments (Montgomery). Powerful computer programs that can guide you to good designs and perform the hard calculations are available. I routinely use JMP, Design-Expert, and Minitab.

You now have a DOE strategy that

  1. Has an appropriate array of points for its position in the experimental sequence
  2. Has the statistical power to give a “clean” answer.

In the next section, we’ll get into the lab (almost).

If you want to jump right to the whole strategy, contact me at +1 413 822 5006 or cawse@cawseandeffect.com!

Leave a Reply

Your email address will not be published. Required fields are marked *