Experimental Strategy: a Blueprint. Step 2: Measurements

By | October 9, 2014

Step 2. What are we measuring? Are we measuring it well?

Not everything that can be counted counts and not everything that counts can be counted. (Albert Einstein)

In the previous step, we discussed ways to move from the goals and objectives of the business to unbiased, specific and measurable laboratory objectives. Now we need to make key decisions on what to measure at each stage of the experimental project so that each data point will advance the laboratory and business objectives. For that to happen each data point must be accurately measured – so your measuring tools must be precise and accurate!

Measurement Flowdown

The easiest way to assure this is to use the Flowdown tool discussed in Step 1. By adding a “Measurements” column to the right, you can assure that each of your flowdown goals is measured appropriately.

The peril is that of measuring something that is easy to measure, but not important to the customer. Remember the drunk searching for his keys under the lamppost – because the light is better there?

At each step in the chart, we must ask what is being measured – and does this measurement relate to the previous steps?


  • General public
  • Truckers
  • Oil Companies
  • Refineries
  • Marketing
  • Product R&D

  • Lower Pollution
  • Lower cost
  • High profit
  • Easy process
  • Convincing case
  • Usable new technology

  • SO2 levels
  • Diesel prices
  • $/barrel
  • Capital costs
  • Pilot throughput
  • Lab %S removal

Measurement Strategy

You can increase your chance of picking the right measurements by using the following strategy.

 1. Are your measurements really giving you insight into what the process is doing?

2. Do they capture as much information as possible from the experimental unit? For instance, if the response is thickness of a plastic part, is it measured at only one point or averaged over several points?

3. What kind of measurement are you making? Is it:

  • Binary – an on-off, yes-no measurement. It broke or it didn’t.
  • Subjective – a one-to-five estimate of a quality like a popcorn taste test.
  • Integer – counted defects like the number of delaminations from glue failures in a sample of plywood.
  • Real numbers – actually measured with an instrument, like percent sulfur in an oil sample?

Each of these types of measurement requires a different experimental plan and data analysis!

4. How do you take your measurements? What are the reproducibility, precision, and accuracy of the measurements?

 Many experimenters do not know the state of control, precision, or bias of their measurement systems. There is an entire discipline often called “Gauge Repeatability and Reproducibility” (GR&R) or “Measurement Systems Analysis” for bringing statistical control to measurement systems.

 5. Do you have baseline experience – the operating results at current factor settings? These can serve as a reference, which allow you to judge the practical magnitude of the effects observed in the experiment.

 For example: From the flowdown above, let’s say the current sulfur in diesel oil varies from 2% to 3%. If your goal is <1%,  you will need measurement precision and accuracy much below 1% before you can be sure you have a real effect – or one that anyone will care about.

 Remember: Information… is a difference which makes a difference. (Gregory Bateson)

 You now have an experimental plan that

  1. is relevant to the organization’s goals and objectives, and
  2. captures the right measurements

 In the next section, we’ll look at the independent variables (factors).

If you want to jump right to the whole strategy, contact me at +1 413 822 5006 or cawse@cawseandeffect.com!



Leave a Reply

Your email address will not be published. Required fields are marked *