The lady tasting tea

In the 1920s, at an agricultural research station in England, a scientist named Muriel Bristol said she could taste whether milk had been poured into a cup before or after the tea. The statistician Ronald Fisher designed a test, and later used it to open his 1935 book The Design of Experiments.

Fisher's design

1. Prepare 8 cups: 4 milk-first, 4 tea-first.
2. Present them in a random order.
3. Tell her there are four of each, and ask her to pick the four milk-first cups.

Why this design is clever

• A comparison: both kinds of cup are tested, not just one.
• Randomisation: the order can't give her clues, and the experimenter can't (even accidentally) arrange easy patterns.
• A known chance level: there are 70 ways to choose 4 cups from 8. If she were only guessing, she'd get all four right just 1 time in 70, about 1.4%.

So if she gets them all right, guessing becomes hard to believe. Fisher worked out before the test what result would count as evidence. That is the heart of experimental design.

The lesson for you

Before running any experiment, ask:

• What exactly am I comparing?
• How is chance ruled out?
• What result, decided in advance, would convince me?

RememberDesign the test before you run it: a fair comparison, a random order, and a known chance level.