In This Is Spin̈al Tap, British heavy metal god Nigel Tufnel says, in reference to one of his band’s less successful creations:
“It’s such a fine line between stupid and…uh, clever.”
This is all too true when it comes to science. You can design a breathtakingly clever experiment, using state of the art methods to address a really interesting and important question. And then at the end you realize that you forgot to type one word when writing the 1,000 lines of software code that runs this whole thing, and as a result, the whole thing’s a bust.
It happens all too often. It has happened to me, let me think, three times in my scientific career and, I know of several colleagues who had similar problems and I’m currently struggling to deal with the consequences of someone else’s stupid mistake.
Here’s my cautionary tale. I once ran an experiment involving giving people a drug or placebo and when I crunched the numbers I found, or thought I’d found, a really interesting effect which was consistent with a lot of previous work giving this drug to animals. How cool is that?
So I set about writing it up and told my supervisor and all my colleagues. Awesome.
About two or three months later, for some reason I decided to reopen the data file, which was in Microsoft Excel, to look something up. I happened to notice something rather odd – one of the experimental subjects, who I remembered by name, was listed with a date-of-birth which seemed wrong: they weren’t nearly that old.
Slightly confused – but not worried yet – I looked at all the other names and dates of birth and, oh dear, they were all wrong. But why?
Then it dawned on me and now I was worried: the dates were all correct but they were lined up with the wrong names. In an instant I saw the horrible possibility: m ixed up names would be harmless in themselves but what if the group assignments (1 = drug, 0 = placebo) were lined up with the wrong results? That would render the whole analysis invalid… and oh dear. They were.
As the temperature of my blood plummeted I got up and lurched over to my filing cabinet where the raw data was stored on paper. It was deceptively easy to correct the mix-up and put the data back together. I re-ran the analysis.
No drug effect.
I checked it over and over. Everything was completely watertight – now. I went home. I didn’t eat and I didn’t sleep much. The next morning I broke the news to my supervisor. Writing that email was one of the hardest things I’ve ever done.Continue reading…







