Earlier one of our writers stumbled on a report that claimed gas-guzzling Hummers were better for the environment than hybrids like the Toyota Prius. This is one of those great stories that everyone loves - where the conventional wisdom is wrong, and we can all have a good laugh knocking someone or something off it's high horse. This story has been passed furiously around the Internet for a week or so, by email and blog, featured on Digg and Slashdot. It's a good anecdote about unintended consequences and a little boost to Hummer owners who are sometimes criticized for their very conspicuous consumption. It's also pretty much a load of crap. But how can that be? The writer did a bunch of research, and came up with numbers and formulas. Lots of people saw it and voted with a thumbs-up in Reddit or StumbleUpon. Welcome, dear readers, to the world of white papers and press releases. Let's say you had a conclusion you wanted to support, or clients you wanted to flatter. You do a bunch of research, finding information that backs your conclusions. Now what to do with it? You can try presenting it at a conference or submitting it to an academic journal, but then you run a risk. The risk is that peer review will knock it down. The scientific method has a key difference from the method mentioned above - instead of creating a conclusion then finding evidence, you create a hypothesis, gather all the evidence, then form your conclusion. Take this pesky detail and add a dash of scrutiny by experts in the field and you have a pretty good recipe for coming up with useful theories and knowledge. The recipe just won't make the muffins come out exactly the way you want them every time. So what do you do with this research? Put it in a white paper and/or type up and good press release. The term "white paper" used to refer to government policy documents, but now it's often used to mean a report by a company or individual in an industry intended to inform and persuade customers and partners. There's absolutely nothing wrong with this, so long as everyone knows that the purpose of the document is often to persuade or sell something, not to impartially report on all the facts. IT workers have become very familiar with the uses of white papers since we are constantly bombarded with them. They are notoriously available to prove nearly any point you want to make. Is Oracle the fastest database system? You can probably find 20 white papers that say so authoritatively and conclusively. Is Oracle bloated and inefficient? Look, there's 20 papers that say so authoritatively and conclusively. They aren't all worthless, because each one might give you some good factual information. It's up to you to find and use what you need without drinking the Kool-Aide. The use of email by lots of people outside the IT realm and recently the hug number of non-IT bloggers has given press releases and white papers some new possibilities. They can be spread around the world in record time and quoted by a 1,000 blog posts as if they were primary sources. Notice that this report on the Prius and Hummer is 450+ pages. How many people, do you think, read all 450 pages? There are a number of problems with this report that make the conclusion that Hummers are green and Priuses are baby- seal-clubbing smog machines hard to swallow. For one thing, it assumes that a Prius will only last 109,000 miles, which is lower than some parts of the warranty in California and similar states, while extending the life of a Hummer H1 to 379,000 miles. Plenty of Priuses have already passed 200,000 miles, often working in taxi fleets. It also mentions the use of Nickel in the batteries and the damage Nickel mining did to Sudbury, Ontario. The batteries are warrantied to 100,000 miles Nickel is recyclable. Sudbury has done a great deal to mitigate past environmental damage and is no longer a wasteland. As the TrueDelta blog points out, the cost of ownership numbers are amazingly high for all the vehicles in the report. If Priuses cost this much to build and operate, we have to assume that Toyota is taking a huge loss on every one sold, and in fact the entire auto industry is grievously undercharging us. There are a number of other problems with the report, but others have already done a pretty good job outlining them. Probably the biggest problem is that the source data is not available for review, since it is considered valuable intellectual property by CNW research. So what's the answer to the question posed in the title? How can a Hummer be better for the environment than a Prius? By using whatever methodology you want, using whatever data you want, and closing your research up from peer review, that's how. And what's the lesson for today? It's certainly not "don't believe everything that you read," because that is glib and cynical without being precise enough to be useful. Here are some thoughts that might be a bit more practical:
- White papers and press releases are fine, but keep in mind they are often intended persuade you or sell you something.
- Starting with a conclusion makes research easier, but doesn't validate your conclusion.
- Digg, email forwards, and 1,000 blogs do not count as peer review.