Get Energy Smart! NOW!

Polling seems a national mania, able to get on the front-page of the newspaper, top of the news report, top of the pile for agenda-setting for (at least some) politicians.  Doing polling right is a serious and difficult challenge. All too often, for any number of reasons from shoddiness to intentional deceit, polling is done poorly (or wrong) and reporting of the polling leads to confusion, rather than enlightenment, on the public engagement in and knowledge of issues.  The extremely insightful Devilstower purposefully created a poor polling question and, in this guest post, uses this to examine weaknesses in polling science and reporting on those polls.

Only 4 in 10 Americans believe in continental drift.  How can we know that? We asked.

QUESTION: Do you believe that America and Africa were once part of the same continent?
	YES	NO	NOT SURE
ALL	42%	26%	32%
DEM	51%	16%	33%
REP	24%	47%	29%
IND	44%	23%	33%
OTH/REF	42%	25%	33%
NON VOTERS	46%	22%	32%
Results by region
	YES	NO	NOT SURE
NORTHEAST	50%	18%	32%
SOUTH	32%	37%	31%
MIDWEST	46%	22%	32%
WEST	43%	24%	33%

The collection of all the continents into a unified land mass around 250 million years ago (along with several other instances of such mergers at more distant times) is supported by fossil evidence, by evidence of magnetic fields recorded in the rocks, and by swaths of rocks exchanged between the modern continents along their ancient borders. Plate tectonics and the resulting movement that drove the continents apart is supported by even more direct evidence — we can see it happening. Sensitive instruments are quite capable of measuring the ongoing movement of the continents around the globe. We can see that the African Plate is currently heading northeast at around 2 centimeters per year.  The North American Plate is moving southwest at around 1 cm/year (the Pacific Plate is trucking along at 8cm/yr in another direction, which explains much about why the west coast is such a geological fun zone).  In around 300 million years, the continents will hold a reunion, once again forming a single mass before the moving plates carry them away on separate paths.

So why would only 42% of Americans say that they believe in this extensively-documented, measurable, well-established theory?  Should we worry that America is hopelessly backward and mired in some kind of anti-science dark age?

No. For one very good reason: both the question itself, and the presentation of the results are deeply flawed. Intentionally idiotic.

The questions was written to press emotional hot buttons. It’s not “were all the continents once merged into Pangea?” or “Was the North American continent once closer to Europe?” or simply “do you believe in continental drift?” It’s were “America and Africa were once part of the same continent.”  If you think that doesn’t matter, I invite you to look at the regional breakdown of results — that particular association of words “America” and “Africa,” probably accounts for the 10% greater “No” vote in the same region where “Birtherism” is at its height.

The question is also framed in terms of individual “belief” in a scientific theory. This gives the impression that data are of greater worth when they’re more popular. It’s the kind of wording that not only garners bad responses, it encourages bad interpretation of the results. If 90% of Americans believe in the photoelectric effect, but only 20% believe in quantum theory, what effect does that have on electronic performance? Absolutely none. Asking the question as a belief question encourages a complete misinterpretation of what science is about. What you get when a question is asked this way confuses “is this true” with “do you like it.” In short, a belief question is totally worthless in measuring American’s knowledge on the subject or the value of the theory. It’s a measure of a theory’s popularity. And science is not a popularity contest. Worse, this kind of question intractably mingles favorability and knowledge — it’s not even a good test of popularity.

Finally, the presentation of the results is particularly egregious.  “Only 4 in 10 Americans…” may reflect the 42% who voted “yes” in the poll, but it completely ignores the fact that this is the most popular answer in the survey, and by a broad margin.  It would be a much better description of the results  to say that “far more Americans believe that Africa and America were merged in the past” than it is to misuse the raw numbers in a form that makes Americans look, well, ignorant. The results show that only a quarter of Americans don’t believe this well-supported theory, while a third of the population admits that they don’t understand well enough to have an opinion — a result that should shock no one. It is, in fact, the kind of number you should expect when asking about any scientific theory.

The poll above does reveal that Republicans are much more likely to not believe in the results predicted by continental drift. Which is interesting. But even that result is of little value when it’s attached to a question so deeply flawed.

So why do this? Why puposely ask a poorly framed question about science? Why purposely present the results in a way that makes Americans appear ignorant and ill-informed?  Because that’s exactly what the Gallup organization, and the media that reports on their results, have been doing.

Here’s a poll that Gallup ran this year on the 200th birthday of Charles Darwin.

On Darwin’s Birthday, Only 4 in 10 Believe in Evolution
Belief drops to 24% among frequent church attenders

That headline ran not only on Gallup’s site, but was repeated almost word for word by every news organization reporting on the poll. What did Gallup ask?

Do you, personally, believe in the theory of evolution, do you not believe in the theory of evolution, or don’t you have an opinion either way?

Gallup — quite intentionally — framed a scientific theory in the form of a personal belief, distorting any possible result. What kind of results did Gallup get by encouraging people to declare their fondness or opposition to a theory?

On the eve of the 200th anniversary of Charles Darwin’s birth, a new Gallup Poll shows that only 39% of Americans say they “believe in the theory of evolution,” while a quarter say they do not believe in the theory, and another 36% don’t have an opinion either way.

In other words, just as with the “continents” question we phrased this week, belief in evolution was actually the most common response once again followed by those who said they didn’t know. In fact, the relationship is best expressed again as far more Americans believe in evolution than don’t. Even that statement is of extremely dubious value, but at least it’s somewhat accurate in representing the results.

So why does Gallup (among others) insist on asking questions in a way they know will negate the value of any results, then present those results in a sensationalized way that presents Americans as ignorant and anti-science? Why present these results on evolution as something extraordinary, when asking about any scientific theory can (as we demonstrated) generate similar numbers? Because it gets headlines. Because it keeps them in the news between elections.

Or maybe I’m giving them too much credit. Maybe the Gallup organization itself is too ignorant to see the numerous flaws in what they’re asking and how they’re presenting it. It’s hard to be sure.

Maybe we should have a poll.