Sunday, October 21, 2012

The Imaginary World of Polls




In the build up to the American Presidential election, we're hearing a lot about the impact that the debates are having in the polls, how in day to day polls, the lead that Obama has is increasing slightly or decreasing slightly, with pollsters giving the reasoning behind this based on the performance (or lack of) in the Presidential debates, or due to Romney's behind the scenes comments.

The Irish Times reported yesterday that "46% of likely voters said they would vote for Obama in the election, while 45% said they would back Romney. Yesterday, Obama was ahead by three percentage points at 46% compared to 43% for Romney."

Interesting stats right? Might we infer that Romney's strong performance in the first debate and solid performance in the second debate has seen him make up ground? Should Obama be worried?

Unfortunately for the pollsters, their little polls have a problem. They aren't accurate to infer anything of the sort.
"The precision of Reuters/Ipsos online polls is measured using a credibility interval. In this case, the poll has a credibility interval of plus or minus 3.0 percentage points for likely voters."
So any gain that we have seen from Romney could be completely (or mostly) due to the fact that the polls have a margin of error of plus or minus 3.0 percentage points. Of course that's not as interesting as declaring a tightening in the race, or assigning causation between what's happening in the race and what's happening in your polls.

While the Irish Times note the margin of error at the bottom of the article, the rest of the piece reports the figures as if they are accurate, as if the margin of error doesn't exist. Our media need to get more savvy and honest in reporting these statistics and make it clear to readers that nothing at all can be inferred from these numbers.