Polls, polls, polls

So many polls, so many numbers, they all say different things. How do we make sense of the numeric gobbledy-gook? Let’s look to the media.

Tons of websites have guides about how to read polls: FiveThirtyEight, ABC News, Mic, Vox, Washington Post (strange that no conservative outlet offers such a guide). 

There are poll aggregators (RealClearPolitics, New York Times, HuffPo, Talking Points Memo), forecasters (FiveThirtyEight, New York Times, Election Projection (finally a conservative site!)) and lots of opinions. What’s the difference between the forecasts at FiveThirtyEight versus New York Times? Osita Nwanevu at Slate breaks it down for you. Which polls are more reputable? Nate Silver ranks them and the Daily Kos tallies the scores from 2014.  It’s enough to make me hyperventilate;

I’m so confused.

Pew Research has a great article on the basics of reading political polling. I also think that Pew is a pretty neutral/unbiased source. If you disagree, let me know. Pew discusses at length the margin of error, a number that often misunderstood by media and their consumers. First, the margin of error indicates how accurately we can expect the survey to represent the true beliefs of the population. (Surveys measure a specific sample and infer the results of the larger population.) It’s important to note that the margin of error measures the accuracy of the survey to measure a single candidate’s level of support. Confidence intervals play a role. (This comes from the concept of sampling. Check out Pew’s explanation here, fourth paragraph). In order to compare the results between two candidates you need to determine the margin of error in the difference between the candidates’ levels of support.

This is where things get tricky. If the poll only has two answers, then a general rule of thumb is to simply double the margin of error. Unfortunately, most polls have more than two possible responses or, in the case of the election, measure a four-way race that includes independent candidates. When this occurs, there is a specific formula to use which you can find here. It’s complicated.

Keep in mind there are also other important considerations like sample size, sample composition, sample "representativeness" and methodology (all explained here).

Some people study this stuff for years. Confused yet? Me too. For simplicity sake, let’s look at the latest 2-way polling care of RealClearPolitics.  

Looking at the chart you will see that the difference margin of error for each poll is larger than the point spread. This means that the lead may be the result of a sampling error. In other words, it’s not a statistically reliable lead.

Wait. Most of the polls that favor Trump now, the ones that the media is falling all over themselves to report, aren’t statistically reliable?

Yes and no.

The recent spate of poll results (excerpted above) is not statistically reliable. But…(and this is the operative word here), in some ways they are. 

Nearly all poll-reading guides state that movement and trends are significant. Trump’s spread is now positive over Clinton in many polls. Will this lead will remain and/or become statistically significant? Trump’s current leads are the first step in a turning of the tides. We won’t know how significant it is, and whether it is significant until more results come in.