November 11, 2004
Exit Polling - David Anick
A lot of the screaming about the election is due to the gap between exit polling and the result of the election. There is talk about some mathematician named David Anick completing a study that says that the variance between the exit polling and the election result had a 1 in 50,000 chance of occurring.
First, please note: Being a mathematician does not mean you know how exit polling works.
I'm really skeptical of that study, for one reason: it uses the 4pm exit polling data.
Exit polling uses stratified samples, not random "representative" samples. The numbers are useless until you have the end-of-day turnout figures to apply to the exit polling, to balance and normalize the numbers correctly. In other words, they are supposed to be wrong in the middle of the day, and they are designed to be "corrected" (re-weighted; normalized) at the end of the day. It just doesn't make statistical sense to create a study directly comparing unweighted mid-day exit poll data to the final result.
Political campaigns and the media look at the exit polls during the day just for really inexact indications. But it's bogus to treat mid-day exit poll data with the same accuracy that we'd treat a telephone poll that uses random samples. The mid-day exit polls are less accurate, not more, because the exit poll samples are not random, evenly distributed, or representative of the voting population.
It is so fascinating to make this point over on the political discussion sites and experience the resistance to it. Huge tangent here, but it has me thinking more and more about "political oxygen", and what conditions need to exist for a population to be open to a truth that is different than their conventional wisdom. Or, how to create those conditions. We've got people painstakingly posting facts and conclusions over there, and being called Republican "plants" in response.