By Scott Tilley – ASCF Senior Fellow
November 2020 – They call economics the “dismal science” because economists’ predictions are rarely correct, and the models used to make the predictions are opaque at best. It’s an inherently difficult task to make quantitative predictions based on qualitative input, and our economy is primarily driven by people who are fundamentally emotional, not logical, beings. But as bad as economics is, there’s an area of current interest that is consistently worse: polling, and there are serious national security implications to erroneous polls – particularly when it comes to presidential elections.
Polling can be considered a form of technical power because it relies on reasonably sophisticated data science to convert survey responses to election predictions. However, like all analytical techniques, polling is only as good as the data used to build the models and feed the algorithms. There’s an old computer science phrase, “garbage in, garbage out” (GIGO), and polling suffers from this ailment.
Consider the first part of the polling process: the survey (questionnaire). I learned long ago that there’s definitely an art to constructing a proper survey, and this is not a skill you typically learn in school. A survey master can tune the questions to steer the response towards whatever answer they want. This is done all the time by partisan pollsters who are looking for data to support their predetermined conclusions. Shockingly, not all pollsters are independent and objective.
Then there’s the issue of who participates in the polling process. I can honestly say that I’ve never been contacted to answer an election survey in my entire life, and I don’t personally know anyone else who ever has either. So, who is providing the survey responses? It would seem unlikely that they are representative of the general population. For example, millennials rarely answer their phones (they communicate differently), and they probably won’t spend 20-30 minutes talking to a stranger about their political beliefs and values. The result is a skewed sample.
A related issue is the size of the sample. During the recent election cycle, many media outlets repeated poll results without fully understanding the poll source. In one instance, the poll was conducted by a small community college, with a sample size of just over 800. This is generally considered to be too small to provide a reliable prediction. Still, the conclusion reached was that one candidate led the other by about 10 points, which suited the media’s narrative.
Lastly, the input is often just plain wrong because people lie to pollsters all the time. They answer a survey question one way and then do the complete opposite when they get into the voting booth. This happened during the presidential election in 2016, with Brexit in the UK, and again for the the presidential election in 2020. The commonality is that people are conditioned to avoid confrontation and being seen as “politically incorrect,” and if they say they are voting for candidate X, who the media views with disdain, then the respondent will say they are voting for candidate Y. Such people were called “Closet Leavers” during Brexit and “Shy Trumpers” in 2020.
There are other issues with garbage in, but garbage out also plays an important role. The polling predictions are based on models of the relative importance of each question in the survey, respondent demographics, and other historical data that might affect the election outcome. Each year, poll results seem to take on more importance, which is why the media constantly report the pollster’s findings as if they were guaranteed accurate. The last few years have proven to us that polls are incredibly inaccurate, but they still affect the election. In particular, they are used as a form of voter suppression.
Polls are a technical power supporting the soft power of influence. Suppose the electorate regularly hears that candidate X is 10 points ahead of candidate Y, and these figures are often repeated from different sources. In that case, the voter will come to believe the statistics are correct. If the voter supports candidate Y, they may choose not to vote because if their candidate is so far behind, and everyone says so, what’s the point? How can one vote make any difference? So, they don’t vote. Sometimes, they even leave lineups when preparing to vote if they hear dire projections of landslide victories for their opponent on the radio or social media.
When presidential elections are decided on razor-thin margins, voter suppression becomes even more important. This is why polls are so important and so untrustworthy. Even though they are rarely accurate, they sway people’s opinions. The data science fails us because it’s tough to model people, but it’s straightforward for foreign entities to muddy the waters. As Mark Twain said, “There’s lies, damn lies, and statistics.”
– END –
Photo: ILLUSTRATION BY FIVETHIRTYEIGHT