Political Polling Trivia Quiz

Answer: Low voter turnout

When polling firms conduct polls they typically use two voter screens: registered voters (RV) or likely voters (LV). The RV screen polls all registered voters in the state of Texas while the LV screen typically weeds out people the firm considers unlikely to vote.

The group of people who are registered voters is larger (sometimes even much larger) than the pool of likely voters so if a candidate is doing better with likely voters than with registered voters, that candidate benefits from lower voter turnout.

When turnout is low that means only the most frequent voters show up to cast a ballot. When turnout is high that means more voters who typically do not show up are voting so any candidate who does better with registered voters benefits from higher turnout.

Answer: A poll showing Don Key up 49%-46% taken over the course of a month

A good quality poll is taken over the course of two to four days. A poll that takes over a month is likely to be inaccurate because a month is a long time during the course of a campaign where many different potentially race-changing events may have occurred. Two to four days is roughly the duration of a standard news cycle and is enough time to reach a variety of people. Polling firms would ideally use a mix of landlines and cell phones to contact voters. Using just one of these means certain groups are likely to be severely undersampled.

A high-quality poll also would have a minimum of 300 respondents as any fewer than that is too little and could cause unusual crosstab results even more so than normal.

Answer: False

This is an extremely common mistake made by members of the media, even long-time political and polling experts. The margin of error does not apply to the margin between the candidates. It applies to each candidate's individual number. What this poll actually says is Ella Phant's level of support ranges from 46.2%-53.8% and Don Key's level of support is actually between 42.2% and 49.8%. Because Ella Phant's level of support at its lowest number is not greater than Don Key's level of support at its highest, the results of this poll are not outside the margin of error.

Another common mistake made regarding margins of error is to call any poll inside the margin of error a "statistical tie". Only a result showing a tie (such as 49%-49%) should be considered a tie because one candidate ahead would have an advantage, otherwise.

The phrase "statistical tie" is commonly used in the news media to create drama and suspense.

Answer: A FunPolling Firm poll that shows Don Key leading 51%-46%

When examining swings in polling, it is important to compare polls from the same pollster only. Different polling organizations use different methodologies so it is not helpful to compare polls from different pollsters for the purposes of determining a swing.

It is more useful to compare polls from the same pollster because that would show in which direction things are moving because the pollster uses the same methodology from poll to poll. This consistency means it is much easier to see any changes and amongst which groups are changing.

Answer: Ask issue questions before horse-race questions

The proper way to ask questions when conducting a poll is to ask demographic questions about the respondent first to judge whether or not that person is an eligible voter and belongs to a demographic whose quota for the poll has not yet been met. Then the pollster should ask the horse-race questions (that is asking the respondent which candidate they will vote for from a list) before asking other questions about a voter's opinion on issues. When issue questions are asked before the horse-race questions, it can sometimes influence which candidates the respondent might select as the person is then thinking about the issues. Generally, a good rule of thumb is to ask the questions in the order they would appear on the ballot. Other questions asked by the pollster may also prime the respondent in a certain direction. Sometimes polls are sponsored by a campaign who may be testing certain messages. This information could push a respondent in a direction he or she would not normally otherwise go.

Well-respected polling firms are transparent. They publish all of the information they collected, especially crosstabs that break down all kinds of demographic groups such as race, gender, age, political affiliation etc. This is necessary so a firm cannot skew results by oversampling or undersampling certain groups or by improperly weighing them. Polling firms should also publish outliers, polls with results that do not fall in line with other publicly available data. Outlier results are perfectly normal and may signify a change in trajectory. It is not transparent to hide a result that seems odd to avoid appearing wrong. It is perfectly ethical for a polling firm to take on political candidates so as long as the firm discloses who its candidate is and does not skew the results to show their client in better position. In fact, a polling firm needs to be honest with its clients so the client knows exactly what position they are in and which kinds of voters to target.

Answer: "Ella Phant is down slightly in a traditionally Republican state but she should win it because the undecideds are overwhelmingly Republican."

There are a lot of common polling myths out there, even in political media and political pundits who often repeat these myths. One of the most popular myths is that if an incumbent is polling under 50%, they are in danger of losing. This is false. While that incumbent may very well end up losing, a poll showing the incumbent under 50% but leading has better odds of winning than losing. It is not true that undecideds always break against the incumbent. They may do so, however. By examining the crosstabs of the undecided voters, one can reasonably assume which way they will break. If the undecideds are normally partisans, they will overwhelming break toward the candidate of their preferred party. This is why Ella Phant is likely to win the traditional Republican state she was down slightly in. In that situation, the undecides are overwhelmingly Republican so they are much more likely to break toward her because of the very common and proven pattern that partisans "come home" by Election Day. This is also sometimes referred to as a "come to Jesus moment" because those undecideds may have been considering voting for the other party, they almost always do not. If anything, the polling shows an enthusiasm issue for Ella Phant which may be a problem in other states but not one strongly leaning toward her party.

Another common myth is called crosstab diving, that is closely examining the crosstabs to look for errors in an attempt to dismiss a poll. Demographic subgroups have very high margins of error because the samples are very small and that does sometimes cause unusual results. In the example, the traditionally Democratic voting group may have been undersampled but the polling firm should have weighed them properly into the overall results. While weighing is tricky and prone to error, one cannot simply skew the results by adding points to candidates. First of all, skewing the results based on crosstab analysis is not proper because you're essentially reweighting already weighted data and the number of points you're adding is arbitrary and unscientific. Moreover, unusual crosstabs tend to cancel each other out so the same poll would likely show Don Key doing better with traditional Republican groups than normal. Related is a myth about polling error. Polling errors are common but not consistent. Polling errors do not carry over from election cycle to election cycle as pollsters change their methods to correct past mistakes. Just because Republicans were underestimated in the last election, does not mean it will happen again.

Answer: Put it in a polling average

A polling aggregate, that is an average of different polls, is the best response. Because different pollsters have different methodologies, a polling average is a good way to asses the state of play. One individual poll should not change the way candidates and voters look at a race. It is much more informative to look at a series of polls from different sources of the same electorate (whether state or national) than to look at one individual poll and putting more stock in that than the average.

When creating a polling average, it is important to remember a few guidelines. The most important thing to remember is to use only polls from the same electorate. Do not combine national polls with state polls and do not combine state polls of different states. When a new poll comes out, you should remove any other polls from the same pollster and you should not include polls that are over a month old as the race dynamics have likely changed in a month's time.

Answer: Recalled vote from the last election

While it may seem strange, asking respondents who they voted for in the previous election is actually not that helpful for a variety of reasons. Some voters forget who they voted for, lie about who they voted for, prefer not to answer or didn't vote in the previous election.

Moreover, it does not take into account voter turnout. If a state voted for Don Key in the previous election by a margin of 55%-45%, that does not mean the electorate that shows up in the hypothetical current election would have voted for Don Key 55%-45% the previous time.

The recalled vote is not useless and a pollster would have interest in asking the recalled vote to make sure their sample is accurate but it is not very useful compared to race, gender and age which are far more important, predictive and consist from election to election.

Answer: He will underperform his polling numbers

Third-party and independent candidates tend to poll much better than they perform when the results come in. This is especially true in the United States where voters commonly say they are dissatisfied with the two-party system and want other options.

There are many theories as to why third-party candidates underperform their polls consistently. Most common is that American voters know there is a two-party system and consider votes for any candidate but a Republican or Democrat to be wasted. This is especially true for the president where the Electoral College is at play and most states do not have instant or regular runoffs. Typically speaking, third party candidates take more votes from the major party whose platform is more similar to the third-party candidate's or from the party whose voters are more unsatisfied with their candidate. So it all depends on the candidate which party they tend to take more votes from.

This is true at all levels of elections, even in races where an independent or third-party candidate is a de-facto major party candidate if one of the two main parties does not run a candidate in a race.

Answer: Don Key's internal poll that showed him up 11%

Because of the way polls are conducted, it is perfectly normal and natural for an "accurate" poll to be off a few points in either direction. In this example, the actual results were Don Key by 1% so it would make sense to have polls showing a range of anywhere from (approximately) Key up 4% to Phant up 2%.

However, Don Key's internal poll that showed him up 11% would not be considered accurate or within normal range of polling margin. It is off 10% from the actual results. It does not matter that that poll showed the correct winner and Ella Phant's internal poll did not. Phant's internal poll is significantly closer to the actual election results and within the normal range of outcomes.