The perils of crystal ball polling

Posted in: Market Research on April 18, 2019

By Heidi Dietzsch

In any election year the media are flooded by election poll findings. These polls certainly make for good news stories, but you have to wonder how accurate they are and how much faith we should put in political polls that attempt to predict the outcome of an election.

Peter Attard Montalto, Intellidex’s head of capital markets research, believes that good polls, with a proper understanding of their construction and limitations, are a useful tool for everyone, from voters to analysts and investors. “The outright rejection of polls as somehow invalid given they are not the actual vote, is bizarre and anti-scientism,” says Attard Montalto.

Political polling is a type of public opinion polling and, when done right, is an accurate social science with strict rules regarding sample size, random selection of respondents and margins of error. However, even the best public opinion poll is only a snapshot of public views at the particular moment in time, not an eternal truth. Voter opinions shift dramatically from week to week, even day to day, as political parties battle it out on the campaign field.[1]

Although election polls attract a great deal of attention for their ability to predict the outcome of elections, their most important function is to help journalists and citizens understand the meaning of the campaign and the election. Polls help to explain, among other things, what issues are important, how candidates’ qualities may affect voters’ decisions and how much support there is for particular policy changes.[2]

Attard Montalto reckons that there are two problems with South African election polls. First, polls are often poorly constructed, without appropriate weightings and sample sizes and with many methodological errors. Indeed, some are not even opinion polls in the proper sense, especially those derived from social media or ones that fail to understand the difference between registered and non-registered respondents. The second problem is that the South African media fail to report polls properly, seeing them as forecasts rather than snapshots in time, and not clearly explaining methodological issues.

So, just weeks before the election, what are the South African polls telling us about how South African are feeling?

The Association for Free Research and International Cooperation (Afric) conducted two polls of more than 3,500 South Africans in October 2018 and February 2019. Almost 60% of respondents painted the ANC in a favourable light, citing new initiatives to support businesses, anti-corruption policies, new taxes to improve the economy and other policies targeting social ills such as crime as reasons for their positive attitude towards the ruling party. Fifty-eight percent said they would vote for the ANC, 17% for the EFF and 10% for the DA.[3]

In a separate poll by global market research firm Ipsos, 61% of respondents said they would vote for the ANC. The governing party is followed distantly by the DA (14%), EFF (9%) and IFP (2%). Ipsos conducted face-to-face interviews with 3,571 adults between 23 October and 14 December last year. [4]

The latest poll by the Institute for Race Relations (IRR) suggests that in the national picture, the ANC is sitting on 55%, the DA on 22% and the EFF on 12%. When considering the provincial picture, the results are quite startling. In Gauteng the ANC is well below the 50% mark, hovering on 42%. The DA is on 32% and the EFF is on 18%. The key aspect here may not be so much that the ANC is below 50%, but so far below 50%. It seems quite possible, based on the IRR’s poll, that the ANC could lose the province, which could have complex and important ramifications for the party.

In the Western Cape, the DA would be in power with 50%, while the ANC is on 34%. The EFF is almost nowhere in this province; it is behind the ACDP, UDM and Patricia de Lille’s GOOD Movement.[5]

Attard Montalto believes that Ipsos polls just before an election are generally quite accurate given their large sample sizes and good fieldwork, especially in rural areas and townships. However, their polls have shown methodological problems in earlier campaigns, and last-minute polling is not particularly useful for financial markets specifically. He says the IRR polls are more interesting since they focus on trends and shifts that occur during campaigns. But, he warns, the IRR polls do not have a long enough history to be meaningfully assessed over time.

Despite good intentions and sound methodology, polls often fail dismally to accurately predict outcomes. For instance, what went wrong for pollsters during the 2016 US election and the British Brexit referendum?

One factor could be the gap between rational behaviour and emotional behaviour. How people feel in the privacy of a voting booth might differ from how they will react to a polling question when they feel that a justifiable, sensible and politically correct answer is required.

There is also the phenomenon of the “shy voter”. Through social media, people have learnt that serious repercussions can follow when they express opinions contrary to the mainstream discourse. Therefore, voters may tell pollsters that they are undecided or will vote for the socially acceptable option, while planning to vote for a more controversial candidate on election day.

Political polls are part and parcel of election times and these periods will probably be significantly less exciting without them. Polls that don’t apply appropriate methodology should be viewed sceptically, while those that do can certainly be useful.

What’s important, though, is that there is no such thing as a political crystal ball and pollsters are not clairvoyants. There are many variables and every election presents fresh challenges with unique parameters and issues.

One thing is clear – when the pollsters do get it right, they are lauded almost as much as the winning candidate or party. But when they are wrong, their critics have a field day.

[1] https://people.howstuffworks.com/political-polling.htm

[2] https://www.pewresearch.org/methods/u-s-survey-research/election-polling/

[3] https://businesstech.co.za/news/government/300728/what-the-2019-election-results-would-look-like-if-south-africans-voted-tomorrow/

[4] https://www.iol.co.za/capeargus/news/polls-show-anc-dominating-2019-elections-18732126

[5] https://www.dailymaverick.co.za/article/2019-03-07-new-2019-poll-predicts-a-massive-battleground-gauteng/

RELATED ARTICLES

STUART THEOBALD: If only we could shrug off labels and base policy on evidence

November 11, 2019

The strands of ideology that shape SA’s political economy are deep. Sometimes we are not even conscious of them. One of the most important is Marx’s theory of history, a vision of the world that... read more

PETER ATTARD MONTALTO: Action, reform, decisiveness needed, but pragmatism in short supply

November 4, 2019

Moody’s has been very clear that it is up to government now to consolidate expenditure and undertake growth boosting reform before February or … or […] read more

STUART THEOBALD: Investors’ eyes are on Ramaphosa over needed reforms

October 29, 2019

What is needed instead is evidence of genuine structural reform; the kind that will make analysts return to their models, change parameters, and deliver to their clients forecasts of better economic... read more