As tensions rise in US elections, leading pollsters feud over methods
The leaders of two of America’s major polling services are at loggerheads over whether FiveThirtyEight.com’s predictions about Donald Trump becoming president are reliable.
The 2016 elections have seen more polls than any previous event, many of them small-sample or otherwise unreliable.
Media outlets and anxious voters have turned to growing services of poll aggregators to merge the contradictory polls.
Sites such as RealClearPolitics, the New York Times Upshot and Huffington Post are all confirming that Hillary Clinton has maintained a lead – of varying strength – throughout the lead up to the US Election on 8 November.
But several sites are also now offering probability estimates, and the leader in the field – Nate Silver’s FiveThirtyEight – has been alarming millions of people worldwide with his estimated probability of Republican Donald Trump winning the presidency rising from 12% to over 35% in the past fortnight.
FiveThirtyEight’s worrying prediction four days before the US election (image: FiveThirtyEight.com)
Huffington Posts’ Ryan Grim has criticised Silver’s underlying methodology, alleging that a range of assumptions and adjustments that Silver feeds into the FiveThirtyEight calculations amount to a form of political punditry, rather than pure data presentation.
Silver has responded angrily on Twitter, declaring that his methods are carefully chosen to generate a more accurate reading on the divergent and contradictory polls.
Part of what makes the issue tense is the public’s own anxiety at which form of polling to believe.
When 100% of several polls all show a candidate holding a 2% lead, it does not follow that the candidate is 100% certain to win.
Every poll has a margin of error, and can be charted as a bell curve. A 2% lead means the centre of the bell curve sits 2% to one side of the mid-point, but the edge of the curve may well extend over into the space of the alternative outcome.
A lead of 2% may indicate that the probability of the favoured alternative winning is (say) 75%, but that still leaves a 25% chance of the alternative occurring.
Probability assessments – such as those Silver is publishing prominently this election season – are actually the more serious way of making a prediction.
Statistician Nate Silver is a leader in polling methodologies, having pioneered multiple innovations over the past decade (image: 2012, AP/Huffington Post)
The problem comes when the psychology of the anxious observer is taken into account. Considering probability is scarier to the observer than considering simple poll leads.
For voters terrified at the prospect of Trump winning, seeing all the polls show Clinton with a 2% or 3% lead is fairly comforting, because many draw the false conclusion that she is certainly going to win.
But learning that there is a 35% probability that Trump will win after all is alarming, because suddenly the importance of the feared option – a 35% possibility of the apocalypse! – looms large to the viewer.
In the last week or so Silver has just slightly shifted his probability predictions for the election result in four states – Florida, North Carolina, Ohio and Nevada – from having a narrowly greater than 50% probability of going to Clinton to having an equally narrow probability of going to Trump. The colours of these states immediately changed on Silver’s front-of-website map, and alarm spread.
In fact Clinton can still win the election even without these four states, and she still has the greater probability – currently 65% – of winning the election in Silver’s overall conclusion.
Silver is getting critical press because his presentation is making people afraid. That doesn’t mean his methods are necessarily wrong.
No doubt all his assumptions, adjustments and methods will get subjected to extensive post-election critique.
Meanwhile, the apocalypse has not happened yet, and is still only 35% likely.
People should keep calm and (for US citizens) carry on voting.
A controversy knows it has arrived when it becomes a lead story on satirical website The Onion (image: The Onion)
Update: a good review of this debate today (8 Nov) at Slate.com.
On Elections 