Who is better at predicting: the expert or the novice or someone else? Intuitively people believe experts. But science tells us a very different story. And Brexit provides some startling evidence in support.
When forecasting "... experts performed worse ... than dart-throwing monkeys" 2001 Nobel Laureate Professor Daniel Kahnemann explains in "Thinking, fast and slow" [pps 218-221].
This simple conclusion, grounded in statistics [assumed randomness of dart-throwing monkeys], is the result of Professor Philip Tetlock's 20 year research project covering 80,000 predictions of 284 experts all explained in his book "Expert Political Judgment: How Good Is It? How Can We Know?".
Part of the explanation is the illusion created by experts who provide credible explanations of past events. Experts gain credibility with the public from such explanations. We thus believe their forecasts when we should be highly skeptical.
So the most serious questions for all to consider are:
- is the UK economy being harmed by unrealiable forecasts of doom which are not true?
- Are the doom-sayers responsible for harms to the UK economy which would not otherwise happen?.
Nobel Laureate Professor Daniel Kahnemann explains:
"The illusion that we understand the past fosters overconfidence in our ability to predict the future."
Experts have been busy predicting what will happen if and when the UK leaves the European Union. And Remainers, including of course politicians who oppose Brexit and many in the media have been wheeling out the experts to tell us all what a disaster it is going to be unless we remain.
So we need to look to what the science tells us about the reliability of expert predictions.
Science tells us that:
"specialists are not significantly more reliable than non-specialists in guessing what is going to happen ..... Knowing a little might make someone a more reliable forecaster, but .. knowing a lot can actually make a person less reliable."
"Everybody's an Expert - Putting predictions to the test." A review by Louis Menand of Professor Philip Tetlock's Book "Expert Political Judgment: How Good Is It? How Can We Know?"
If experts know better how does science explain this howler by HM Treasury under former Chancellor George Osborne, which became a major part of "Project Fear" in the days and weeks leading to the 23rd June 2016 Brexit Referendum.
The predicted disaster to follow a leave vote did not happen and in over three years since it has still not happened:
"This paper focuses on the immediate economic impact of a vote to leave and the two years that follow. ....The Treasury analysis in this document uses a widely-accepted modelling approach .... one of our country's foremost economists and a former Deputy Governor of the Bank of England, ...... says that it "provides reasonable estimates of the likely size of the short-term impact of a vote to leave on the UK economy".
The analysis .... comes to a clear central conclusion: a vote to leave would represent an immediate and profound shock to our economy. That shock would push our economy into a recession and lead to an increase in unemployment of around 500,000, GDP would be 3.6% smaller, average real wages would be lower, inflation higher, sterling weaker, house prices would be hit and public borrowing would rise compared with a vote to remain."
Experts gain credibility by providing the public with their analyses and explanations of past events. Kahnemann expands on this:
"The idea that the future is unpredictable is undermined every day by the ease with which the past is explained. ... our tendency to construct and believe coherent narratives of the past makes it difficult for us to accept the limits of our forecasting ability. Everything makes sense in hindsight, a fact that financial pundits exploit every evening as they offer convincing accounts of the day's events. And we cannot suppress the powerful intuition that what makes sense in hindsight today was predictable yesterday.
"Tetlock interviewed 284 people who made-their living "commenting or offering advice on political and economic trends:' He asked them to assess the probabilities that certain events would occur in the not too distant future, both in areas of the world in which they specialized and in regions about which they had less knowledge. Would Gorbachev be ousted in a coup? Would the United States go to war in the Persian Gulf? Which country would become the next big emerging market? In all, Tetlock gathered more than 80,000 predictions. He also asked the experts how they reached their conclusions, how they reacted when proved wrong, and how they evaluated evidence that did not support their positions. Respondents were asked to rate the probabilities of three alternative outcomes in every case: the persistence of the status quo, more of something such as political freedom or economic growth, or less of that thing.
The results were devastating. The experts performed worse than they would have if they had simply assigned equal probabilities to each of the three potential outcomes. In other words, people who spend their time, and earn their living, studying a particular topic produce poorer predictions than dart-throwing monkeys who would have distributed their choices evenly over the options. Even in the region they knew best, experts were not significantly better than nonspecialists."
"Prediction" is in-apposite when dealing with complex circumstances. "Forecasting" is more appropriate. Weather forecasts are a typical example. The ability to forecast the weather reliably beyond a few days is highly limited. This is a consequence of complexity. The further into the future we look the less reliable the forecasts become. Reliable prediction is impossible when confronted by complexity.
Kahnemann further describes what during over three years of Brexit we see practically every day and sometimes several times a day and covering many broadcast hours:
"Television and radio stations and newspapers have their panels of experts whose job it is to comment on the recent past and foretell the future. Viewers and readers have the impression that they are receiving information that is somehow privileged, or at least extremely insightful. And there is no doubt that the pundits and their promoters genuinely believe they are offering such information. Philip Tetlock, a psychologist at the University of Pennsylvania, explained these so-called expert predictions in a landmark twenty-year study, which he published in his 2005 book Expert Political Judgment: How Good Is It? How Can We Know? Tetlock has set the terms for any future discussion of this topic."
And what can we say about Brexit? The state of the UK's economy today is far far better than the expert predictions over three years ago were claiming with such certainty.
1) experts gain credibility by explaining past events cogently and coherently;
2) when it comes to the future, experts are no better at forecasting than non-experts.