One of the central problems in economic forecasting is the problem of being unable to run randomised trials on different courses of action. Forecasting is therefore based on limited data and models with a strong ideological input. Now that the result of the EU Referendum is known, the arguments over the trajectory of the economy are likely to intensify.

The great statistician George E P Box is often quoted as declaring “All models are wrong, but some are useful”. When I was a student, a friend doing PPE showed me a very simple mathematical model that linked trade and GDP growth with a slight lag that had worked from the early 50s and apart from a few incidents (1967 devaluation for instance) was a reliable model. After the end of Bretton Woods and then the oil shocks in 72-74, no such simple relation was re-established. Knowing when to abandon a model is a major challenge.

Shocks can have beneficial as well as deleterious effects on both short and longer term economic prospects for sure. The problem however is to know if the model used for prediction is “useful”.

It is argued by some that after the ERM crash in 1992 that the economy boomed despite warnings of the “catastrophic” downsides. The example of Canada was cited as historical evidence for austerity post 2008 because it had cut public spending while the economy grew during its period of crisis.

In both cases, the US economy was in a long period of growth and demand was rising, helping both countries. That is not now the case.

Before the Referendum, most economists argued that exit would lower the UK’s long term growth and Osborne warned of a slow down or recession. He now calls it an adjustment. Michael Gove talked about being fed up with experts.

It strikes me that the difference between the camps is an attitude to equilibrium based models.

Those arguing for the optimistic picture are arguing that the shock will be short lived and there will be a boost as the economy goes back to its original course with fewer regulations boosting investment and productivity.

The other camp appears to be arguing that the shock will essentially be structural not cyclical with long term destruction of potential. Time will tell which it is. However, holding economists to account for their forecasts is rarely practised.

These differences essentially are the difference between stable and unstable equilibria. In a stable system the shock oscillates until dissipated. In an unstable equilibrium there is no way back.

I suspect that it will have a mixture of both factors. W Edwards Deming, one of the fathers of statistical process control and quality systems had a number of games to demonstrate the difference between common cause variations in a systems model from “special cause” variations. Common cause variations are those that are variations within a system while special cause variations are external factors. The important lesson is that tampering with a stable system and not knowing if the variation was common or special makes the outcome uniformly worse, there is no better outcome.

In his final book, the “New Economics” he showed how the interest rates in the US had been adjusted throughout the 1980s to respond to trade figures. Yet looking at it from a system viewpoint, there were only 1 or 2 quarters in the 1980s where there was evidence of behaviour outside variation of a stable system. That is to say that there was no evidence to support most of the interest rate changes as having any beneficial effects.

It’s easy to throw examples of poor forecasts at economists of all persuasions. The problem now is that if the shock of Brexit makes some of the models used obsolete then after a decade of nonstandard measures such as negative interest rates and quantitative easing we may be heading for another decade of exotic policies on the basis of limited evidence.

One interesting take on the potential that the models may be broken is given by Ross Anderson who argues that the Treasury's analysis of Brexit effects underestimates the network effects around technology companies.

J K Galbraith’s quote “It is my guiding confession that I believe the greatest error in economics is in seeing the economy as a stable, immutable structure” may well be tested.

I suspect another of his quotes “The only function of economic forecasting is to make astrology look respectable” will be used many times in the coming months.

Nassim Nicholas Taleb’s “Antifragile” needs to be read and understood if this process is to work.

It is his sub-title that matters here: “things that gain from disorder”. Once you abandon the notions of the economy as a stable equilibrium model resilient to shocks the world changes. The pre-2008 conceit that the global financial system was resilient was ripped apart and has not been put together again in 8 years. My reading of the book is that our global financial system is still brittle by design.

Going forward driving with the limited view of a broken rear mirror won’t be helpful if the process is to be made to work. I fear that this shock will have made many models in use obsolete, so these tools may not be useful. Time will tell.