23 February 2018

Dr Michael Gasiorek is Senior Lecturer in Economics at the University of Sussex and Director and  Managing Director of InterAnalysis respectively. He is a Fellow of the UKTPO.

Before and since the Brexit referendum there have been numerous criticisms made of economic models, of the views of ‘experts’ and the supposed inaccuracy of their forecasts.[1] But these critiques are mostly based on misunderstandings, so, as an economist and long-time modeller, I want to explain the value – and the limitations – of modelling.  Models are indeed extremely useful and should be used to help inform public policy – but you need to use them appropriately.

Myth buster: “Models are not designed to provide accurate predictions / forecasts of future reality”.

This is for at least two reasons. First, by definition, any prediction or forecast (I am using these terms interchangeably) is concerned with an uncertain event in the future.  There may be uncertainty both with regard to what is being explicitly included in the model, and there may be uncertainty with regard to other events that will also impact on the final outcomes. There are always unforeseen events and interactions which no model can capture.

Second, and closely related, all models require simplification. Think of a geographical map as a two-dimensional spatial model.  Most people find maps extremely useful as a way of navigating around. But a physical map that was a complete representation of reality would have to be on a scale of 1:1. This would not be useful as it contains too much information and would be unmanageable. If I am travelling from A to B, I want to know where the roads are, and the quality of those roads and I do not need to know the position, size and structure of every single building along the way. If I am going for a trek in the mountains, I am likely to want to have a map that includes the paths and the contours which give me the lie of the land. I choose the map (model) that is suited to the objective, and each map involves simplifications, and those simplifications are useful. They are useful precisely because they do not capture all of reality. They highlight areas of importance and enable us to examine those areas which are of interest.

Consider the two maps below of the London Underground. The first is geographically accurate and is useful for understanding where the actual location of the different underground services. The second is more schematic and is more useful for the traveller wishing to use the underground to get from A to B.

The same reasoning applies to economic models. They involve simplifications. Each model will have its own objectives, in the same way as do different maps.  Each model will shed light on particular characteristics and mechanisms and by design leave others out. This is intentional.  Of course in setting some things aside, the model cannot fully capture all the underlying economic mechanisms and therefore can never provide a completely accurate prediction of the future. It is not designed to do so. A good model will provide insights into those aspects which it deals with; for policy work, it will provide a coherent and consistent framework for considering how changes in policy may translate into economic outcomes. Frameworks can and should be assessed and challenged, but they should not be dismissed simply because they are frameworks, i.e. models.

Let me give a simple Brexit related example, derived from modelling work I have recently been engaged in with colleagues at the UKTPO.[1] There is much talk about different possible Brexit outcomes ranging from an EEA style deal, to a CETA++ deal, and to crashing out with no-deal. If I were a policymaker I would want to understand how an EEA style option might impact on trade and output in comparison to, for example, having no deal with the EU. Similarly, I might want to understand how a given Brexit scenario might impact differentially on UK regions. Suppose the model suggested that no deal was considerably worse: first I would want to make sure that I understood why this was the case, and what was driving the differences in magnitude, or the regional differences.  Second, understanding these factors may help me in deciding on the desired policy (or route). Third, it may help to focus on policies that may mitigate against negative outcomes or  consider other factors that may also impact on trade and output.

If I want to know how long it will take to get from A to B, I would look at a map to see, for example, whether or not there is a motorway between A and B, and if not how hilly the road is. I might also wish to understand why route A, might take longer than route B. If I want to know how much bigger are the impacts from no deal in comparison to an EEA style deal, I would look at how much the UK produces, how much it trades with the EU, how much it trades with other countries and what the possible increase in trade barriers associated with these two scenarios might be.

Note that all of the preceding is data (fact) and is central to any model just as it is central to any map. To that data, we then need to add some assumptions as to how producers and consumers might respond to the changes in prices driven by the changes in policy. These are also central to the model. Just like in travelling from A to B we need to make some assumption about the normal level of traffic on the road, and the speed I am likely to drive at. If my assumptions about speed and the level of traffic are reasonably accurate then my predicted time of arrival will be more or less right. Similarly, if the assumptions the modeller makes about the way producers and consumers respond are reasonably accurate then the predicted impact will be more or less right given those assumptions. As one of my colleagues puts it (paraphrasing Keynes): “the numbers are precisely wrong, but roughly right”.

This does not mean that this will necessarily be the outcome. Clearly, in reality, other factors may intervene, so to continue with my analogy there may be a traffic jam or poor weather conditions. This does not make the map wrong. It simply means that the map and the model did not and could not take certain factors into account. That is the nature of uncertainty. Note also, this does not mean that all models are good. Just as it is perfectly possible to create an illegible or inaccurate map, it is equally possible to create a bad model. But, just because bad models are possible, this does not mean that all models are bad. It is also possible to have an excellent model but simply not to understand how to use and interpret the results from the model. In the same way, as it is possible to read a map and still get lost.

But to generically rubbish models and denigrate experts is dangerous for public policymaking.  What is needed is to understand the purpose and limitations of models, and thus to recognise their usefulness in informing the debate and decision-making as opposed to providing the last word.

To answer the rhetorical question at the beginning – can economists gaze into crystal balls? The answer is no. We cannot predict the future. But we can provide useful models to guide policymaking.

 

Footnote

[1] Some examples of this from politicians engaging in the Brexit debate include: “people in this country have had enough of experts”; “The reports from the government on the economic effects of Brexit have been such rubbish so far that I can’t think it is worth the paper it is written on; “I am not a fan of economic models because they are all proven wrong”; the only purpose of economic forecasts is to make astrology look respectable” .

 

Disclaimer:

The opinions expressed in this blog are those of the author alone and do not necessarily represent the opinions of the University of Sussex or UK Trade Policy Observatory.

Republishing guidelines

The UK Trade Policy Observatory believes in the free flow of information and encourages readers to cite our materials, providing due acknowledgement. For online use, this should be a link to the original resource on our website. We do not publish under a Creative Commons license. This means you CANNOT republish our articles online or in print for free.