The economic calendar is packed with forecast releases, and each time a new one comes out, social media bursts forth with comments denigrating forecasts.
Common themes are: forecasting and economic modelling are different activities; forecasting is trash-talk, modelling or analysing the present is the serious business; forecasting requires one set of skills, modelling another (viz discussion of whether you need ‘foxes’ or ‘hedgehogs’ for them).
My response to this is to point out that there is not a good separation between modelling and forecasting. A model implies a forecast.
An example of what prompted me to write this is this piece by John Kay, which, while not sinning, is open to a mis-reading by sinners.
Tom Sargent was fond of repeating every lecture that ‘a model is a probability distribution over a sequence’. What he meant was that if you have written down a model, it will contain in it a statement about how likely certain things [defined in your ‘sequence’] like output or inflation are to take on different values at different times. Or, if you project into the past, it will tell you how likely it was that output and inflation in the past ended up being what it was.
This statement was always couched in terms of macroeconomic time series. But it is more general than that. The ‘sequence’ could be the set of individuals in a workforce, whose behaviour you haven’t measured yet, but you are trying to predict from what you have measured.
Take the Bank of England’s model as an example. This model involves a decomposition of all past output and inflation into a set of ‘shocks’. The model will tell you the contribution productivity shocks were making at any point in time in the past. But also, and since the model will tell us that shocks take time to have their full effect, we can work out without any extra maths, assumptions, or clairvoyance, what the model tells us about the chance of output and inflation being within certain ranges out into the indefinite future.
The Monetary Policy Committee don’t just do this. They introduce off model-judgements to bend the forecast to what they think is most likely. But this doesn’t mean that forecasting is something else apart from modelling. It means that they are averaging across models; or modifying the model.
So, when John Kay writes:
“A bane of this economist’s life is the belief that economics is clairvoyance. I should, according to this view, be offering prognostications of what gross domestic product growth will be this year and when the central bank will raise interest rates.”
I’d say: Fair enough. But, without any clairvoyance, your understanding of the macroeconomy [your model] means that you are implying something – even if you haven’t stated it explicitly – about what would, other things equal, given what you know now, happen to growth and interest rates in the future, and when.
Or when he writes:
“It is usually easy to move the subject on to something more interesting than macroeconomic forecasting.”
I’d say: maybe so. But your conversation partners are failing to parse the logic of economics fully if they think that you really have changed the subject away from forecasting.
John is right to say that economics is not clairvoyance. But economics – explanations of the economy’s present workings – contains within it statements about the future.
There are of course lots of differences between models [or, equivalently, between forecasts]. Models that are chosen to best fit in sample. And those chosen to best fit out of sample. Models with or without explicit Bayesian priors. These are models where the probability distribution over the sequence – to indulge in the Sargent language – is bent to achieve different criteria.
Following this line of thought, it doesn’t make sense to talk of modelling being interesting but forecasting boring. Or models requiring one set of skills and forecasts another. Or forecasts, but not models, being trash.
Although I admit that it would be a bit of a turn off to try to pitch a column about probability distributions over sequences bent to achieve one criterion needing foxes, while those bent to meet another needed hedgehogs.
longandvariable 
As I used to say, “If you don’t make forecasts, you don’t know what to be surprised by”.
Yep. And, if you make forecasts, and screw up, it is telling you something about the stories you are telling about what is going on at the moment, or what happened recently.
I’ll have to disagree on this one. Models have implications that are like forecasts, but not in the same way as what many people think of as macroeconomic forecasts from the private sector or policy institutions. The kind of forecasts Tetlock or Kay were talking about or the probability distribution of future outcomes discussed by Sargent takes the model and the actual numbers very seriously. But most models (or economic or political commentary criticised by Tetlock) are meant more as rough directional suggestions of what is likely to happen (where likely could leave a lot of room for other outcomes), or of x effect being significant in a general sense of being worth watching out for. i.e the model is not meant to be taken as much more than a guide to intuition or an exploration of alternative ideas on how the economy works, maybe as a guide for restrictions on more empirically oriented models (e.g DSGE-VAR). See for example the classic article by Varian and Gibbard (1978)
Click to access Economic_Models.pdf
or Gilboa et al here,
Click to access GPSS.pdf
.
Rough predictions coming from taking a model less seriously+the recognition that each model on its own provides only a very partial perspective (best to be seen as part of a suite of many models and perspectives, rather than used on its own) may not be as falsifiable or scientific as would be demanded by a formal forecast comparison. But they’re still useful, and maybe more humble than the claim that we can forecast the macroeconomy well with ultimately highly unrealistic policy or private sector macro models.
But isn’t it more useful to spell out your intuition as a well defined mathematical model that other people can example and use? Intuition is famous for being person specific and difficult to share.
Plus if models aren’t taken seriously for forecasting, then why should we take them seriously for explaining the past? If we look at the past and then select from a library of different models for the best fit, isn’t that the same as telling a “just so” story?
I’d say that if you ultimately resort to intuition to produce a forecast, you’re still using a model: only it’s one that can’t be examined by anyone else.
Tony, in the spirit of forecasting, if you were looking for forecasts of the effective federal funds rate in the US after last month’s rate rise, where would you turn? I’m curious, because I know of one model that did well, but I wonder if there’s anything to compare it to (other than reality).
I allways think forecast is only an advice and you dont have to trust it at 100%