How economic models can be improved

The recent financial crisis of 2007-09 (the so-called ‘Great Recession’) uncovered the difficulties that recent structural economic models face in explaining the data. The fact that structural models face limitations in explaining and forecasting is well known. For example, Edge and Gurkaynak (2010), among others, have shown that the forecast performance of the Smets and Wouters (2007) model is not better than that of a naïve constant growth rate model during the Great Moderation period.

Moreover, the severity and the prolonged duration of the Great Recession have challenged the adequacy of existing predictors and raised the possibility that existing models might be mis-specified. For example, Ng and Wright (2013) have argued that the features of ‘financial-crisis-induced’ recessions (such as the Great Recession) are distinct from those of ‘typical’ recessions driven by supply or monetary policy shocks. This distinction may explain why the study of the Great Recession might require alternative models and predictors. Furthermore, it is unclear whether the recent financial crisis resulted from unexpected shocks or changes in the transmission mechanism. On the one hand, Stock and Watson (2012) argue that the transmission mechanism during the Great Recession is not different from that of any other post-war recession, and that the deep and prolonged recession originated simply from the larger shocks hitting the economy during that period. On the other hand, other researchers emphasise that existing macroeconomic models do not fully capture the mechanisms behind the Great Recession and argue that substantial modifications are necessary. For example, in an effort to improve the fit of structural economic models to the crisis, Del Negro and Schorfheide (2014) include information from inflation expectations, financial frictions, and interest rate spreads.

If the reason behind the severe and prolonged recession is indeed larger shocks hitting the economy, there is little that economists can do. However, when it is suspected that the poor fit of the model comes from inadequately modelled transmission mechanisms or missing channels, a typical approach to improve the model is to introduce additional features. But what features should be added? Typically, researchers use their expertise and intuition to identify which parts of the model are responsible for the poor fit. But what if a researcher has no idea where to start? And, if there are various aspects of the model that could possibly be responsible for the poor fit, how can the researcher identify which one(s) he should focus on?

New research on identifying sources of mis-specification

In a recent paper, (Inoue et al. 2014), we propose a methodology that can help researchers in that situation. Our idea is very simple. We propose to identify the sources of model mis-specification by introducing additional exogenous processes in the researchers’ models, beyond the structural shocks that the models already include. These additional exogenous processes, or ‘wedges’, are not structural shocks, but only additional processes that can potentially improve the fit of the model. In other words, wedges are distortions that incorporate possible model mis-specification which is not already accounted for by frictions built into the model. The idea is then to empirically identify which ‘wedges’ improve the fit of the models the most. Using this information, the researcher can identify which parts of the model are most likely to be mis-specified and can discover how the mis-specification of the model evolves over time. It also allows researchers to assess quantitatively how important the mis-specification is.

To locate which parts of the model are mis-specified and how empirically relevant the mis-specification is, we propose to use forecast error variance decompositions and marginal likelihood comparison. Forecast error variance decompositions measure the contributions of structural shocks as well as wedges in explaining the overall variability of the data. The larger the contribution of one or more wedges, the more likely the model at hand is mis-specified. The marginal likelihood of a model is a goodness-of-fit measure. If the original model is mis-specified, the model with wedges will have a higher marginal likelihood. Because the marginal likelihood has a built-in penalty term for un-necessary features, if the model is correctly specified, the marginal likelihood of the model with a wedge will be lower than the marginal likelihood of the model without a wedge.

Concluding remarks

Our methodology points out the lack of proper modelling of financial and labour markets in the representative macroeconomic model during the financial crisis, which may help shed light on Ng and Wright’s (2013) results, and suggests that additional work to include more labour and asset market frictions in the models would be especially useful. Our empirical findings confirm the conventional wisdom that appears in much of the existing literature, indicating that model mis-specification cannot be ignored in policy analyses.1 Furthermore, our techniques might prove to be useful more generally to guide researchers in improving their models.

References

Corradi, V and N R Swanson (2007), “Evaluation of Dynamic Stochastic General Equilibrium Models Based on Distributional Comparison of Simulated and Historical Data”, Journal of Econometrics 136, 699-723.

Del Negro, M, and F Schorfheide (2014), “Real-time DSGE Model Density Forecasts During the Great Recession”, mimeo.

Del Negro, M, F Schorfheide, F Smets, and R Wouters (2007), “On the Fit of New Keynesian Models”, Journal of Business & Economic Statistics 25, 123-143.

Edge, R M, and R S Gurkaynak (2010), “How Useful Are Estimated DSGE Model Forecasts for Central Bankers?”, Brookings Papers on Economic Activity Fall 2010, 209-259.

Inoue, A, C-H Kuo and B Rossi (2014), “Identifying the Sources of Model Misspecification”, CEPR Discussion Paper 10140.

Ng, S, and J H Wright (2013), “Facts and Challenges from the Great Recession for Forecasting and Macroeconomic Modeling”, Journal of Economic Literature, 51, 1120-1154.

Smets, F and R Wouters (2007), “Shocks and Frictions in US Business Cycles: A Bayesian DSGE Approach”, The American Economic Review 97, 586-606.

Stock, J H, and M W Watson (2012), “Disentangling the Channels of the 2007-2009 Recession”, Brookings Papers on Economic Activity Spring 2012, 81-135.

Published in collaboration with VoxEU

Author: Atsushi Inoue is a professor of economics at Vanderbilt University. Chun-Hung Kuo is an Assistant Professor of Economics at the Graduate School of International Relations at the International University of Japan. Barbara Rossi is a Professor at Pompeu Fabra University.

Image: A man walks past buildings at the central business district of Singapore February 14, 2007. REUTERS/Nicky Loh.