How often do economic forecasters get it right?

The accuracy of long-term macroeconomic forecasts substantially affects the sustainability of government debt and social security systems. However, long-term forecasts cannot avoid significant uncertainty because the future developments of new innovations and their diffusion are difficult to assess quantitatively. In addition, many unforeseeable events such as financial crises, international conflicts, and major natural disasters occur.

Upward bias in growth forecasts by governments

Many studies have indicated that economic growth forecasts by government agencies tend to be biased upward. In the case of Japan since the fiscal year 2000, the government’s GDP growth forecast for the next fiscal year has, on average, had an upward bias of 0.5 percentage points and 1.4 percentage points for real and nominal GDP growth rates, respectively. The upward bias in economic growth forecasts often causes optimistic bias in the projections of fiscal balance and debt level. However, these studies have mostly focused on short-term forecasts of one or two years.

Frankel (2011) and Frankel and Schreger (2013), which analyse three-year forecasts of government agencies, are two of the very few studies evaluating medium-term forecasts. These studies indicate that the optimistic bias tends to increase with the time horizon of the forecasts. However, studies on the accuracy of long-term forecasts, such as those involving a ten-year time horizon, have been scarce. In addition, as far as the author is aware, there has never been a study evaluating macroeconomic forecasts of academic researchers in economics.

Against this background, in recent work (Morikawa 2019) I empirically assess whether academic researchers in economics make accurate long-term growth forecasts. Specifically, I use unique survey data on macroeconomic forecasts on real/nominal GDP growth made in the mid-2000s to evaluate the ex-post performance of the 10-year forecasts.1

A ssurvey for economics researchers

The survey data are taken from the Survey of Long-Term Outlooks of the Japanese Economy, designed by the author and conducted by the Ministry of Economy, Trade and Industry in 2006 and 2007. The survey questionnaire was sent to all economics researchers belonging to the Japanese Economic Association (JEA) and to about 100 professional forecasters in private economic institutes who are not members of the association. The Japanese Economic Association is the counterpart of the American Economic Association in the US and the European Economic Association in Europe. A total of 608 responses were obtained; 94% of the respondents were association members and 77% were professors/researchers belonging to universities.

The survey asked respondents to report their long-term forecasts of real/nominal GDP growth rates for the next ten years. Specifically, the survey asked for the expected annualised growth rates (up to the first decimal point). Since realised figures of the variables are now available, ex-post forecast errors or the accuracy of the ten-year forecasts can be evaluated.

It should be noted that the survey asked for forecasts based on an assumption of no significant exogenous shocks. Given the effect of the 2008–2009 Global Crisis on the Japanese economy, we calculate the mean realised growth rates by removing figures for the fiscal years 2008 and 2009 and use these rates to calculate the forecast errors by subtracting realised rates from the forecasted rates. Hence, positive figures mean upward bias in the forecasts, and negative figures denote downward bias.

Ex-post evaluation of the accuracy of forecasts

Figure 1 shows the means and medians of forecast errors. Forecasts by the researchers in economics have an optimistic bias, similar to the findings for the forecasts by the government agencies. Even after removing the two years affected by the Global Crisis to calculate annual growth rates, a non-negligible upward bias remains: about 0.5-0.6 percentage points for real GDP growth and about 1.3-1.4 percentage points for nominal GDP growth.

However, the bias in economics researchers’ growth forecasts is smaller than those of the government. According to the New Economic Growth Strategy drawn up by the Japanese government in 2006, the real GDP growth forecast was 2.2% per year for fiscal years 2005–2015.2 The realised real GDP growth rate for this period, after removing the two years affected by the Global Crisis, was 1.3%, meaning that the government forecast had an upward bias of 0.9 percentage points.

Why are economic researchers’ growth forecasts, which are unlikely to be affected by political pressure, upward biased? Although it is difficult to determine from the data, a possible interpretation is that the respondents tend to extrapolate past growth performance at the time of making forecasts. Another possibility is that the actual economic policies conducted by the government were inappropriate relative to the respondents’ expectations.

Size of the bias by individual characteristics

The survey collected information about the respondents’ gender, age, affiliation, and research field (19 JEL classifications).3 Figure 2 shows the regression coefficients for the individual characteristics to explain their forecasting errors.

The coefficients for the Japanese Economic Association member dummy are negative and significant at the 1% level in real and nominal GDP growth forecast errors. Although the size of the coefficients is not large, the upward bias is about 0.2–0.3 percentage points smaller for forecasts by academic researchers than by professional forecasters from private institutes who are not members of the Japanese Economic Association.

In contrast, the coefficients for the dummy for researchers specialising in macroeconomics are positive and significant in real and nominal GDP growth forecast errors. The long-term economic forecasts by researchers specialising in macroeconomics and/or economic growth are somewhat less accurate, although the differences are quantitatively small (about 0.2 percentage points).

Why are their forecasts relatively inaccurate? One possible reason is that they were frequently in contact with information about the statistics of high growth rates and the government’s economic outlook at the time of forecasting.

These results show that long-term growth forecasts involve significant uncertainty, even for economics researchers specialising in macroeconomics or economic growth.

References

Frankel, J (2011), “Over-optimism in forecasts by official budget agencies and its implications”, Oxford Review of Economic Policy 27(4): 536–62.

Frankel, J, and J Schreger (2013), “Over-optimistic official forecasts and fiscal rules in the Eurozone”, Review of World Economics 149(2): 247–72.

Morikawa, M (2019), “Uncertainty in long-term macroeconomic forecasts: Ex post evaluation of forecasts by economics researchers”, RIETI Discussion Paper 19-E-084.

Endnotes

1 See Morikawa (2019) for details of the study.

2 The New Economic Growth Strategy did not provide nominal GDP growth forecasts.

3 Respondents were allowed multiple answers to the question regarding field of research. On average, about two fields were chosen per respondent.