Do Professional Forecasters Make the Same Systematic Forecast Errors Over Time?
Our recent working paper shows that the answer is no. And we reconcile this with individuals' rationality by acknowledging they face Knightian uncertainty arising from unforeseeable structural change.
In economics, the rational expectations hypothesis (REH) is broadly interpreted as representing the expectations of rational individuals in real-world markets.
A key implication of REH is that individuals’ forecast errors should be unpredictable: they should be unbiased (zero, on average) and uncorrelated with information that was available at the time expectations are formed.
A vast empirical literature has, however, rejected this prediction using survey data on individuals’ expectations of inflation and other key macroeconomic variables. Coibion, Gorodnichenko, and Kamdar (2018) provide an overview of this literature in the context of inflation expectations.
REH core prediction has been tested by regressing survey-based ex post forecast errors—computed as the difference between ex post inflation and survey-based ex ante inflation expectations—on an intercept and various ex ante variables, including ex ante revisions of the survey forecasts of inflation or ex ante inflation. According to REH, the intercept and slope of such regressions should be zero.
In contrast to REH’s prediction, the intercept and slopes have been empirically estimated as significantly different from zero.
A particularly influential example is the paper by Coibion and Gorodnichenko (2015). Using survey-data from the Survey of Professional Forecasters, they show that ex post inflation forecast errors are significantly positively correlated with ex ante inflation forecast revisions.
Coibion and Gorodnichenko argue, however, that this should not be interpreted as demonstrating that the professional forecasters are irrational.
Indeed, they show that if individuals only have access to limited, or noisy, information instead of full information, REH predicts that their ex post forecast errors should be positively correlated with ex ante forecast revisions in line with their empirical findings. Thus, they conclude that their empirical findings are:
unlikely to be driven by departures from rationality (…) and instead reflect deviations from the assumption of full-information. (p. 2646)
In other words, they argue that their findings are consistent with the assumption that individuals are rational, but only when they base their expectations on limited information.
Several other interesting theoretical explanations have been proposed to account for the empirical rejection of full-information REH. Following the influence of Coibion and Gorodnichenko’s paper, these different theories attempt to account for the empirical finding of a positive correlation between ex post forecast errors and ex ante forecast revisions, which Coibion and Gorodnichenko call a “stylized fact” (p. 2646).1
Professional Forecasters Do Not Make the Same Forecast Errors Over Time
In a recent working paper, Roman Frydman and I examine whether the parameters of two widely used forecast-error regressions have remained constant or undergone structural shifts over time.
The first is the forecast-error regression also considered by Coibion and Gorodnichenko (2015), which regresses ex post inflation forecast errors on an intercept and ex ante forecast revisions. The second regresses them on an intercept and ex ante inflation. We use survey data on U.S. inflation expectations from the Survey of Professional Forecasters over the effective sample period from 1970 to 2021.
In other words, we ask whether the average of the professional forecasters forecast errors (the intercept) and/or their correlation with ex ante information (the slope) have shifted over the sample period.
To address this question, we first estimate the forecast error regressions for three distinct subsamples. We choose the timing of the subsamples to match times when inflation dynamics are broadly agreed-upon to have changed. Thus, similar to Hajdini and Kurmann (2025) and Angeletos et al. (2021), we split the sample into three subsamples: 1970-1979, 1980-2019, and 2019-2020.
The first subsample, from 1970 to 1979, is characterized by high inflation and large inflation forecast errors and revisions. The second subsample is the Great Moderation period from 1980 to 2019, characterized by low and stable inflation and inflation expectations. The third subsample, from 2020 to 2021, is the COVID-19 period, characterized by high inflation and forecast errors.
(The following conclusions regarding the subsample estimates do not change if we split the sample into subsamples at slightly different times.)
The figure below shows the scatterplot of the ex ante forecast revisions on the horizontal axis and the ex post forecast errors on the vertical axis for the full sample, in Panel (a), and each of the three subsamples, in Panels (b) to (d). The red lines correspond to the estimated slope measuring the correlation between ex post inflation forecast errors and ex ante inflation forecast revisions.
Panel (a) illustrates the positive correlation found when estimated based on the full sample period, which confirms Coibion and Gorodnichenko’s finding.
The other panels, however, show that the estimated correlation between ex post forecast errors and ex ante forecast error revisions has shifted significantly over the three subsamples.
During the 1970s subsample, the estimated correlation between ex post inflation forecast errors and ex ante forecast revisions is positive. But it is, by a small margin, not significantly different from zero. The estimate is 1.117, and the HAC standard error is 0.595, corresponding to a p-value of 0.069.
We also get a positive estimate during the final subsample, and for this subsample, it is significantly different from zero. This should, however, be interpreted cautiously as this subsample only has 8 observations.
In contrast, the estimated correlation is a lot smaller and clearly statistically insignificant during the Great Moderation subsample, which has 160 quarterly observations. The estimate is 0.072, and the HAC standard error is 0.139, which corresponds to a p-value of 0.604.
In other words, the professional forecasters have not made the same type of inflation forecast errors during the three subsamples.
These estimates suggest that a positive correlation with ex ante forecast revisions does not appear to be a stylized fact of inflation forecast errors. Rather, it appears to be the result of not accounting for structural changes. Indeed, the significantly positive estimate based on the full sample, as illustrated in Panel (a) in the figure above, seems to be mainly driven by the positive correlation during the 1970s.
Identifying Structural Breaks from the Data
A potential problem with splitting the sample into subsamples at chosen dates, as we did above, is that they need not correspond to the actual dates of structural changes in the time-series data.
To overcome this problem, we also identify structural breaks in the forecast-error regressions from the data. To this end, we use multiplicative indicator saturation with automatic model selection with the Autometrics algorithm, which I will explain in a future post. An important feature of this approach is that it enables identifying structural breaks in the intercept and slope of the forecast error regressions at different times and during the entire sample period.
Using this approach, we find multiple structural breaks in both the intercept and slopes of the two forecast error regressions.
Figure 3 below, reprinted from our working paper, illustrates the shifts in the estimates. Panels (a) and (b) show the estimates of the intercept and slope of the regression of ex post forecast errors on ex ante forecast revisions, so the estimated slope in Panel (b) corresponds to the subsample estimates illustrated in Figure 2 above. Panels (c) and (d) show the estimates of the intercept and slope of the regression on ex ante inflation. In all panels, the red lines show the estimates at each point in time over the sample period, and the vertical bars show the estimated HAC standard errors. Panels (a) and (d) also show the ex post inflation forecast errors in black lines.
Here, I will focus only on the estimate of the correlation between ex post inflation forecast errors and ex ante forecast revisions, as illustrated in Panel (b). The graph shows that this estimated correlation was only significantly positive during a few observations in the 1970s (specifically from 1972:Q2 to 1974:Q1). In contrast, the estimated correlation was insignificantly different from zero for the rest of the sample period.
This confirms the overall conclusion from the subsample estimates. There was only a positive correlation between ex post forecast errors and ex ante forecast revisions during a brief period in the 1970s. For the rest of the sample period, there appears to be no such correlation.
Thus, instead of accounting for a positive correlation between ex post forecast errors and ex ante forecast revisions, the key feature we should seek to account for appears to be the shifts in this correlation—and in the average level of the ex post forecast errors—between values with different signs and statistical significance over time.
Reconciling Our Findings with Rationality
Although our empirical findings are inconsistent with REH’s predictions, Roman Frydman and I propose in the working paper that they are consistent with individuals’ rationality once we acknowledge that they face unforeseeable structural changes in the inflation process.
We show theoretically that our approach to formalizing unforeseeable change and rational expectations in the presence of the Knightian uncertainty arising from such change gives rise to structural shifts in the parameters of the two forecast error regressions.
The intuition behind this theoretical result is that nonrepetitive and unforeseeable structural changes imply that rational individuals cannot have perfect probabilistic knowledge of how inflation will unfold in the future. Thus, their expectations are based on inherently imperfect knowledge of the future, even when they are completely rational and have access to full information.
This implies that their inflation expectations deviate from actual inflation outcomes in nonrepetitive and unforeseeable ways. This gives rise to a systematic component in their ex post forecast errors: they can have a non-zero mean and be correlated with ex ante information, including ex ante forecast revisions and ex ante inflation.
Importantly, however, the mean and correlation of their ex post forecast errors with ex ante information undergo nonrepetitive and unforeseeable structural shifts over time. These shifts arise from shifts in the inflation process, as well as from shifts in individuals’ inflation expectations in anticipation of and response to those shifts.
This illustrates that acknowledging unforeseeable structural change changes not only our understanding of rational expectations, but also the empirical evidence that we rely on to test our theories.
References
Angeletos, Huo, and Sastry (2021), “Imperfect Macroeconomic Expectations: Evidence and Theory,” NBER Macroeconomics Annual, 35, 1–86. https://doi.org/10.1086/712313
Coibion and Gorodnichenko (2015), “Information Rigidity and the Expectations Formation Process: A Simple Framework and New Facts.” American Economic Review, 105 (8), 2644–2678. https://doi.org/10.1257/aer.20110306
Frydman and Tabor (2025), “A New Explanation of REH’s Empirical Difficulties: Unforeseeable Change in Rational Participants’ Inflation Expectations,” INET Center on Knightian Uncertainty Working Paper 25-02.
Hajdini and Kurmann (2025), “Predictable Forecast Errors in Full-Information Rational Expectations Models with Regime Shifts,” Federal Reserve Bank of Cleveland, Working Paper 24-08. https://doi.org/10.26509/frbc-wp-202408
An important paper within this literature is the working paper by Hajdini and Kurmann (2025). They show that if the inflation process switches between repetitive regimes according to a Markov switching process, full-information REH implies that rational individuals’ forecast errors are non-zero on average and correlated with ex ante information. Thus, full-information REH’s core prediction that rational individuals’ expectations should be unpredictable only holds when based on models that assume the inflation process remains constant.
Hajdini and Kurmann’s REH model and the approach Roman Frydman and I have developed differ in their formalizations of structural change. In Hajdini and Kurmann’s model, the structural change is repetitive and can be foreseen, in probabilistic terms. In our approach, the structural change is nonrepetitive and unforeseeable, even in probabilstic terms.