Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

P. Goodwin and G. Wright, 1994, Heuristics, biases and improvement strategies in judgmental time series, Omega, 22, 553-568.

Goodwin and Wright summarize some of the research on judgmental time series forecasting. Most of the literature is recent with all but two studies being done since 1980. The review overlaps substantially with Goodwin and Wright (1993). In total, this review covers 86 studies, while their previous one covered 83; about 2/3 of the studies are the same.

In their 1993 review Goodwin and Wright described research on uses of judgment beyond direct application to extrapolations. These include (1) making judgmental adjustments to quantitative extrapolations, (2) deciding how to decompose the forecasting problem, and (3) combining judgmental and extrapolation forecasts. There are of course, other ways that judgment can be applied to extrapolations. It can be used to select an appropriate extrapolation method (e.g. Hill and Fildes, 1984; Armstrong and Collopy, 1993). It can also be used to weight the components of a combined forecast. For example, Collopy and Armstrong (1992) found that structured domain knowledge can improve the weights.

This 1994 paper builds upon their previous work by providing advice under two conditions: (1) judgmental forecasting when one has no information other than the time series of interest, and (2) when one has a time series and domain knowledge. They do not address judgmental forecasting when one has no time series data. Thus, for example, their recommendations do not apply to new product forecasting.

Their summary of research on how to improve judgmental extrapolation is useful. One conclusion is to use graphical time series displays where short-term forecasts are required, the series is non-seasonal, and the data are not noisy,

Some of the conclusions on improving judgment are counter intuitive. For example, training in forecasting does not seem to improve a person’s ability to make judgmental extrapolations. Of course, one wonders whether this might depend on the type of training.

Researchers can use this review (or Goodwin and Wright, 1993) as a good starting point for studying the conditions under which various judgmental strategies are useful. Goodwin and Wright recommend that more research is needed for the conditions under which one has time series and domain knowledge. Practitioners can use the recommendations to improve their judgmental forecasting.

My reservations apply not only to the current paper, but also to their earlier paper (Goodwin and Wright, 1993). In each case, the summary of prior research is impressionistic. The use of meta-analytic procedures might have helped (Rosenthal, 1995). We do not know how the literature search was conducted or what criteria were used for including studies. Other researchers, may, like me, note that their studies have been ignored. For example, I v as surprised that Adam and Ebert (1976) was not included. Also, the conclusions of the studies were not summarized in a systematic way. Judgmental calls are important to the conclusions or reviews (Wanous et al. 1989). One procedure that we have found helpful in reviews is to send a draft of the paper to each person whose research is summarized to ensure that the interpretation is correct (Armstrong and Lusk, 1987). Thus, while I agree with most of their conclusions and recommendations, I had this uneasy feeling that a more systematic review is needed in this important and rapidly developing area.


Adam, E.E., Jr. and R.J. Ebert, 1976, A comparison of human and statistical forecasting, AIIE Transactions, 8, 120-127.

Armstrong, J. and F. Collopy, 1993, Causal forces: Structuring knowledge for time series extrapolation, Journal of Forecasting, 12, 103-115.

Armstrong, J. and E.J. Lusk, 1987, Return postage in mail surveys: A meta-analysis, Public Opinion Quarterly, 51, 233-248.

Collopy, F. and J.S. Armstrong, 1992, Rule-based forecasting: Development and validation of an expert systems approach to combining time series extrapolations, Management Science, 38, 1394-1414.

Goodwin, P. and G. Wright, 1993, Improving judgmental time series forecasting: A review of the guidance provided by research, International Journal of Forecasting, 9, 147-161.

Hill, G. and R. Fildes, 1984, The accuracy of extrapolation methods: An automatic Box-Jenkins package (SIFT), Journal of Forecasting, 3, 319-323.

Rosenthal, R., 1995, Writing meta-analytic reviews, Psychological Bulletin, 118, 183-192. Wanous, J.P., S.E. Sullivan and J. Malinak, 1989, The role of judgment calls in meta-analysis, Journal of Applied Psychology, 74, 250-264.