On September 21, 2011 we reported on the lawsuit against scientists who provided earthquake forecasts to the Italian government. The lawsuit arose after the deadly earthquake in L'Aquila in April 2009. On October 22, 2012, the seven forecasters were sentenced to 6 years in jail and have been ordered to pay $10.2 million in damages. The verdict is unusual over the past century in free societies. Based on the reports we have seen, there was no claim that there was intent to harm, nor are we aware that evidence was presented that the scientists failed to use proper procedures. It is extremely difficult to predict the timing of earthquakes, so "bad forecasts" are common. Until this matter is resolved, we advise experts to avoid making forecasts of natural disasters in Italy.

When forecasting outside of Italy, we advise forecasters to use evidence-based principles of forecasting. For example, forecasters should provide evidence-based information on uncertainty. Failing to use evidence-based principles seems to be the only sound basis to challenge bad forecasts—other than intent to deceive, which sometimes occurs in financial markets. Consider the field of medicine: Doctors are often successful sued when they fail to use evidence-based procedures. For forecasting, we refer readers to "Standards and Practices for Forecasting". As far as we are aware, it is the only published list of evidence-based principles of forecasting. It contains a short review of law cases involving faulty forecasts in the last section.

Scott Armstrong and Kesten Green

Compare your forecast for Chart A with the blue trend line in the following chart of temperatures. Monitor your Chart B forecast accuracy progress at The Climate Bet site. The Quiz is based on an idea from Warren Meyer's presentation of two charts cut and pasted from a larger temperature chart (see p. 48 of his presentation here).


Copy of "Mystery" temperature chart quiz result as pdf, here.

Grade your response out of 10*, as follows...

A squiggly line with a strong trend: 0

A squiggly line with a moderate trend: 1

A squiggly line with no obvious trend: 2

A straight line with a strong trend: 3

A straight line with a moderate trend**: 7

A straight line with no trend***: 10

*The grades are based on the principle that one should be conservative in forecasting when there is much uncertainty and complexity."

**Give yourself a 3 point bonus if your line matched the very modest downward trend exhibited by the actual temperature series over the 25 years you forecasted.

***Congratulations! Your forecast was consistent with the evidence-based forecasting principle of conservatism that applies on this situation.

Results for the Mystery Temperature Chart Quiz from an informal survey

# of responses

A squiggly line with a strong trend: 15

A squiggly line with a moderate trend: 8

A squiggly line with no obvious trend: 4

A straight line with a strong trend: 9

A straight line with a moderate trend: 5

A straight line with no trend: 9

A straight line that best matched the blue trend line above 1

In a new (2 May 2012) report from the Global Warming Policy Foundation, Indur Goklany concludes that policies intended to reduce global warming may cause more harm than good to the health of people in both developing and industrialized countries. In particular, poverty is a much greater threat to health. The report is available here.

Current version of working paper (please do not cite):

Green, KC, Soon, W, & Armstrong, JS. (2013) "Evidence-based Forecasting for Climate Change Policies".

Temperature Forecasting Quiz

Just for fun, test your climate forecasting skill by printing the image of the Quiz page showing two charts, below, and draw in your 25-year forecasts. (A pdf file copy of the Mystery Temperature Chart Quiz is available here.)


When you have made your forecasts, check them against the actuals and grade them, here.


Slides used in 25 June 2012 International Symposium on Forecasting (Boston) panel session on climate forecasting. Panelists providing commentary were Leonard Smith, Richard Lindzen, and Robert Fildes.

Previous evidence-based climate forecasting papers

Fildes, R. & Kourentzes, N. (2011). Validation and forecasting accuracy in models of climate change. International Journal of Forecasting, 27, 968–995.

Armstrong, J. S., Green, K. C., & Soon, W. (2011). Research on forecasting for the manmade global warming alarm. Energy and Environment, 22, 1091-1104. [Testimony to U.S. House Committee on Science, Space, and Technology]

Green, K. C., Armstrong, J. S., & Soon, W. (2009). Validity of climate change forecasting for public policy decision making. International Journal of Forecasting, 25, 826-832.

Armstrong, J. S., Green, K. C., & Soon, W. (2008). Polar bear population forecasts: A public-policy forecasting audit. Interfaces (with commentary), 38, 382-404.

Armstrong, J. S., Green, K. C., & Soon, W. (2008). What is the appropriate public-policy response to uncertainty? Interfaces, 38, 404-405.

Green, K. C. & Armstrong, J. S. (2007). Global warming: Forecasts by scientists versus scientific forecasts. Energy and Environment, 18, 997-1021.

Testimony to the U.S. House of Represntatives on forecasting for the manmade global warming alarm by Scott Armstrong, Kesten Green, and Willie Soon has now been published in Energy & Environment. A copy of of the published testimony is available here.