And the winner is…
(not Climate experts Latif and Keenlyside)
group @ 17 November 2015
Remember the forecast of a temporary global cooling which made headlines around the world in 2008? We didn’t think it was reliable and offered a bet. The forecast period is now over: we were right, the forecast was not skillful.
Back around 2007/8, two high-profile papers claimed to produce, for the first time, skilful predictions of decadal climate change, based on new techniques of ocean state initialization in climate models. Both papers made forecasts of the future evolution of global mean and regional temperatures. The first paper, Smith et al. (2007), predicted “that internal variability will partially offset the anthropogenic global warming signal for the next few years. However, climate will continue to warm, with at least half of the years after 2009 predicted to exceed the warmest year currently on record.” The second, Keenlyside et al., (2008), forecast in contrast that “global surface temperature may not increase over the next decade, as natural climate variations in the North Atlantic and tropical Pacific temporarily offset the projected anthropogenic warming.”
This month marks the end of the forecast period for Keenlyside et al and so their forecasts can now be cleanly compared to what actually happened. This is particularly interesting to RealClimate, since we offered a bet to the authors on whether the results would be accurate based on our assessment of their methodology. They ignored our offer but now the time period of the bet has passed, it’s worth checking how it would have gone.
Keenlyside and colleagues specifically forecast temperatures for the overlapping decadal periods of Nov 2000-Oct 2010, and Nov 2004-Oct 2015. At the end of the first period, we checked in on the forecast and noted that the predicted global cooling had not occurred. We can now update this for the second period as well.
Blue dots and curve show global temperature data ( NASA GISTEMP), red dots with confidence intervals the model forecasts and hindcast, for the same 10-year intervals chosen by Keenlyside et al in their paper. (The observed 12-month running average is also shown.) Temperature anomalies are shown relative to the period November 1994 – October 2004 as in the Keenlyside paper. The forecasts for a cooling (relative to the previous 10-year interval) were clearly wrong: decadal average temperatures have kept increasing, despite the ever-present shorter-term variability shown in the grey curve.
It is clear that prediction of global cooling or even stasis was way off the mark, with global warming continuing and observations running more than 0.15ºC warmer than the Keenlyside et al forecast.
If we examine the spatial pattern, we can see, again, that the initialized prediction did much worse than the free-running, radiatively forced model ensemble (which was surprisingly accurate). This initialization was to be the major innovation of the paper, leading to specific forecasts that were supposed to be closer to observations. The opposite turned out to be the case:
Above: Predicted spatial patterns in Keenlyside et al (2008), in the initialized runs (top) and in the free running A1B scenario. Below: The spatial patterns for the prediction period derived from the GISTEMP analysis.
We stand by our original assessment that the ‘coupling shock’ in the Keenlyside methodology can produce cooling artefacts (as shown in Rahmstorf 1995) and that this was knowable at the time of publication. We also gave further reasons that should have raised red flags with the reviewers of the paper, such as the negative skill scores for the model in the critical region of the northern Atlantic and the poor performance of the hindcasts (two hindcasts for earlier periods were both unskilful, including that for the decade immediately preceding the forecasts, shown in the graph above).
We can also assess the forecasts of the Smith et al (2007) paper (through to the end of 2014):
While the trajectory in the initialized simulations was slightly closer to the observations than their “No Assimilation” simulations, the real world trajectory was significantly outside their error bars for most of the period. Their figure also demonstrates that the impact of initialisation is only detectable for about 5 or 6 years. Some of the offset may well be related to the forcings that were applied post-2005 which may have been too strong (see Schmidt et al. (2005) for some assessment of that).
Summary
Had our bet been accepted, it is clear we would have won unambiguously.
It’s important to be clear about what these unsuccessful attempts at prediction mean. The authors of the papers involved are to be commended for trying something new and making real and falsifiable predictions. However, much more care should have been taken to self-critically examine their potential skill and emphasise their experimental nature, and the levels of certainty expressed in press releases and popular media should have been dialled way down. We offered the bet at the time as we were concerned that the failed forecasts would in the end cast a shadow on the credibility of climate science as a whole, so we felt a need to emphasise that other climate scientists disagreed with these forecasts. The techniques for decadal prediction have advanced somewhat since these earliest attempts (as described in Meehl et al., 2014), but no skilful initialized predictions have yet been verified. Claims then that this kind of decadal prediction is useful for policy-makers or scientists thus remain very premature.
In October 2010, the German newspaper Stuttgarter Zeitung, asked the Keenlyside authors for their comment on the failure of the first period of the prediction. Mojib Latif responded that the continued warming did not speak against their study; one should look at this long term and not attach too much significance to a few years. And he said that if the forecast turns out to be wrong by 2015, “I will be the last one to deny it”. We showed these results to him yesterday and he responded that this was an experimental product, and that the main idea was to shake up the scientific community. In that at least, it was successful!
- See more at: realclimate.org |
