Why don't errors accumulate in climate models when the time horizon increases?

Question

First of all, I want to add the disclaimer that I am not a climate sceptic or anything. I honestly want to understand this phenomenon. Hopefully, someone can also go into the details as I do have somewhat of a statistical background.

---

According to the Guardian article, "‘Word salad of nonsense’: scientists denounce Jordan Peterson’s comments on climate models", Jordan Peterson made the claim that climate models couldn't be relied on because errors compound as you forecast further into the future:

[Peterson] said: "Another problem that bedevils climate modelling, too, which is that as you stretch out the models across time, the errors increase radically. And so maybe you can predict out a week or three weeks or a month or a year, but the farther out you predict, the more your model is in error.

"And that’s a huge problem when you’re trying to model over 100 years because the errors compound just like interest."

Scientists have responded saying this understanding is wrong:

Dr Sarah Perkins-Kirkpatrick, a climate scientist at the University of New South Wales Canberra, said Peterson’s description of how climate models work was fundamentally wrong. While weather forecasts do become less accurate the further out they go, this was a different process to climate modelling. [...]

Prof Steve Sherwood, of the Climate Change Research Centre at the University of New South Wales, said Peterson was “making the ancient climate sceptic error of mixing up weather and climate”.

“Anyone who has taken an introductory course in climate or atmospheric science would spot this problem,” he said. “Errors in a weather forecast indeed accumulate such that after a couple of weeks the forecast is useless.”

But with climate, Sherwood said, the models work differently to project how the climate will respond to different factors, such as higher levels of CO2.

So why is this the case? Why don't errors accumulate?

Answer 1

Error does accumulate in climate models out into the future -- that's why the forecasts include a range of possible results, that gets wider the further into the future you look.

See, for instance, this figure from the IPCC's Fifth Assessment Report, Summary for Policy Makers (SPM):

Each RCP is a pathway, depending on how much carbon we emit. Per Wikipedia, the number represents the expected radiative forcing, in watts per meter squared (W/m²) that can be expected in the year 2100.

The SPM explains what each RCP represents:

Representative Concentration Pathways, (RCPs) which are used for making projections based on these factors, describe four different 21st century pathways of GHG emissions and atmospheric concentrations, air pollutant emissions and land-use. The RCPs include a stringent mitigation scenario (RCP2.6), two intermediate scenarios (RCP4.5 and RCP6.0) and one scenario with very high GHG emissions (RCP8.5). Scenarios without additional efforts to constrain emissions ('baseline scenarios') lead to pathways ranging between RCP6.0 and RCP8.5 (Figure SPM.5.a). RCP2.6 is representative of a scenario that aims to keep global warming likely below 2°C above pre-industrial temperatures.

So if we stay on our current path, we can expect warming of 3 to 5.5 degrees, with sea level rise of 0.5 to 1 m.

Answer 2

There is a huge confusion between errors in weather forecasting and climate prediction.

While weather predictability is fundamentally limited to a few weeks, in climate modelling we are interested in the statistical properties of the modelled weather, even if the modelled weather for a given day in 10 years from now is completely random.

The climate system is extremely complex but perhaps it is instructive to compare this with simple fluid dynamics. In a turbulent pipe flow we cannot predict where each small vortex will be. If we have an initial condition, we can only accurately follow it for a very limited time even in very accurate direct numerical simulations. But it does not matter, because the statistical properties of such a turbulent flow can be captured very accurately by such simulations.

Answer 3

Our uncertainty in climate prediction increases over time, but not because of accumulated errors in the way that (for example) compound interest accumulates.

In compound interest each calculation relies on the one before it. If you tried to build on a weather forecast in this way the errors very quickly become enormous, since a weather forecast relies on making short-term but (hopefully) high-accuracy predictions based on relatively small-scale things like wind speed and surface temperature. The trouble arises if we then try to use those forecasted values as the input for the next day's calculations – our errors will quickly snowball because the uncertainty in our forecasts is pretty big compared to the measurements. Because of this, we don't accumulate predictions as the timescales increase.

Climate forecasts are made across much longer periods, with larger-scale observations (we're talking decades, centuries or more), and different statistical models, but have nothing to do with what the weather is now, or what we think it will be tomorrow or the day after. The fact it's currently 9ºC and a bit cloudy is as irrelevant to climate forecasts as the fact that 1904 was the coldest year on record is to if it's going to rain tomorrow.

There are actually a whole bunch of forecasts which use progressively longer-term and larger-scale data, from the hyperlocal ("it's probably going to rain in your neighbourhood in the next five minutes") up to the geological ("there's probably going to be an ice age on the Earth in the next few thousand years"). You can find a 'long range forecast' for most places (here's the one for the UK) which gives an example of an intermediate, looking a month or so out at what the weather is likely to be but without specific predictions.

Answer 4

Consider a Galton board. A simple bead pummels down the cascade of pegs. Can we predict what path it is going to take?

Well, in principle, yes – it's after all just mechanics. If it hits the first peg a little bit to the left of its center, it'll go to the left there. Measure the start state very accurately, and you can predict all that comes later.

But it's evidently hopeless to actually predict the whole path – it's way too sensitive. Even the tiniest change at the start will mean it scatters off the first peg with a slightly different spin, and then this again changes the behaviour at the second, so already after two iterations prediction is sure to be off. For all intents and purposes, the behaviour is just random then, despite following deterministic microscopic dynamics.

Roughly the same reason is responsible for weather forecasts being little use over more than a week.

Nevertheless, what we can say with very good confidence is that the bead is going to end up somewhere near the middle after 1000 rows of pegs. It's not strictly speaking impossible that it goes to the left twice as often as to the right, but so very unlikely that we might rather worry about the board being destroyed by a plane crashing into the lab.

And if all the pegs are slightly biased to make it 5% more likely for the bead to go to the right at each of them somehow, that's not going to be evident from observing a couple of beads on the first rows – it'll still look totally unpredictable. However, it will show up clearly in the shifted peak of the distribution after 1000 rows.

Answer 5

The claim in the quote is not what the question title is asking about. Climate models do accumulate error over time, just as weather models (and most models attempting to predict the future of almost anything) do.

What the quote is saying is that climate models are not based on weather models. Weather models are designed to produce very high precision forecasts (relative to climate models, at least.) This requires them to use very precise (and very recent) inputs to the models, such as current surface temperatures, pressure gradients, wind fields, jet stream/streak locations, frontal boundaries, temperatures and winds aloft, etc. in order to predict the likely weather at a particular location at a particular time of day a couple of days from now.

Climate models aren't just plugging today's weather observations into the GFS, NAM, and EURO models and grabbing the predicted conditions for 3:00 pm on Jan 31, 2072. Instead, their inputs are averages over long periods of time (generally decades or more) with a goal of predicting averages over long periods of time (again, generally decades or more.) This doesn't mean that error doesn't accumulate over time; it does. But the error doesn't accumulate over time on anywhere near the same scale as in weather forecasts because the inputs (and outputs) are far less variable than with weather forecasts, due to averaging over much, much longer periods of time.

As with almost any sort of time data - especially rather variable data like weather observations - averages over long periods of time are generally much more stable than instantaneous observations. Error still accumulates over time, just not nearly as quickly.

Answer 6

My understanding is that errors in both types of models result from subtle differences in the approaches.

In weather models, the full Navier-Stokes equations are solved. Due to the non-linear and hence chaotic nature of these equations, one generates an uncertainty in the time-evolution by varying the initial conditions, such as the temperature $T_0$ by some Gaussian, or other prior function.

Climate models on the other hand, have the chaoticity averaged out from them, hence varying the initial conditions will produce no variability in the long-term evolution. That variability there originates in the uncertainties of the evolution parameters, like the cloud condensation pressure.

Answer 7

Climate models are not purely mathematical constructs that exist independent of physical reality; various parameters represent real world conditions and the modeling seeks to incorporate known climate processes as closely as possible.

Representation of those processes and interaction within a model seeks to conform with what is known about the physical processes; if it is well modeled it won't be able to do something that isn't physically possible - like have decades of enhanced greenhouse effect with reduced outbound IR but no ocean warming. Or statistically unlikely, like an iterative succession of uncertainty errors biased in one direction of the kind I suspect Mr Peterson imagines, where the statistically unlikely but physically possible tails of probability bell curves are considered to be just as likely as near the middle of the bell curve.

We really should assume a genuine intent by climate scientists to better understand climate process interactions and an intent to produce the more accurate climate projections that they are being asked to provide. Accusations otherwise - of manipulating climate data or modeling to support a political agenda or to cover incompetence or scam grant money - should come with evidence and be presented to the proper authorities.

Hindcasting - taking known past real-world conditions and modeling the climate within time frames where the real world conditions are known - is an excellent check on how well they work.