San José State University
Thayer Watkins
Silicon Valley
& Tornado Alley

Uncertainty in Climate Model Projections

There is a distinction between forecasts and projections. A forecasting model or method attempts to predict what actually happens on the basis of information known before it happens. A projection, on the other hand, says what will happen under a set of assumptions. Some of the assumptions are reasonable and necessary such as that there will be no major volcanic eruptions during the period over which the projections are made. Such assumptions have to be made for forecasts as well. However there is one overwhelming assumption involved with a projection and that is that the projection model is valid. Given that assumption there can be no conceptualization of uncertainty in the projections.

But this is not acceptable. The projectors release their projections to the media who present them to the public as forecasts without the media or the public being aware of the crucial assumption about the validity of the model. And it is not that projection models cannot be validated without waiting decades. The projection models can be run backwards to compute what the model says should have been the climate in the known past. These are known as backcasts (probably more properly linguistically they should be called hindcasts).

There can be two types of such backcasts. One type would include the known past conditions such as volcanic eruptions to try to reproduce past climate conditions. A good performance in this matter would provide validation of the model. The second type would project past conditions using only the type of information that is involved in making future projections. The deviations between the actual past and the computed past provide information for establishing confidence limits for the future projections.

As it is now the matter of uncertainty of the projections is handled in an absurd way by the Intergovernmental Panel on Climate Change (IPCC). The IPCC uses a rediculous number of projections created by various groups. At one time it was 15 and later 22. These are rediculous numbers. If climate forecasting were a hard science then it would be easy to choose the best one or the best few and forget the rest. But it is not a hard science and the models have not been validated by backcasting so the IPCC is stuck with a whole herd of projections. The IPCC then reports the extremes from the models and treats this as though it represents an objective measure of the uncertainty of the projections. At least it sounds scientific but it is in fact bogus. Not all of the climate models are unvalidated. A few performed some version of the backcasting test and they were found to be invalidated. The IPCC nevertheless kept these invalidated models because they were useful for generating scary forecasts.

Some scientists outside of the field of climatology think a climate model is valid if it depends only upon valid scientific principles. This is incorrect. Climate models generally are in error not due to what they include but because of what they fail to include. To illustrate this principle consider the matter of the ice-albedo feedback. Albedo is the proportion of solar radiation reflected back into space. According to the ice-albedo feedback mechanism as the Earth warms more ground and water would be uncovered which would absorb a higher proportion of the incoming solar radiation thus raising the temperature and melting more ice and snow.

Seiji Kato and his associates at the Langley Center of NASA published in 2006 the results of an investigation of the effect of decreasing sea ice in the Arctic on the amount of radiant energy reflected from the Arctic. This study concluded that although there was a decrease in sea ice in recent years there was an increase in cloudiness that more than made up for the loss of albedo from the sea ice. Thus there was not only no ice-albedo positive feedback presumed by climate modelers, there was in fact a negative feedback. Kato's result illustates the admonition that in climatology every theory has to be checked empirically.

Thus when a physicist looks at the components of a climate model and sees that they are based upon thermodynamics and fluid mechanic principles it is easy to think this make the model valid. When the model does not take into account the more difficult problem of cloud cover it is not valid. J.R. Houghton expressed the situation as follows:

Clouds are, in fact, probably the dominant influence in the radiative budget of the lower atmosphere but adequately taking them into account raises many problems [�] The Physics of Atmospheres, p. 41.

J.R. Houghton after the time of this statement went on to supervise the work of the IPCC.

The ultimate test of validity of a climate model is empirical; i.e., does it pass the backcasting test? Such backcasting is generally not done or if it is done it is not released to the public. Patrick J. Michaels in his book, Meltdown, gives the backcasting of two climate models from about 1993 back to 1905. One is the first Coupled Global Climate Model (CGCM1) from the Canadian Centre for Climate Modeling and Analysis and the second is British, from the Hadley Centre for Climate Prediction and Research. The data presented below were scaled from the Michaels' graph. These backcast made considerable use of current past data to reproduce some approximation of the past temperature time series. They are not the backcast analogous to the future projection which would be driven by carbon dioxide concentration. Never the less they are what little is available.

The correspondence of the backcast with the actual temperature for the Canadian model is displayed below.

The correspondence is not impressive but it is definite. The coefficient of determination for the regression of actual temperature on the CGCM1 computed temperature is 0.37. The regression cofficient is 0.82, indicating that the projected temperature change for the model should be reduced 18 percent.

For the British Hadley model the results are much less impressive.

Here the coefficient of determination is a statistically insignificant 0.01. Basically the Hadley model fails to be of any value for forecasting. The regression coefficient is -0.1, indicating that the projected changes from the Hadley model should to scaled back 90 percent and changed in direction.

For a climate model that has some correlation with the past data the model estimates should be converted into a recalibrated estimate using the regression equation. The regression results then allow upper and lower confidence limits to be establsihed as shown in the diagram below.

The confidence limits have their smallest spread at the mean value of the data used in the regression. They increase the farther distance from that mean value. In the case of the climate model projections the projections start from a known value so there can be no error for the starting point. For the first steps of the projection the error is likely to be very small but the potential for error grows with each step of the projection. Therefore a simple correlation overstates the performance of the model because the correlation gives equal weight to the correspondences in the early steps of the projections where there can be very little error as to the correspondences in the later part of the projection.

The Nature of the Problem

Consider a leaf on a tree that is about to fall. There is a simple differential equation that describes the fall of a body under gravitation. This equation is the model and it will give the position and velocity as a function of time. Applied to the leaf it will tell when the leaf will hit the ground immediately below its initial position. The model is based upon confirmed laboratory science.

What is left out is the element of air resistance. The equation of the model can be modified to allow for air resistance and the resulting equation can be solved numerically. However the simplistic modification of the equation will not give a realistic description of leaf as it tumbles turbulently back and forth before reaching its resting place. If the event were repeated over and over again there would be a distribution of leaves on the ground under the tree. Some would be piled directly under the initial position and others scattered farther away. The pile might have the shape of a bell.

If there was a wind the high point of the distribution would be shifted away from the point directly below the initial position.

Although the model projected a specific end point really what should have been sought was the distribution of the final points. The errors in the projections were due to applying a model that was valid for special cases under laboratory conditions, but did not take into account factor of air resistance.

There is book called Irrelevant Arithmetic that makes the point that making predictions about natural phenomena is an entirely different matter than making predictions concerning phenomena in a controlled, laboratory setting. Projecting the Earth's climate is about the ultimate in a natural phenomenon that is entirely divorced from a laboratory setting.

(To be continued.)

HOME PAGE OF applet-magic
HOME PAGE OF Thayer Watkins