Uncertainty in science and its role in climate

Smith, Leonard A. and Stern, Nicholas (2011) Uncertainty in science and its role in climate policy. Philosophical transactions of the Royal Society A: mathematical, physical and engineering sciences, 369 (1956). pp. 4818-4841

This review is from a general science and policy perspective, not specific to climate.

1. Introduction

Policy-making, or at least sound policy-making, is often about risk management. Thus, … science supports sound policy when it informs risk management, informing the selection of … policy measures that influence key aspects of the [relevant] causal chain … .

… Science often focuses on what is known and what is almost known; dwelling on what one is unlikely to know even at the end of one’s career may not aid the scientist’s career, yet exactly this information can aid the policy-maker. Scientific speculation, which is often deprecated within science, can be of value to the policy-maker as long as it is clearly labelled as speculation. … Thus, for the scientist supporting policy-making, the immediate aim may not be to reduce uncertainty, but first to better quantify, classify and communicate both the uncertainty and the potential outcomes in the context of policy-making. The immediate decision for policy-makers is whether the risks suggest a strong advantage in immediate action given what is known now.

…. Writing in 1921, Knight noted that economics ‘is the only one of the social sciences which has aspired to the distinction of an exact science . . .  it secures a moderate degree of exactness only at the cost of much greater unreality’. A similar competition between exactness and unreality is present in climate science, where differences between our models of the world and the world itself lead to situations where we cannot usefully place probabilities on outcomes (ambiguity) alongside situations where we can place probabilities on outcomes (imprecision) even if we cannot determine the outcome precisely.

2. Distinguishing some varieties of uncertainty

Imprecision (Knightian risk, conditional probability): related to outcomes … for which we believe robust, decision relevant probability statements can be provided … also called ‘statistical uncertainty’.

Ambiguity (Knightian uncertainty): related to outcomes (be they known, unknown or disputed), for which we are not in a position to make probability statements.Elsewhere called ‘recognized ignorance’ and ‘scenario uncertainty’. Ambiguity sometimes reflects uncertainty in an estimated probability, and is then referred to as ‘second-order uncertainty’.

Intractability: related to computations known to be relevant to an outcome, but lying beyond the current mathematical or computational capacity to formulate or to execute faithfully; also to situations where we are unable to formulate the relevant computations.

Indeterminacy: related to quantities relevant to policy-making for which no precise value exists. This applies, for instance, with respect to a model parameter that does not correspond to an actual physical quantity. It can also arise from the honest diversity of views among people, regarding the desirability of obtaining or avoiding a given outcome. … 

3. The causal chain and sound climate policy

This notes how events and potential events can often be understood in terms of ‘causal chains’, and that often just a vague notion of the chains, without any numbers, is enough to inform initial policy-making, without any numbers. Causality is not defined as such, but seems to mean ‘may lead to change’.

4. The impact of uncertainty in a policy context

An example is given to show how some realistic ambiguity can be represented and accommodated in decision-making.

5. Handling ambiguity in science

Science is often found to be a reliable guide to action, even if it can never provide true certainty. And just as there is a scientific approach to forecasting what will  happen, science can also inform questions of what is believed virtually impossible to happen. Ambiguity covers a third category: things that cannot be ruled out, but to which today’s science cannot attach a decision-relevant probability. It is this third category where scientists often prefer not to tread. Progress here can be of significant value for policy-making. When asked intractable questions, the temptation is to change the question, slightly, to a tractable question which can be dealt with in terms of imprecision and probability, rather than face the ambiguity of the original, policy-relevant, question. Science will be of greater service to sound policy-making when it handles ambiguity as well as it now handles imprecision.

Ideally, science engages with the policy discussion and decisionmaking. Discussing the implications of intractability can also benefit policymaking, as can investigating when indeterminacy might reduce the impact of imprecision or ambiguity in the science on policy-making.

Scientists and statisticians can model anything they can think of. Models then imply probabilities which may or may not reflect the likelihoods of events in the world. Either way, these ‘implied probabilities’ are well-defined mathematical concepts that can inform policy as long as they are accompanied by a clear statement of the probability that model is fundamentally flawed, and thus when the implied probabilities will prove misleading.

… For phenomena that are known to be important, informing decision-makers a priori as to the limited range (of liquidity, rainfall or metal stress) outside which the model is likely to be mis-informative can lead both to better policy-making for the future and to better real-time decisions when using the model daily. The use of ensembles of models cannot be expected to yield robust implied probabilities when all of the models have similar flaws in their mathematical structure.

Ensembles can be informative, but … interpreting them as probabilities is reminiscent of Wittgenstein’s remark on someone buying several copies of today’s morning paper in order to gain confidence that what the first copy said was true. …

There is no principled method for moving from simulations to decision-relevant probabilities …

For phenomena best modelled with nonlinear models, the utility of ‘implied probabilities’ in traditional decision theory is in doubt. …

Fruitful engagement of science with policy-making could yield new forms of model design, and alter the experimental design applied to current climate models. One could debate the probable value of models that offer an interesting perspective with those that aim to give quantitative probabilistic forecasts of outcomes given assumptions that are known not to hold, contrasting the scientist’s desire to develop a series of models which approach reality (eventually) with the decision-maker’s desire for the most relevant information available from science today.

Guidance on the most urgent places to gather information and realistic estimates of when to expect more informative answers from current research are both of immediate value. Even in the presence of ambiguity, today’s science may suggest which observations to make in order to aid model improvement, to distinguish competing hypotheses or to provide early warning that our understanding is more limited than we believed (as when things that cannot happen, happen). …

6. Ambiguity and insight in science

Inasmuch as all probability forecasts are conditional on something, the distinction in climate modelling between predictions and projections is artificial if the word projection is intended merely to flag the fact that the forecast will depend on the emission scenario which comes to pass. … So, in this case the word ‘projection’ is used to improve communication of the fact that the choice of emission scenario is subject to substantial uncertainty. Focusing on scenario uncertainty can suggest that, if (when) the emission pathway is known, today’s models will be able to produce both (what would have been) a realistic simulation and decision-relevant probabilities. It is unlikely that this is the case.

To interpret model-based probabilities for climate at the end of this century as reflecting some aspect of the world is to commit Whitehead’s fallacy of misplaced concreteness.

 7. Improving the support science provides climate policy-making

… Engaging with the policy process and communicating the current level (and limits) of scientific understanding will lead to more effective policy-making than merely providing clear statements of state of the science in terms familiar to the scientists themselves.

… Current incentives for research programmes are not tuned to benefit policy support in the long run. A solid piece of important mathematics or physical analysis that advances our understanding, but is of little immediate practical value, may prove of less value in securing a research position than the development and first implementation of some parametrization scheme for some ‘penguin effect’-like phenomenon.

… To oversimplify: advances in pure science reduce ambiguity and clarify questions of intractability, while advances in applied science and simulation increase the relevance of our conditional probabilities for decision-making by quantifying imprecision better. …

8. Concluding remarks

Sound policy-making embraces the causal chain connecting actions by people to impacts on people. Many varieties of uncertainty are encountered along this chain, including: imprecision, ambiguity, intractability and indeterminacy. Science regularly handles the first with probability theory; ambiguity and intractability are more often used by scientists to guide the advancement of science rather than being handled within science explicitly. A better understanding by scientists of the roles of uncertainty within policy-making may improve the support science offers policy-making. In particular, an improved understanding of which scientific uncertainties pose the greatest challenges to policy-making when projected along the entire causal chain considered by policy, and informed scientific speculation on the likelihood of reducing those specific uncertainties in the near future, would be of immediate value. …

… Communicating to policy-makers the level of confidence scientists have that their model-based probabilities are not mis-informative is at least as important as communicating the model-based probabilities themselves. Engagement of scientists in the policy-making process, not merely by presenting the outputs of models but by explaining the insights from science, can significantly improve the formation of policy. … Scientists who merely communicate results within the comfortable area of reliable theory abandon the decision stage to those who often have little engagement with the science. Sound policy-making is then hindered by the lack of sound scientific speculation on high-impact events, which we cannot currently model but may plausibly experience. Failing to engage with the question ‘What might a 6°C warmer world look like, if it were to occur?’ leaves only naive answers on the table for policy-makers to work with.

The expected utility approach is difficult to apply when one is unable to translate possible outcomes into impacts on people. … This approach also struggles with low-probability events; the vanishingly small probabilities that mathematical modelling may suggest are not actually zero should not distract policy-makers from action either. …

Society might better understand science and benefit from science if science as a whole was more effective at communicating ambiguity and its implications.

[P]olicy-makers could encourage the engagement of scientists by: …
— discrediting the view among some scientists that policy-makers are only interested in ‘one number’ which must be easy to understand, unchangeable and easily explained in less than 15 min.

Clearer distinction between imprecision and ambiguity would also be of value, as would a deeper engagement with ambiguity. It would be better to answer a policy-relevant question directly with ambiguity than to answer a similar sounding approximate but largely irrelevant question precisely.

The advance of science itself may be delayed by the widespread occurrence of Whitehead’s ‘fallacy of misplaced concreteness’. In areas of science, far removed from climate science, an insistence on extracting probabilities relevant in the world from the diversity of our model simulations exemplifies misplaced concreteness. Computer simulation both advances and retards science, as did the astonishing successes of the Newtonian model, Whitehead’s original target. In any event, better communication of uncertainty in today’s science, improved science education in the use of simulation modelling that values scientific understanding of the entire system, and the communication of all (known) varieties of uncertainty will both improve how science handles uncertainty in the future and improve the use of science in support of sound policy-making today. How science handles uncertainty matters.

 Comments

There are useful references to Knight and Whitehead. Whitehead’s student, Keynes, develop much relevant mathematics of uncertainty, which was developed by Russell into an approach to science. These would contribute to the science issues that Smith and Stern raise. In particular:

  • The notion of causality in the paper is very ‘linear’, and could benefit from some of the ideas that Whitehead takes from Kant and Bergson.
  • It is not clear to me that Knight intended his form ‘Knightian uncertainty’ to be merely ambiguity.
  • I would pick out reflexive uncertainty. Thus it may be reasonable to suppose that the emissions of minor, poor, peoples will be the greater the more probable they think it that climate goals will be achieved.  (This is also a ‘tragedy of the commons’.)

 See Also

My notes on uncertainty and related debates.

 Dave Marsay

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

6. Ambiguity and insight in science

Dave Marsay

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: