Sources of Uncertainty

What causes uncertainty, beyond straightforward numeric probability?

Conditions

Uncertainty can affect either ‘prior probabilities’ or likelihoods. Both may be uncontentious when:

• One is working in an area where one has a proven track record at estimating probabilities.

Prior Probabilities are relatively uncontentious when:

• One has good reason to suppose that one has a genuinely random sample from a population for which one has a good statistical calculation.
• One has good reason to suppose that certain possibilities are equally uncertain (so that one can apply the principle of indifference).

Likelihoods are relatively uncontentious when:

• The processes being observed are constrained and routine.
• The hypothesis being considered is precise enough to determine meaningful likelihoods without undue averaging over cases.

Otherwise, the ability to estimate probabilities is questionable, so that one has reason to be uncertain about any estimate. There is a difference of opinion as to whether in such circumstances one should nonetheless make the best estimate one can, and live with the consequences, or take explicit account of uncertainty, and if so whether one simply performs a sensitivity analysis, varying the estimates, or if one needs a more ‘forensic’ approach.

Factors

Some of the things that can specifically contribute to uncertainty are as follows:

Complexity

If the situation is complex, it is hard to have confidence in any estimates. In particular, complexity can give rise to innovation

Reflexivity

If the probability estimate is being made in support of a decision that will impact upon the situation being observed, it may be ‘reflexive’ in the sense that what may happen depends upon the decision being made, which depends upon the estimate.

Source reliability

The likelihood of a source stating that something is the case is not the same as the likelihood of that statement. ‘They would say that, wouldn’t they’.

Vagueness

In assessing a collection of evidence against a hypothesis it is common to asses each item individually and then to ‘fuse’, to establish the overall probability. When assessing a vague hypothesis this can lead to an over-estimate of probability.

Impact on Reasoning

With probabilistic reasoning, as one gets more relevant evidence the probability will always converge to giving a probability of 1 to the truth. Hence the greater the evidence that points to a conclusion, the more one tends to suppose it to be valid.  Sensitivity analysis will tend to discount some evidence, but it remains true that the more evidence the more certain one supposedly can be of the result.

See also

Real examples

Dave Marsay