Who thinks probability is just a number? A plea.

Many people think – perhaps they were taught it – that it is meaningful to talk about the unconditional probability of ‘Heads’ (I.e. P(Heads)) for a real coin, and even that there are logical or mathematical arguments to this effect. I have been collecting and commenting on works which have been – too widely – interpreted in this way, and quoting their authors in contradiction. De Finetti seemed to be the only example of a respected person who seemed to think that he had provided such an argument. But a friendly economist has just forwarded a link to a recent work that debunks this notion, based on wider  reading of his work.

So, am I done? Does anyone have any seeming mathematical sources for the view that ‘probability is just a number’ for me to consider?

I have already covered:

There are some more modern authors who make strong claims about probability, but – unless you know different – they rely on the above, and hence do not need to be addressed separately. I do also opine on a few less well known sources: you can search my blog to check.

Dave Marsay

What should replace utility maximization in economics?

Mainstream economics has been based on the idea of people producing and trading in order to maximize their utility, which depends on their assigning values and conditional  probabilities to outcomes. Thus, in particular, mainstream economics implies that people do best by assigning probabilities to possible outcomes, even when there seems no sensible way to do this (such as when considering a possible crash). Ken Arrow has asked, if one rejects utility maximization, what should one replace it with?

The assumption here seems to be that it is better to have a wrong theory than to have no theory. The fear seems to be that economies would grind to a holt unless they were sanctioned by some theory – even a wrong one. But this fear seems at odds with another common view, that economies are driven by businesses, which are driven by ‘pragmatic’ men. It might be that without the endorsement of some (wrong) theory some practices, such as the development of novel technical instruments and the use of large leverages, would be curtailed. But would this be a bad thing?

Nonetheless, Arrow’s challenge deserves a response.

There are many variations in detail of utility maximization theories. Suppose we identity ‘utility maximization’ as a possible heuristic, then utility maximization theory claims that people use some specific heuristics, so an obvious alternative is to consider a wider  range. The implicit idea behind utility maximization theory seems to be under a competitive regime resembling evolution, the evolutionary stable strategies (‘the good ones’) do maximize some utility function, so that in time utility maximizers ought to get to dominate economies. (Maybe poor people do not maximize any utility, but they – supposedly – have relatively little influence on economies.) But this idea is hardly credible. If – as seems to be the case – economies have significant ‘Black Swans’ (low probability high impact events) then utility maximizers  who ignore the possibility of a Black Swan (such as a crash) will do better in the short-term, and so the economy will become dominated by people with the wrong utilities. People with the right utilities would do better in the long run, but have two problems: they need to survive the short-term and they need to estimate the probability of the Black Swan. No method has been suggested for doing this. An alternative is to take account of some notional utility but also take account of any other factors that seem relevant.

For example, when driving a hire-car along a windy road with a sheer drop I ‘should’ adjust my speed to trade time of arrival against risk of death or injury. But usually I simply reduce my speed to the point where the risk is slight, and accept the consequential delay. These are qualitative judgements, not arithmetic trade-offs. Similarly an individual might limit their at-risk investments (e.g. stocks) so that a reasonable fall (e.g. 25%) could be tolerated, rather than try to keep track of all the possible things that could go wrong (such as terrorists stealing a US Minuteman) and their likely impact.

More generally, we could suppose that people act according to their own heuristics, and that there are competitive pressures on heuristics, but not that utility maximization is necessarily ‘best’ or even that a healthy economy relies on most people having similar heuristics, or that there is some stable set of ‘good’ heuristics. All these questions (and possibly more) could be left open for study and debate. As a mathematician it seems to me that decision-making involves ideas, and that ideas are never unique or final, so that novel heuristics could arise and be successful from time to time. Or at least, the contrary would require an explanation. In terms of game theory, the conventional theory seems to presuppose a fixed single-level game, whereas – like much else – economies seem to have scope for changing the game and even for creating higher-level games, without limit. In this case, the strategies must surely change and are created rather than drawn from a fixed set?

See Also

Some evidence against utility maximization. (Arrow’s response prompted this post).

My blog on reasoning under uncertainty with application to economics.

Dave Marsay

The limits of (atomistic) mathematics

Lars Syll draws attention to a recent seminar on ‘Confronting economics’ by Tony Lawson, as part of the Bloomsbury Confrontations at UCLU.

If you replace his every use of the term ‘mathematics’ by something like ‘atomistic mathematics’ then I would regard this talk as not only very important, but true. Tony approving quotes Whitehead on challenging implicit assumptions. Is his implicit assumption that mathematics is ‘atomistic’? What about Whitehead’s own mathematics, or that of Russell, Keynes and Turing? He (Tony) seems to suppose that mathematics can’t deal with emergent properities. So What is Whitehead’s work on Process, Keynes’ work on uncertainty, Russell’s work on knowledge or Turing’s work on morphogenesis all about?

Dave Marsay

 

Evolution of Pragmatism?

A common ‘pragmatic’ approach is to keep doing what you normally do until you hit a snag, and (only) then to reconsider. Whereas Lamarckian evolution would lead to the ‘survival of the fittest’, with everyone adapting to the current niche, tending to yield a homogenous population, Darwinian evolution has survival of the maximal variety of all those who can survive, with characteristics only dying out when they are not viable. This evolution of diversity makes for greater resilience, which is maybe why ‘pragmatic’ Darwinian evolution has evolved.

The products of evolution are generally also pragmatic, in that they have virtually pre-programmed behaviours which ‘unfold’ in the environment. Plants grow and procreate, while animals have a richer variety of behaviours, but still tend just to do what they do. But humans can ‘think for themselves’ and be ‘creative’, and so have the possibility of not being just pragmatic.

I was at a (very good) lecture by Alice Roberts last night on the evolution of technology. She noted that many creatures use tools, but humans seem to be unique in that at some critical population mass the manufacture and use of tools becomes sustained through teaching, copying and co-operation. It occurred to me that much of this could be pragmatic. After all, until recently development has been very slow, and so may well have been driven by specific practical problems rather than continual searching for improvements. Also, the more recent upswing of innovation seems to have been associated with an increased mixing of cultures and decreased intolerance for people who think for themselves.

In biological evolution mutations can lead to innovation, so evolution is not entirely pragmatic, but their impact is normally limited by the need to fit the current niche, so evolution typically appears to be pragmatic. The role of mutations is more to increase the diversity of behaviours within the niche, rather than innovation as such.

In social evolution there will probably always have been mavericks and misfits, but the social pressure has been towards conformity. I conjecture that such an environment has favoured a habit of pragmatism. These days, it seems to me, a better approach would be more open-minded, inclusive and exploratory, but possibly we do have a biologically-conditioned tendency to be overly pragmatic: to confuse conventions for facts and  heuristics for laws of nature, and not to challenge widely-held beliefs.

The financial crash of 2008 was blamed by some on mathematics. This seems ridiculous. But the post Cold War world was largely one of growth with the threat of nuclear devastation much diminished, so it might be expected that pragmatism would be favoured. Thus powerful tools (mathematical or otherwise) could be taken up and exploited pragmatically, without enough consideration of the potential dangers. It seems to me that this problem is much broader than economics, but I wonder what the cure is, apart from better education and more enlightened public debate?

Dave Marsay

 

 

Traffic bunching

In heavy traffic, such as on motorways in rush-hour, there is often oscillation in speed and there can even be mysterious ‘emergent’ halts. The use of variable speed limits can result in everyone getting along a given stretch of road quicker.

Soros (worth reading) has written an article that suggests that this is all to do with the humanity and ‘thinking’ of the drivers, and that something similar is the case for economic and financial booms and busts. This might seem to indicate that ‘mathematical models’ were a part of our problems, not solutions. So I suggest the following thought experiment:

Suppose a huge number of  identical driverless cars with deterministic control functions all try to go along the same road, seeking to optimise performance in terms of ‘progress’ and fuel economy. Will they necessarily succeed, or might there be some ‘tragedy of the commons’ that can only be resolved by some overall regulation? What are the critical factors? Is the nature of the ‘brains’ one of them?

Are these problems the preserve of psychologists, or does mathematics have anything useful to say?

Dave Marsay

Flood Risk Puzzle

This is a ‘Natural Hazards problem’ that has been used to explore people’s understanding of risk. As usual I question the supposedly ‘mathematical’ answer.

 Suppose that the probability that your house will be hit one or more times by the natural hazard during an exposure period of one year is .005. That is, if 1000 homes like yours were exposed to the natural hazard for one year, 5 of the homes would be damaged. Please estimate the probability that your home would avoid being hit by the natural hazard if exposed to the hazard for a period of 5/10/25/50 years.

You might like to ponder it for yourself.

In elementary probability theory the appropriate formula is 1-(1-p)n, where p=0.005 is the probability for one year and n is the number of years. But is this an appropriate calculation?

My in-laws were being charged a high property insurance premium because homes ‘like theirs’ were liable to flooding. They appealed, and are now paying a more modest premium. The problem is that a probability is rarely objective and individual, but experience-dependent on some classification. UK insurers rely on flood reports and surveys that are based on postcodes. Thus ‘properties like yours have a 0.5% risk of flood in any one year’ would really mean that some properties with your postcode have flooded, or have been assessed at being at risk from flooding. But, if like my in-laws your property is at the higher end of the postcode, this may not be at all appropriate. Just because the insurance company thinks that your risk of flooding is 0.5% does not mean that you should think the same.

Even from the insurance company’s perspective, the calculation is wrong. It would be correct if all post-codes were homogenous in terms of risk, but that clearly isn’t so. If a property hasn’t had a flood for 20 years then it is more likely to be at less risk (e.g., on higher ground) than those that have flooded. Hence its risk of flooding in the future is reduced. Taking more extreme figures, suppose that there are 2 houses on a river bank that flood every year and 8 on a hill that never flood. The chance of a house selected at random being flooded at some time in any long period is just 20%. And if you know that your house is an hill, then for you the probability may be 0%. In less extreme cases – typical of reality – the elementary formula also tends to overstate the risk. But the main point is that – contrary to the elementary theory – one shouldn’t just take probability estimates at face value. This could save you money!

See Also

Similar puzzles.

Dave Marsay

Disease

“You are suffering from a disease that, according to your manifest symptoms, is either A or B. For a variety of demographic reasons disease A happens to be nineteen times as common as B. The two diseases are equally fatal if untreated, but it is dangerous to combine the respectively appropriate treatments. Your physician orders a certain test which, through the operation of a fairly well understood causal process, always gives a unique diagnosis in such cases, and this diagnosis has been tried out on equal numbers of A- and B-patients and is known to be correct on 80% of those occasions. The tests report that you are suffering from disease B. Should you nevertheless opt for the treatment appropriate to A … ?”

My thoughts below …

.

.

.

.

.

.

.

.

If, following Good, we use

P(A|B:C) to denote the odds of A, conditional on B in the context C, Odds(A1/A2|B:C) to denote the odds P(A1|B:C)/P(A2|B:C), and LR(B|A1/A2:C) to denote the likelihood ratio, P(B|A1:C)/P(B|A2:C).

then we want

Odds(A/B | diagnosis of B : you), given
Odds(A/B : population) and
P(diagnosis of B | B : test), and similarly for A.

This looks like a job for Bayes’ rule! In Odds form this is

Odds(A1/A2|B:C) = LR(B|A1/A2:C).Odds(A1/A2:C).

If we ignore the dependence on context, this would yield

Odds(A/B | diagnosis of B ) = LR(diagnosis of B | A/B ).Odds(A/B).

But are we justified in ignoring the differences? For simplicity, suppose that the tests were conducted on a representative sample of the population, so that we have Odds(A/B | diagnosis of B : population), but still need Odds(A/B | diagnosis of B : you). According to Blackburn’s population indifference principle (PIP) you ‘should’ use the whole population statistics, but his reasons seem doubtful. Suppose that:

  • You thought yourself in every way typical of the population as a whole.
  • The prevalence of diseases among those you know was consistent with the whole population data.

Then PIP seems more reasonable. But if you are of a minority ethnicity – for example – with many relatives, neighbours and friends who share your distinguishing characteristic, then it might be more reasonable to use an informal estimate based on a more appropriate population, rather than a better quality estimate based on a less appropriate estimate. (This is a kind of converse to the availability heuristic.)

See Also

My notes on Cohen for a discussion of alternatives.

Other, similar, Puzzles.

My notes on probability.

Dave Marsay

Follow

Get every new post delivered to your Inbox.

Join 28 other followers