Applications of Statistics

Lars Syll has commented on a book by David Salsburg, criticising workaday applications of statistics. Lars has this quote:

Kolmogorov established the mathematical meaning of probability: Probability is a measure of sets in an abstract space of events.

This is not quite right.

  • Kolmogorov established a possible meaning, not ‘the’ meaning. (Actually Wittgenstein anticipated him.)
  • Even taking this theory, it is not clear why the space should be ‘measurable‘. More generally one has ‘upper’ and ‘lower’ measures, which need not be equal. One can extend the more familiar notions of probability, entropy, information and statistics to such measures. Such extended notions seem more credible.
  • In practice one often has some ‘given data’ which is at least slightly distant from the ‘real’ ‘events’ of interest. The data space is typically rather a rather tame ‘space’, so that a careful use of statistics is appropriate. But one still has the problem of ‘lifting’ the results to the ‘real events’.

These remarks seem to cover the criques of Syll and Salsburg, but are more nuanced. Statistical results, like any mathematics, need to be interpreted with care. But, depending on which of the above remarks apply, the results may be more or less easy to interpret: not all naive statistics are equally dubious!

Dave Marsay

AI pros and cons

Henry A. Kissinger, Eric Schmidt, Daniel Huttenlocher The Metamorphosis Atlantic August 2019.

AI will bring many wonders. It may also destabilize everything from nuclear détente to human friendships. We need to think much harder about how to adapt.

The authors are looking for comments. My initial reaction is here. I hope to say more. Meanwhile, I’d appreciate your reactions.

 

Dave Marsay

The limits of pragmatism

This is a personal attempt to identify and articulate a fruitful form of pragmatism, as distinct from what seems to me the many dangerous forms. My starting point is Wikipedia and my notion that the differences it notes can sometimes matter.

Doubt, like belief, requires justification. Genuine doubt irritates and inhibits, in the sense that belief is that upon which one is prepared to act.[2] It arises from confrontation with some specific recalcitrant matter of fact (which Dewey called a “situation”), which unsettles our belief in some specific proposition. Inquiry is then the rationally self-controlled process of attempting to return to a settled state of belief about the matter. Note that anti-skepticism is a reaction to modern academic skepticism in the wake of Descartes. The pragmatist insistence that all knowledge is tentative is quite congenial to the older skeptical tradition

My own contribution to things scientific has been on some very specific issues, but which I attempt to generalise:

  • It is sometimes seems much too late to wait to act on doubt for something that pragmatic folk recognize as a ‘specific recalcitrant matter of fact’. I would rather say (with the skeptics) that we should always be in some doubt, but that our actions require justification, and should only invest in relation to that justification. Requiring ‘facts’ seems too high a hurdle to act at all.
  • Psychologically, people do seek ‘settled states of belief’, but I would rather say (with the skeptics) that the degree of settledness ought to be only in so far as is justified. Relatively settled belief but not fundamentalist dogma!
  • It is often supposed that ‘facts’ and ‘beliefs’ should concern the ‘state’ of some supposed ‘real world’. There is some evidence that it is ‘better’ in some sense to think of the world as one in which certain processes are appropriate. In this case, as in category theory, the apparent state arises as a consequence of sufficient constraints on the processes. This can make an important difference when one considers uncertainties, but in ‘small worlds’ there are no such uncertainties.

It seems to me that the notion of ‘small worlds’ is helpful. A small world would be one which could be conceived of or ‘mentally modelled’. Pragmatists (of differing varieties) seem to believe that often we can conceive of a small world representation of the actual world, and act on that representation ‘as if’ the world were really small. So far, I find this plausible, even if not my own habit of thinking. The contentious point, I think, is that in every situation we should do our best to from a small world representation and then act as if it were true unless and until we are confronted with some ‘specific recalcitrant matter of fact’. This can be too late.

But let us take the notion of  a ‘small world’ as far as we can. It is accepted that the small world might be violated. If it could be violated as a consequence of something that we might inadvertently do then it hardly seems a ‘pragmatic’ notion in terms of ordinary usage, and might reasonably said to be dangerous in so far as it lulls us into a false sense of security.

One common interpretation of ‘pragmatism’ seems to be that we may as well act on our beliefs as there seems no alternative. But I shall refute this by presenting one. Another interpretation is that there is no ‘practical’ alternative’. That is to say, whatever we do could not affect the potential violation of the small world. But if this is the case it seems to me that there must be some insulation between ourselves and the small world. Thus the small world is actually embedded in some larger closed world. But do we just suppose that we are so insulated, or do we have some specific closed world in mind?

It seems to me that doubt is more justified the less our belief in insulation is justified. Even when we have specific insulation in mind, we surely need to keep an open mind and monitor the situation for any changes, or any reduction in justification for our belief.

From this, it seems to me that (as in my own work) what matters is not having some small world belief, but in taking a view on the insulations between what you seek to change and what you seek to rely on as unchanging. And from these identifying not only a single credible world in which to anchor one’s justifications for action, but in seeking out credible possible small worlds in the hope that at least one may remain credible as things proceed.

Dave Marsay

See also my earlier thoughts on pragmatism, from a different starting point.

Addendum: Locke anticipated my 3 bullet points above, by a few centuries. Pragmatists seem to argue that we don’t have to take some of Locke’s concerns too seriously. But maybe we should. It further occurs to me that there are often situations where in the short-run ‘pragmatism pays’, but in the long-run things can go increasingly awry. Locke offers an alternative to the familiar short-term utilitarianism that seems to make more sense. Whilst it may be beneficial to keep developing theories pragmatically, in the longer term one would do well to seek more logical (if less precise) theories from which one can develop pragmatic ‘beliefs’ that are not unduly affected by beliefs that may have been pragmatic in previous situations, but which no longer are. One might say that rather than stopping being pragmtic, one’s pragmatism should -from time to time – consider the potential long-run consequences, lest the long-run eventually burst upon one, creating a crisis and a need for a challenging paradigm shift.

An alternative is to recognise the issues arising from one’s current ‘pragmatic’ beliefs, and attempt to ‘regress to progress’. But this seems harder, and may be impossible under time presssure.

Which pragmatism as a guide to life?

Much debate on practical matters ends up in distracting metaphysics. If only we could all agree on what was ‘pragmatic’. My blog is mostly negative, in so far as it rubbishes various suggestions, but ‘the best is trhe enemy of the good’, and we do need to do something.

Unfortunately, different ‘schools’ start from a huge variety of different places, so it is difficult to compare and contrast approaches. But it is about time I had a go. (In part inspired by a recent public engagement talk on mathematics).

Suppose you have a method Π that you regard as pragmatic, in the sense that you can always act on it. To justify this, I think (like Popper) that you should have criteria , Γ, which if falsified would lead you to reconsider ∏ . So your pragmatic process is actually

If Γ then ∏ else reconsider.

But this is hardly reasonable if we try to arrange things so that Γ will never appear to be falsified. So an improvement is:

Spend some effort in monitoring Γ. If it is not falsified then ∏.

In practice if one thinks that Γ can be relied on, one may not think it worth spending much effort on checking it, but surely one should at least be open to suggestions that it could be wrong. The proper balance between monitoring Γ and acting on ∏ seems  impossible to establish with any confidence, but ignoring all evidence against Γ seems risky, to say the least.

Some argue that if you have no alternative to ∏  then it is pointless considering Γ. This may be a  reasonable argument when applied to concepts, but not to actions in the real world. Whatever evidence we may have for ∏ it will never uniquely prove it. It may be that it rules out all the alternatives that we have thought of, or which we consider credible or otherwise acceptable, but we should think again. Logically, there are always alternatives.

The above clearly applies to science. No theory is ever regarded as asolute and for ever. Scientists make their careers by identifying alternative theories to explain the experimental results and then devising new experiments to try to falsify the current theory. This process could only ever end when we were all sure that we had performed every possible experiment using every possible means in every possible circumstance, which implies the end of evolution and inventiveness. We aren’t there yet.

My proposal, then, is that very generally (not just in science) we ought to expect any ‘pragmatic’ ∏  to include a specific ‘caveat’, Γ(∏). If it doesn’t, we ought to develop one. This caveat will include its own rules for falsifying, tests, and we ought to regard more severe tests (in some sense) to be better. We then seek to develop alternatives that might be less precise (and hence less ‘useful’) than ∏ but which might survive falsification of ∏.

Much of my blog has some ideas on how to do this in particular cases: a work in progress. But an example may appeal:

Faced with what looks like a coin being tossed we might act ‘as if’ we believe it to be fair and to correspond to the axioms of mathematical probability theory, but keep an eye out for evidence to the contrary. Perhaps we inspect it and toss it a few times. Perhaps we watch whoever tosses it carefully. We do what we can, but still if someone tosses it and over a very large runs gets an excess of ‘Heads’ that our statistical friends tell us is hugely significant, we may be suspicious and reconsider

In this case we may decline from gambling on coin tosses even if we lack a specific ‘theory of the coin’, but it might be better if we had an alternative theory. Perhaps it is an ingenious fake coin? Perhaps the person tossing it has a cunning technique to bias it? Perhaps the person tossing it is a magician, and is actually faking the results?

This seems to me a like a good approach, surely better than acting ‘pragmatically’ but without such falsifying criteria. Can it be improved upon? (Suggestions please!)

Dave Marsay

The search for MH370: uncertainty

There is an interesting podcast about the search for MH370 by a former colleague. I think it illustrates in a relatively accessible form some aspects of uncertainty.

According to the familiar theory, if one has an initial probability distribution over the globe for the location of MH370’s flight recorder, say, then one can update it using Bayes’ rule to get a refined distribution. Conventionally, one should search where there is a higher probability density (all else being equal). But in this case it is fairly obvious that there is no principled way of deriving an initial distribution, and even Bayes’ rule is problematic. Conventionally, one should do the best one can, and search accordingly.

The podcaster (Simon) gives examples of some hypotheses (such as the pilot being well, well-motivated and unhindered throughout) for which the probabilistic approach is more reasonable. One can then split one’s effort over such credible hypotheses, not ruled out by evidence.

A conventional probabilist would note that any ‘rational’ search would be equivalent to some initial probability distribution over hypotheses, and hence some overall distribution. This may be so, but it is clear from Simon’s account that this would hardly be helpful.

I have been involved in similar situations, and have found it easier to explain the issues to non-mathematicians when there is some severe resource constraint, such as time. For example, we are looking for a person. The conventional approach is to maximise our estimated probability of finding them based on our estimated probabilities of them having acted in various ways (e.g., run for it, hunkered down). An alternative is to consider the ways they may ‘reasonably’ be thought to have acted and then to seek to maximize the worst case probability of finding them. Then again, we may have a ranking of ways that they may have acted, and seek to maximize the number of ways for which the probability of our success exceeds some acceptable amount (e.g. 90%). The key point here is that there are many reasonable objectives one might have, for only one of which the conventional assumptions are valid. The relevant mathematics does still apply, though!

Dave Marsay

More to Uncertainty than Probability!

I have just had published my paper, based on the discussion paper referred to in a previous post. In Facebook it is described as:

An understanding of Keynesian uncertainties can be relevant to many contemporary challenges. Keynes was arguably the first person to put probability theory on a sound mathematical footing. …

So it is not just for economists. I could be tempted to discuss the wider implications.

Comments are welcome here, at the publisher’s web site or on Facebook. I’m told that it is also discussed on Google+, Twitter and LinkedIn, but I couldn’t find it – maybe I’ll try again later.

(Actually, my paper was published jan 2016,but somehow this request for comments got stuck in a limbo somewhere. Better late than never?)

Which rationality?

We often suppose that rationality is ‘a good thing’, but is it always?

Rationality is variously defined as being in accord with reason, logic or ‘the facts’. Here ‘reason’ may mean one’s espoused or actual reasons, or it may mean in accord with some external standards. Thus in its broadest interpretation, it seems that anything that has a reason for being the way that it is may be considered broadly rational. But the notion of rationality derives from ‘reason’, one aspect of which is ‘sound judgement, good sense’. This suggests some external standard.

If we use the term ‘simple’ to denote a situation in which there are definite ‘objective’ standards of soundness and goodness, then rationality in simple situations is behaviour that accords with those standards. Philosophers can argue endlessly about whether any such situations exist, so it seems sensible to define rationality more generally as being relative to some set of standards. The question then being: What standards?

My natural inclination as a mathematician is that those standards should always include the best relevant logics, including mathematics. Yet I have witnessed many occasions on which the use of mathematics has tended to promote disasters, and the advocates of such approaches (apart from those few who think like me) have seemed culpable. I have a great deal of respect and sympathy for the view that mathematics is harmful in complex situations. Yet intellectually it seems quite wrong, and I cannot accept it.

In each case there seems to be some specific failing, which many of my colleagues have attributed to some human factor, such as hubris or the need to keep a job or preserve an institution. But the perpetrators do not seem to me to be much different from the rest of us, and I have long thought that there is some more fundamental common standard that is incompatible with the use of reason. The financial crises of 2007/8/9 are cases where it is hard to believe that most of those pushing the ‘mathematical’ view that turned out to be harmful were either irrational or rationally harmful.

Here I want to suggest a possible explanation.

From a theoretical perspective, there are no given ‘facts’, ‘logics’ or ‘reasons’ that we can rely on.This certainly seems to be true of finance and economics. For example, in economics the models used may be mathematical and in this sense beyond criticism, but the issue of their relevance to a particular situation is never purely logical, and ought to be questioned. Yet it seems that many institutions, including businesses,  rely on having absolute beliefs: questioning them would be wasteful in the short-run. So individual actors tend not only to be rational, but also to be narrowly rational ‘in the short run’, which normally goes with acting ‘as if’ it had narrow facts.

For example, it seems to me to be  a fact that according to the best contemporary scientific theories, the earth is not stationary. It is generally expedient to for me to act ‘as if’ I knew that the earth moved. But unless we can be absolutely sure that the earth moves, the tendency to suppose that it is a fact that the earth moves could be dangerous. (You could try substituting other facts, such as that economies always tend to a healthy equilibrium.)

In a healthy society there would be a competition of ideas,  such that society as a whole could be said to be being more broadly rational, even while its actors were being only narrowly rational. For example, a science would consist of various schools, each of which would be developing its own theories, consistent with the facts, which between them would be exploring and developing the space of all such credible theories. At a practical level, an engineer would appreciate the difference between building a bridge based on a theory that had been tested on similar bridges, and building a novel type of bridge where the existing heuristics could not be relied upon.

I do not think that human society as a whole is healthy in this sense. Why not? In evolutionary theory separate niches, such as islands, promote the development of healthy diversity. Perhaps the rise of global communications and trade, and even the spread of the use of English, is eliminating the niches in which ideas can be explored and so is having a long-run negative impact that needs to be compensated for?

Thus I think we need to distinguish between short and long-run rationalities, and to understand and foster both. It seems to me that most of the time, for most areas of life, short-run rationality is adequate, and it is this that is familiar. But this needs to be accompanied by an understanding of the long-run issues, and an appropriate balance achieved. Perhaps too much (short-run) rationality can be harmful (in the long-run). And not only in economies.

Dave Marsay

Decision-making under uncertainty: ‘after Keynes’

I have a new discussion paper. I am happy to take comments here, on LinkedIn, at the more formal Economics e-journal site or by email (if you have it!), but wish to record substantive comments on the journal site while continuing to build up a site of whatever any school of thought may think is relevant, with my comments, here.

Please do comment somewhere.

Clarifications

I refer to Keynes’ ‘weights of argument’ mostly as something to be taken into account in addition to probability. For example, if one has two urns each with a mix of 100 otherwise identical black and white balls, where the first urn is known to have equal number of each colour, but the mix for the other urn is unknown, then conventionally one has equal probability of drawing a black ball form each urn, but the weight of argument is greater for the first than the second.

Keynes does fully develop his notion of weights and it seems not to be well understood, and I wanted my overview of Keynes’ views to be non-contentious. But from some off-line comments I should clarify.

Ch. VI para 8 is worth reading, followed by Ch. III para 8. Whatever the weight may be, it is ‘strengthened by’:

  • Being more numerous.
  • Having been obtained with a greater variety of conditions.
  • Concerning a greater generalisation.

Keynes argues that this weight cannot be reduced to a single number, and so weights can be incomparable. He uses the term ‘strength’ to indicate that something is increased while recognizing that it may not be measurable. This can be confusing, as in Ch. III para 7, where he refers to ‘the strength of argument’. In simple cases this would just be the probability, not to be confused with the weight.

It seems to me that Keynes’ concerns relate to Mayo’s:

Severity Principle: Data x provides a good evidence for hypothesis H if and only if x results from a test procedure T which, taken as a whole, constitutes H having passed a severe test – that is, a procedure which would have, with very high probability, uncovered the discrepancies from H, and yet no such error is detected.

In cases where one has performed a test, severity seems to roughly correspond to have a strong weight, at least in simpler cases. Keynes’ notion applies more broadly. Currently, it seems to me, care needs taking in applying either to particular cases. But that is no reason to ignore them.

 

 

Dave Marsay

Mathiness

(Pseudo-)Mathiness

Paul Romer has recently attracted attention by his criticism of what he terms ‘mathiness’ in economic growth theory. As a mathematician, I would have thought that economics could benefit from more mathiness, not less. But what he seems to be denigrating is not mathematics as I understand it, but what Keynes called ‘pseudomathematics’. In his main example the problem is not inappropriate mathematics as such, but a succession of symbols masquerading as mathematics, which Paul unmasks using – mathematics. Thus, it seems to me the paper that he is criticising would have benefited from more (genuine) mathiness and less pseudomathiness.

I do agree with Paul, in effect, that bad (pseudo) mathematics has been crowding out the good, and that this should be resisted and reversed. But, as a mathematician, I guess I would think that.

I also agree with Paul that:

We will make faster scientific progress if we can continue to rely on the clarity and precision that math brings to our shared vocabulary, and if, in our analysis of data and observations, we keep using and refining the powerful abstractions that mathematical theory highlights … .

But more broadly some of Paul’s remarks suggest to me that we should be much clearer about the general theoretical stance and the role of mathematics within it. Even if an economics paper makes proper use of some proper mathematics, this only ever goes so far in supporting economic conclusions, and I have the impression that Paul is expecting too much, such that any attempt to fill his requirement with mathematics would necessarily be pseudo-mathematics. It seems to me that economics can never be a science like the hard sciences, and as such it needs to develop an appropriate logical framework. This would be genuinely mathsy but not entirely mathematical. I have similar views about other disciplines, but the need is perhaps greatest for economics.

Media

Bloomberg (and others) agree that (pseudo)-mathiness is rife in macro-economics and that (perhaps in consequence) there has been a shift away from theory to (naïve) empiricism.

Tim Harford, in the ft, discusses the related misuse of statistics.

… the antidote to mathiness isn’t to stop using mathematics. It is to use better maths. … Statistical claims should be robust, match everyday language as much as possible, and be transparent about methods.

… Mathematics offers precision that English cannot. But it also offers a cloak for the muddle-headed and the unscrupulous. There is a profound difference between good maths and bad maths, between careful statistics and junk statistics. Alas, on the surface, the good and the bad can look very much the same.

Thus, contrary to what is happening, we might look for a reform and reinvigoration of theory, particularly macroeconomic.

Addendum

Romer adds an analogy between his mathiness, which has actual formulae and a description on the one hand, and computer code, which typically has both the actual code and some comments. Romer’s mathiness is like when the code is obscure and the comments are wrong, as when the code does a bubble sort but the comment says it does a prime number sieve. He gives the impression that in economics this may often be deliberate. But a similar phenomenon is when the coder made the comment in good faith, so that the code appears to do what it says in the comment, but that there is some subtle, technical, flaw. A form of pseudo-mathiness is when one is heedless to such a possibility. The cure is more genuine mathiness. Even in computer code, it is possible to write code that is more or less obscure, and the less obscure code is typically more reliable. Similarly in economics, it would be better for economists to use mathematics that is within their competence, and to strive to make it clear. Maybe the word Romer is looking for is obscurantism?

Dave Marsay 

Artificial Intelligence?

The subject of ‘Artificial Intelligence’ (AI) has long provided ample scope for long and inconclusive debates. Wikipedia seems to have settled on a view, that we may take as straw-man:

Every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it. [Dartmouth Conference, 1956] The appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds. [John Searle’s straw-man hypothesis]

Readers of my blog will realise that I agree with Searle that his hypothesis is wrong, but for different reasons. It seems to me that mainstream AI (mAI) is about being able to take instruction. This is a part of learning, but by no means all. Thus – I claim – mAI is about a sub-set of intelligence. In many organisational settings it may be that sub-set which the organisation values. It may even be that an AI that ‘thought for itself’ would be a danger. For example, in old discussions about whether or not some type of AI could ever act as a G.P. (General Practitioner – first line doctor) the underlying issue has been whether G.P.s ‘should’ think for themselves, or just apply their trained responses. My own experience is that sometimes G.P.s doubt the applicability of what they have been taught, and that sometimes this is ‘a good thing’. In effect, we sometimes want to train people, or otherwise arrange for them to react in predictable ways, as if they were machines. mAI can create better machines, and thus has many key roles to play. But between mAI and ‘superhuman intelligence’  there seems to be an important gap: the kind of intelligence that makes us human. Can machines display such intelligence? (Can people, in organisations that treat them like machines?)

One successful mainstream approach to AI is to work with probabilities, such a P(A|B) (‘the probability of A given B’), making extensive use of Bayes’ rule, and such an approach is sometimes thought to be ‘logical’, ‘mathematical, ‘statistical’ and ‘scientific’. But, mathematically, we can generalise the approach by taking account of some context, C, using Jack Good’s notation P(A|B:C) (‘the probability of A given B, in the context C’). AI that is explicitly or implicitly statistical is more successful when it operates within a definite fixed context, C, for which the appropriate probabilities are (at least approximately) well-defined and stable. For example, training within an organisation will typically seek to enable staff (or machines) to characterise their job sufficiently well for it to become routine. In practice ‘AI’-based machines often show a little intelligence beyond that described above: they will monitor the situation and ‘raise an exception’ when the situation is too far outside what it ‘expects’. But this just points to the need for a superior intelligence to resolve the situation. Here I present some thoughts.

When we state ‘P(A|B)=p’ we are often not just asserting the probability relationship: it is usually implicit that ‘B’ is the appropriate condition to consider if we are interested in ‘A’. Contemporary mAI usually takes the conditions a given, and computes ‘target’ probabilities from given probabilities. Whilst this requires a kind of intelligence, it seems to me that humans will sometimes also revise the conditions being considered, and this requires a different type of intelligence (not just the ability to apply Bayes’ rule). For example, astronomers who refine the value of relevant parameters are displaying some intelligence and are ‘doing science’, but those first in the field, who determined which parameters are relevant employed a different kind of intelligence and were doing a different kind of science. What we need, at least, is an appropriate way of interpreting and computing ‘probability’ to support this enhanced intelligence.

The notions of Whitehead, Keynes, Russell, Turing and Good seem to me a good start, albeit they need explaining better – hence this blog. Maybe an example is economics. The notion of probability routinely used would be appropriate if we were certain about some fundamental assumptions. But are we? At least we should realise that it is not logical to attempt to justify those assumptions by reasoning using concepts that implicitly rely on them.

Dave Marsay

Distilling the Science from the Art

Geoff Evatt (U o Manchester, UK) gave a ‘Mathematics in the Workplace’ talk at the recent Manchester Festival of Mathematics and its Applications, printed in the Oct 2014 Mathematics Today.

He showed how the Mathematical modeller could turn their hand to diverse subjects of financial regulation and … .

He is critical of the view that ‘Mathematical Modelling is like an Art’ and advocates the prescriptive teaching of best-practice. His main motivation seems to be to attract more students and the up-take by industry (etc).

This … will be achieved by academics from a variety of universities agreeing in what is ‘best practice’ in teaching modelling is … .

Comments

Taking the title, I accept that the term ‘art’ may be misleading, but I am not convinced that there is much science in, for example, finance, or that those funding the mathematics really care, so the term ‘science’ could be equally misleading and more dangerous. I would say that mathematical modelling is often a craft. Where it is part of a proper scientific endeavour, I would think that this would be because of the domain experts and ought to be certified from a scientific rather than mathematical point of view. To me ‘best practice’ is to work closely with domain experts, to give them what they need, and to make sure that they understand what they do – and don’t have. It is good to seek to be scientific and objective, but not to misrepresent what has actually been achieved.

In the run-up to the financial crash best practice included characterising mathematical modelling in this area as an ‘art’ and not a science, to prevent financiers and politicians from thinking that the ‘mathematical’ nature of the models somehow lent them the same credibility normally accorded to mathematics. A key part of the financial problem was that this was not well-enough understood.

A key part of economics is the concept of ‘uncertainty’. The classical mathematical models did not model uncertainty beyond mere probability, possibly because was not covered by contemporary mainstream courses.

Best practice would include ensuring that the mathematics used was appropriate to the domain, or at least in explaining any short-falls. I think that this requires more development than Evatt supposes. I also think that one would need to go beyond academics, to include people who understand the issues involved.

Dave Marsay

What should replace utility maximization in economics?

Mainstream economics has been based on the idea of people producing and trading in order to maximize their utility, which depends on their assigning values and conditional  probabilities to outcomes. Thus, in particular, mainstream economics implies that people do best by assigning probabilities to possible outcomes, even when there seems no sensible way to do this (such as when considering a possible crash). Ken Arrow has asked, if one rejects utility maximization, what should one replace it with?

The assumption here seems to be that it is better to have a wrong theory than to have no theory. The fear seems to be that economies would grind to a holt unless they were sanctioned by some theory – even a wrong one. But this fear seems at odds with another common view, that economies are driven by businesses, which are driven by ‘pragmatic’ men. It might be that without the endorsement of some (wrong) theory some practices, such as the development of novel technical instruments and the use of large leverages, would be curtailed. But would this be a bad thing?

Nonetheless, Arrow’s challenge deserves a response.

There are many variations in detail of utility maximization theories. Suppose we identity ‘utility maximization’ as a possible heuristic, then utility maximization theory claims that people use some specific heuristics, so an obvious alternative is to consider a wider  range. The implicit idea behind utility maximization theory seems to be under a competitive regime resembling evolution, the evolutionary stable strategies (‘the good ones’) do maximize some utility function, so that in time utility maximizers ought to get to dominate economies. (Maybe poor people do not maximize any utility, but they – supposedly – have relatively little influence on economies.) But this idea is hardly credible. If – as seems to be the case – economies have significant ‘Black Swans’ (low probability high impact events) then utility maximizers  who ignore the possibility of a Black Swan (such as a crash) will do better in the short-term, and so the economy will become dominated by people with the wrong utilities. People with the right utilities would do better in the long run, but have two problems: they need to survive the short-term and they need to estimate the probability of the Black Swan. No method has been suggested for doing this. An alternative is to take account of some notional utility but also take account of any other factors that seem relevant.

For example, when driving a hire-car along a windy road with a sheer drop I ‘should’ adjust my speed to trade time of arrival against risk of death or injury. But usually I simply reduce my speed to the point where the risk is slight, and accept the consequential delay. These are qualitative judgements, not arithmetic trade-offs. Similarly an individual might limit their at-risk investments (e.g. stocks) so that a reasonable fall (e.g. 25%) could be tolerated, rather than try to keep track of all the possible things that could go wrong (such as terrorists stealing a US Minuteman) and their likely impact.

More generally, we could suppose that people act according to their own heuristics, and that there are competitive pressures on heuristics, but not that utility maximization is necessarily ‘best’ or even that a healthy economy relies on most people having similar heuristics, or that there is some stable set of ‘good’ heuristics. All these questions (and possibly more) could be left open for study and debate. As a mathematician it seems to me that decision-making involves ideas, and that ideas are never unique or final, so that novel heuristics could arise and be successful from time to time. Or at least, the contrary would require an explanation. In terms of game theory, the conventional theory seems to presuppose a fixed single-level game, whereas – like much else – economies seem to have scope for changing the game and even for creating higher-level games, without limit. In this case, the strategies must surely change and are created rather than drawn from a fixed set?

See Also

Some evidence against utility maximization. (Arrow’s response prompted this post).

My blog on reasoning under uncertainty with application to economics.

Dave Marsay

Flood Risk Puzzle

This is a ‘Natural Hazards problem’ that has been used to explore people’s understanding of risk. As usual I question the supposedly ‘mathematical’ answer.

 Suppose that the probability that your house will be hit one or more times by the natural hazard during an exposure period of one year is .005. That is, if 1000 homes like yours were exposed to the natural hazard for one year, 5 of the homes would be damaged. Please estimate the probability that your home would avoid being hit by the natural hazard if exposed to the hazard for a period of 5/10/25/50 years.

You might like to ponder it for yourself.

In elementary probability theory the appropriate formula is 1-(1-p)n, where p=0.005 is the probability for one year and n is the number of years. But is this an appropriate calculation?

My in-laws were being charged a high property insurance premium because homes ‘like theirs’ were liable to flooding. They appealed, and are now paying a more modest premium. The problem is that a probability is rarely objective and individual, but experience-dependent on some classification. UK insurers rely on flood reports and surveys that are based on postcodes. Thus ‘properties like yours have a 0.5% risk of flood in any one year’ would really mean that some properties with your postcode have flooded, or have been assessed at being at risk from flooding. But, if like my in-laws your property is at the higher end of the postcode, this may not be at all appropriate. Just because the insurance company thinks that your risk of flooding is 0.5% does not mean that you should think the same.

Even from the insurance company’s perspective, the calculation is wrong. It would be correct if all post-codes were homogenous in terms of risk, but that clearly isn’t so. If a property hasn’t had a flood for 20 years then it is more likely to be at less risk (e.g., on higher ground) than those that have flooded. Hence its risk of flooding in the future is reduced. Taking more extreme figures, suppose that there are 2 houses on a river bank that flood every year and 8 on a hill that never flood. The chance of a house selected at random being flooded at some time in any long period is just 20%. And if you know that your house is an hill, then for you the probability may be 0%. In less extreme cases – typical of reality – the elementary formula also tends to overstate the risk. But the main point is that – contrary to the elementary theory – one shouldn’t just take probability estimates at face value. This could save you money!

See Also

Similar puzzles.

Dave Marsay

Disease

“You are suffering from a disease that, according to your manifest symptoms, is either A or B. For a variety of demographic reasons disease A happens to be nineteen times as common as B. The two diseases are equally fatal if untreated, but it is dangerous to combine the respectively appropriate treatments. Your physician orders a certain test which, through the operation of a fairly well understood causal process, always gives a unique diagnosis in such cases, and this diagnosis has been tried out on equal numbers of A- and B-patients and is known to be correct on 80% of those occasions. The tests report that you are suffering from disease B. Should you nevertheless opt for the treatment appropriate to A … ?”

My thoughts below …

.

.

.

.

.

.

.

.

If, following Good, we use

P(A|B:C) to denote the odds of A, conditional on B in the context C, Odds(A1/A2|B:C) to denote the odds P(A1|B:C)/P(A2|B:C), and LR(B|A1/A2:C) to denote the likelihood ratio, P(B|A1:C)/P(B|A2:C).

then we want

Odds(A/B | diagnosis of B : you), given
Odds(A/B : population) and
P(diagnosis of B | B : test), and similarly for A.

This looks like a job for Bayes’ rule! In Odds form this is

Odds(A1/A2|B:C) = LR(B|A1/A2:C).Odds(A1/A2:C).

If we ignore the dependence on context, this would yield

Odds(A/B | diagnosis of B ) = LR(diagnosis of B | A/B ).Odds(A/B).

But are we justified in ignoring the differences? For simplicity, suppose that the tests were conducted on a representative sample of the population, so that we have Odds(A/B | diagnosis of B : population), but still need Odds(A/B | diagnosis of B : you). According to Blackburn’s population indifference principle (PIP) you ‘should’ use the whole population statistics, but his reasons seem doubtful. Suppose that:

  • You thought yourself in every way typical of the population as a whole.
  • The prevalence of diseases among those you know was consistent with the whole population data.

Then PIP seems more reasonable. But if you are of a minority ethnicity – for example – with many relatives, neighbours and friends who share your distinguishing characteristic, then it might be more reasonable to use an informal estimate based on a more appropriate population, rather than a better quality estimate based on a less appropriate estimate. (This is a kind of converse to the availability heuristic.)

See Also

My notes on Cohen for a discussion of alternatives.

Other, similar, Puzzles.

My notes on probability.

Dave Marsay

Cab accident

“In a certain town blue and green cabs operate in a ratio of 85 to 15, respectively. A witness identifies a cab in a crash as green, and the court is told [based on a test] that in the relevant light conditions he can distinguish blue cabs from green ones in 80% of cases. [What] is the probability (expressed as a percentage) that the cab involved in the accident was blue?” (See my notes on Cohen for a discussion of alternatives.)

For bonus points …. if you were involved , what questions might you reasonably ask before estimating the required percentage? Does your first answer imply some assumptions about the answers, and are they reasonable?

My thoughts below:

.

.

.

.

.

.

If, following Good, we use

P(A|B:C) to denote the odds of A, conditional on B in the context C,
Odds(A1/A2|B:C) to denote the odds P(A1|B:C)/P(A2|B:C), and
LR(B|A1/A2:C) to denote the likelihood ratio, P(B|A1:C)/P(B|A2:C).

Then we want P(blue| witness: accident), which can be derived by normalisation from Odds(blue/green| witness : accident).
We have Odds(blue/green: city) and the statement that the witness “can distinguish blue cabs from green ones in 80% of cases”.

Let us suppose (as I think is the intention) that this means that we know Odds(witness| blue/green: test) under the test conditions. This looks like a job for Bayes’ rule! In Odds form this is

Odds(A1/A2|B:C) = LR(B|A1/A2:C).Odds(A1/A2:C),

as can be verified from the identity P(A|B:C) = P(A&B:C)/P(B:C) whenever P(B:C)≠0.

If we ignore the contexts, this would yield:

Odds(blue/green| witness) = LR(witness| blue/green).Odds(blue/green),

as required. But this would only be valid if the context made no difference. For example, suppose that:

  • Green cabs have many more accidents than blue ones.
  • The accident was in an area where green cabs were more common.
  •  The witness knew that blue cabs were much more common than green and yet was still confident that it was a green cab.

In each case, one would wish to re-assess the required odds. Would it be reasonable to assume that none of the above applied, if one didn’t ask?

See Also

Other Puzzles.

My notes on probability.

Dave Marsay

Quantum Minds

A New Scientist Cover Story (No. 2828 3 Sept 2011) opines that:

‘The fuzziness and weird logic of the way particles behave applies surprisingly well to how human thinks’. (banner, p34)

It starts:

‘The quantum world defies the rules of ordinary logic.’

The first two examples are The infamous two-slit experiment and an experiment by Tversky and Shamir supposedly showing violation of the ‘sure thing principle’. But do they?

Saving classical logic

According to George Boole (Laws of thought), when a series of assumptions and applications of logic leads to a falsehood I must abandon one of the assumptions of one of the rules of inference, but I can ‘save’ whichever one I am most wedded to. So to save ‘ordinary logic’ it suffices to identify a dodgy assumption.

Two-slits experiment

The article says of the two-slits experiment:

‘… the pattern you should get – ordinary physics and logic would suggest – should be ..’

There is a missing factor here: the classical (Bayesian) assumptions about ‘how probabilities work’. Thus I could save ‘ordinary logic’ by abandoning common-sense probability theory.

Actually, there is a more obvious culprit. As Kant pointed out the assumption that the world is composed of objects with attributes and having relationships with each other belongs to common-sense physics, not logic. For example, two isolated individuals may behave like objects but when they come into communion the sum may be more than the sum of the parts. Looking at the two-slit experiment this way, the stuff that we regard as a particle seem isolated and hence object-like until it ‘comes into communion with’ the apparatus, when the whole may be un-object-like, but then a new steady-state ’emerges’, which is object-like and which we regard as a particle. The experiment is telling us something about the nature of the communion. Prigogine has a mathematization of this.

Thus one can abandon the common-sense assumption that ‘a communion is nothing but the sum of objects’, thus saving classical logic.

Sure Thing Principle

An example is given (pg 36). That appears to violate Savage’s sure-thing principle and hence ‘classical logic’. But, as above, we might prefer to abandon out probability theory rather than our logic. But there are plenty of alternatives.

The sure-thing principle applies to ‘economic man’, who has some unusual values. For example, if he values a winter sun holiday at $500 and a skiing holiday at $500 then he ‘should’ be happy to pay $500 for a holiday in which he only finds out which it is when he gets there. The assumptions of classical economic man only seem to apply to people with lots of spare money and are used to gambling with it. Perhaps the experimental subjects were different?

The details of the experiment as reported also repay attention. A gamble with an even chance of winning $200 or losing $100 is available. Experimental subjects all had a first gamble. In case A subjects were told they had won. In case B they were told they had lost. In case C they were not told. They were all invited to gamble again.

Most subjects (69%) wanted to gamble again in case A. This seems reasonable as over the two gambles they were guaranteed a gain of $100. Fewer subjects (59%) wanted to gamble again in case B. This seems reasonable, as they risked a $200 loss overall. Least subjects  (36%) wanted to gamble again in case C. This seems to violate the sure-thing principle, which (according to the article) says that anyone who gambles in both the first two cases should gamble in the third. But from the figures above we can only deduce that – if they are representative – then at least 28% (i.e. 100%-(100%-69%)+(100%-59%)) would gamble in both cases. But 36% gambled in case C, so the data does not imply that anyone would gamble for A and B but not C.

If one chooses a person at random, then the probability that they gambled again in both cases A and B is between 28% and 100%. The convention in ‘classical’ probability theory is to split the difference (a kind of principle of indifference) yielding 64% (as in the article). A possible explanation for the dearth of such subjects is that they were not wealthy (so having non-linear utilities in the region of $100s) and that those who couldn’t afford to lose $100 had good used in mind for $200, preferring a certain win of $200 to an evens chance of winning $400 or only $100. This seems reasonable.

Others’ criticisms here. See also some notes on uncertainty and probability.

Dave Marsay