Smith’s Consistency

Cedric A. B. Smith Consistency in Statistical Inference and Decision Journal of the Royal Statistical Society. Series B (Methodological), Vol. 23, No. 1(1961), pp. 1-37.

It is suggested that the strength of a person’s beliefs may be tested by finding at what odds he is prepared to bet on them. This leads to a system of numerical “medial personal probabilities” obeying the classical laws of probability. However, these do not have precisely defined values, but are contained within specified intervals. … (Summary)

1. INTRODUCTION

We suppose that an investigator has data consisting of random samples from certain distributions

2. EVENTS AND PROPOSITIONS

To avoid possible difficulties, we restrict attention to “verifiable” events. That is to say, if an event B is being considered (say, that Cambridge wins the 1961 Boat Race) there will be evidence available (if desired) by some fixed date which all concerned agree will settle the question whether B has occurred or not. …

5. COMPARISON OF FREQUENCY AND PERSONAL ODDS

If, therefore, a bet is only one of a long sequence it seems reasonable that Bob should be willing to offer any odds w < λ, but not ≥ λ, … This argument is less cogent (and possibly untrue) when the bet is a single one, and not one of a series … .

10. THE DIVISION THEOREM 

… Suppose that we know nothing definite about a certain man, other than that he is European. What is the probability that he has blood group AB? Among Southern Irishmen the proportion of AB is about 2 per cent., and among the Tatars of Kazanj about 13 per cent.; in the remainder of Europe it lies between these values. It seems reasonable to say that the man has “frequency probability between ·02 and ·13” of being of group AB; and the ordinary calculus of probabilities would have to be modified in a fairly obvious way to accommodate such statements.

13. PROBABILITY CURRENCY

 … We have not discussed whether cases could arise in which he would be prepared to bet at a stake xl but not at a smaller one x2. However, such cases can occur; e.g. a hungry man with 2s. 6d. in his pocket in a town in which the cheapest lunch costs 3s. might well be prepared to make a bet on which he could win 6d., but no smaller bet (Chernoff and Moses, 1959). We have also supposed, implicitly that if Bob is prepared to make a bet on B against C or alternatively a bet on F against G, then he is prepared to make both bets simultaneously. This again is not necessarily true in all cases.
To avoid these difficulties it is helpful to use the following device, adapted from Savage (1954). Instead of presenting cash to Bob and Charles, the Umpire takes 1 kilogram of beeswax (of negligible value) and hides within it at random a very small but valuable diamond. He divides the wax into two parts, presenting one to each player, and instructs them to use it for stakes. After all bets have been settled, the wax is melted down and whoever has the diamond keeps it.

using … “probability currency” the acceptability of a bet depends only on the odds, w, and not on the stake x provided that is not too large.

14. CURRENCY EXCHANGE

… the correspondence between bets and beliefs fails when the attractiveness of the rewards is affected by the occurrence or non-occurrence of the events betted on: some people particularly appreciate the payments from the insurance company when they come as compensation for a wet holiday. If this is so, then in order to get the most accurate results from the theory we should choose as a reward R something whose desirability is independent of the events under consideration. With the really crucial events of life this could be difficult: but most statistical problems are of no great personal importance, and any reward R not directly bound up with the events considered should be suitable.

Thus it is assumed that the events being bet on have no value in themselves.

18. GENERAL DECISIONS AND UTILITY

    This traditional approach will fail if the value of a consequence is not measurable numerically, or if the decision is not looked upon as one of a long series, so that the expected value is not necessarily relevant, or if there do not exist initial probabilities in the frequency sense

… We can imagine Bob surrounded by self-assured friends, Sam, Steve, etc. (corresponding to the possible vectors t). He prefers mr to m, if and only if all his friends agree on this. When he has to choose one of the strategies from X, he asks for and abides by the advice of just one of his friend.

19. DECISIONS BASED ON STATISTICAL EVIDENCE

… a better mixed strategy has a greater expected utility. In general, however, the utility function is not uniquely fixed.

For each possible ‘medial’ probability function there is a corresponding decision, and hence there may be a need to ‘hedge’.

DISCUSSION ON DR SMITH’S PAPER

There is some discussion on stopping rules and the likelihood principle (originally due to the socialist Barnard). The general suggestion is that stopping rules do not matter unless they are ‘obviously’ biased, as they may be in experimental treatments.

Professor G. A. BARNARD

… although Dr Smith’s Bob and Charles are less unreal than the individuals discussed by other writers, they nevertheless seem to remain so remote from real human beings that they should not be thought of in any sense as “reasonable and consistent men”, but rather as “stat rats”, the term invented in U.S.A. for creatures of this kind.

… The late Christopher Caudwell (The Crisis in Physics, 1939) once suggested that the Newtonian universe of atoms moving under a law of inverse square, but otherwise self-contained, bore a suggestive resemblance to the social conception of emergent capitalist society as a collection of independent buyers and sellers. I feel that the current expositions of subjective probability owe a good deal to sociological influences of this sort.
In restricting himself to propositions which can be verified in principle by a fixed date, Dr Smith seems to be dividing up the process of gaining scientific knowledge into a series of self-contained inferences each of which is eventually proved right or wrong. That this cannot in fact be done was recognized by Savage, who saw that a proper analysis of life’s activity would require a babe in arms to assess all the strategies of life available to it, to make a choice while still in the cradle, and then spend the rest of its life following this through.

… If we think of Dr Smith’s construction as a contribution to the logic of a modest betting man, then we shall not be misled by it. And of course we all do go in for this sort of thing from time to time.

Dr D. J. FINNEY  

(Of the blood group example) … if a haphazard or wholly unknown method of selection has been used, in what way is Dr Smith’s argument appropriate?

Mr F. J. ANSCOMBE

… Dr Smith has studied in considerable detail the consequences of relaxing the postulate of ordered preferences, and he has exhibited his results in a form that closely parallels the existing theory. His work does not, I think, make previous work obsolete but shows it to us in a better perspective. We see now with greater clarity one of the respects in which the preceding theory was over-idealized. We can live more comfortably with the over-idealization for understanding more about how it may be corrected.

Dr SMITH

… replied briefly at the meeting and subsequently in writing as follows:

… Bob considers only his own advantage when coming to a decision; hence we might expect theory to fail when he is considered as a member of society, and not merely as an individual acting on his own. In a game with one or more intelligent opponents, it can be advantageous to behave inconsistently (von Neumann, 1928; Fisher, 1934; von Neumann and Morgenstern, 1944). Decisions by committees, companies, etc., will therefore not necessarily be covered by the theory and Professor Barnard’s remarks are relevant.

Comments

The big picture

Mr Anscombe’s view seem right, except that it seems to me that some who have built on this work have ignored the caveats. In particular, it appears not to be true that all decisions can be related to decisions about beeswax. Indeed, the notion of ‘probability currency’ seems carefully contrived to focus on a single aspect of uncertainty, namely ‘probability’. While the notion of probability intervals is usefully more general than conventional (precise) probability, Smith is not arguing that it is complete, referring us to von Neumann and Morgenstern for evidence of its inability to deal with competing intelligences. Indeed, the theory is only proposed for decisions ‘of no great personal importance’.

Caveats

In essence, it is mostly assumed that the situation is like roulette, where the result is of no significance other than through the bets. Further, this will be a long session and there is no risk of us meeting not being able to meet our obligations at the end of the night, so that utility maximization is the only consideration.

There are some extensions beyond this, but none of it applies to situations where the decision-maker is embroiled in the events.

Stopping rules

The material on stopping rules is inconclusive, but I note that the counter-example (medical trails) is very far from meeting Smith’s conditions.

Probabilities for sub-groups

Dr Finney draws attention to the fact that if we only have statistics for different countries and wish to estimate a probability for an individual from an unknown country Dr Smith’s approach is half-baked. A conventional Bayesian probability for Europe as a whole could be interpreted as the appropriate probability for a person drawn at random from Europe. To say that the probability could be anywhere between the extremes of the probabilities for the individual countries is informative, but not conclusive. Suppose, for example, that the propensities vary considerably between different ethnic groups, and the variation in country statistics is simply a reflection of this. Then the stated range seems too narrow and not very meaningful unless the basis for the estimate is clearly given.

If we think of a number of statisticians who have sliced the data in different ways, then each could give Bob a different probability interval. Bob might reasonably take the bounding interval, but what if that interval is narrow and yet he knows that there is a lot of other data that has not yet been taken account of? Should he gamble or confess his ignorance?

Coalitions

Von Nuemann and Morgenstern point out that if there are at least three interacting actors then ‘the game’ and hence the appropriate values and probabilities will depend on any coalitions. Thus if one should try to apply probabilistic methods to important decisions and should be aware that the methods only make sense ‘in the short run’, for as long as  ‘the game’, with its coalitions and combinations, remains the same. For example, goverments should only rely on these techniques for as long as ‘the social conception of emergent capitalist society as a collection of independent buyers and sellers’ is valid.

Dave Marsay

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: