Bayes’ Essay …
Bayes (Ed. Price) An Essay towards solving a Problem in the Doctrine of Chances 1763
The Rev. Price describes the purpose as being:
“to find out a method by which we might judge concerning the probability that an event has to happen, in given circumstances, upon supposition that we know nothing concerning it but that, under the same circumstances, it has happened a certain number of times, and failed a certain other number of times.”
“In what follows therefore I shall take for granted that …[it is] the rule to be used in relation to any event concerning the probability of which nothing at all is known antecedently to any trials made of observed concerning it.”
The main result is stated by Bayes as:
If nothing is known concerning an event but that it has happened p times and failed q in p + q or n trials, and from hence I guess the probability that of its happening in a single time lies somewhere between any two degrees of probability as X and x, the chance I am right in my guess is …”
The mathematical interest was in determining the chance above and, later, in providing practical approximations.
Rule 1 is used when there is no grounds for a ‘prior probability’. It yields a probability interval, much like one of Keynes’ approaches. The first and simplest example is:
Let us then first suppose, of … an event about the probability of which, antecedently to trials, we know nothing, [but] that it has happened once, and that it is enquired what conclusion we may draw from hence with respect to the probability of it’s happening on a second trial.
The answer is that there would be an odds of three to one for somewhat more than an even chance that it would happen on a second trial. [I.e.] the chance there is that the probability of an event that has happened once lies somewhere between 1 and ½.”
Thus Bayes’ view seems to be that purely empirical probabilities are often not just numbers, but are confidence intervals. This problem is meant to be archetypal:
“Suppose a solid or die or whose number of sides and constitution we know nothing; and that we are to judge of these from experiments made in throwing it.
… after the first throw and not before, we should be in the circumstances required by the conditions of the present problem … .
I have made these observations chiefly because they are all strictly applicable to the events and appearances of nature.”
Sometimes, we have reason to expect regularity, but even a regular mechanism may fail:
“… After having observed for some time the course of events … it would be found that the operations of nature are in general regular, and that the powers and laws which prevail in it are stable and permanent. The consideration of this will cause one or a few experiments often to produce a much stronger expectation of success in further experiments than would otherwise have been reasonable. .. What has been said seems sufficient to shew us what conclusions to draw from uniform experience. It demonstrates, particularly, that instead of proving that events will always happen agreeably to it, there will be always reason against this conclusion. In other words, where the course of nature has been the most constant, we can have only reason to reckon upon a recurrency of events proportioned to the degree of this constancy, but we can have no reason for thinking that there are no causes in nature which will ever interfere with the operations the causes from which this constancy is derived, or no circumstance of the world in which it will fail. …”
Thus, in the relatively short term, we expect the sun to rise every morning, but we dont expect it to last for ever.
In his introduction, Price notes:
“The purpose I mean is, to shew what reason we have for believing that there are in the constitution of things fixt laws according to which things happen, and that, therefore, the frame of the world must be the effect of the wisdom and power of an intelligent cause … . It will be easy to see that the converse problem solved in this essay is more directly applicable to this purpose; for it shews us, with distinctness and precision, in every case of any particular order or recurrency of events, what reason there is to think that such recurrency or order is derived from stable causes or regulations in nature, and not from any irregularities of chance.”
The argument is related to the law of large numbers. If sample statistics tend to stabilise then the generating mechanism must be de-facto probabilistic, like a gambling mechanism (which must have a creator). This assumption, while vital to many people’s conception of Bayesianism, sems to be Price’s own view, not necessarily Bayes’. Bayes does not say in his ‘rule 1’ or anywhere else that a sequence that has been stable will necessarily remain so, no matter how long it has been stable for: there is always the possibility of a surprise, no matter how small.
What is now known as Bayes rule is given in passing (as Prop. 5), and not designated by Bayes as a rule, or otherwise emphasised by him.
Bayes defines probability in terms of expected value (utility). His examples are all in terms of gambling. While the Rev Price clearly thought of the theory in very broad terms, it is not clear if Bayes had the same ambition. In any case, no justification of the definition is given. Thus, while probability, oddly, is define in terms of the value which ought to be computed for an expectation, but few hints are given as to how to do the calculation. It seems simpler to reverse engineer from aleotoric probability.
Suppose that I have promised to take my partner out for a meal tonight. Our two favourite restaurants are both full but think that there is a 50% chance that they may have a cancellation. Our next favourite restaurant has a table. I do not wish to ruin our reputation by holding a table and then not using it.
I regard the first two offers as having no value, and would accept the third. But then I find that a local big-wig has booked tables at both our favourite restaurants, and will only use one (depending which her new companion prefers). I book both restaurants.
Bayes’ approach seems to me to fail, here. It only seems to make sense when a wealthy person is gambling, and so the ‘value’ is the mathematical expectation. But then the definition seems rather circular.