Laplace’s … Essay on Probabilities
Laplace believed that all events follow laws, and devised a theory of probability based on ‘expectation’ that is consistent with gambling. But he did not suppose that such probabilities could always be estimated. For example, of probabilities derived by the use of analogies he wrote:
It is almost always impossible to submit to calculus the probability of the results obtained by these various means ; this is true likewise for historical facts.
Similarly, while he advocated the use of something like the principle of indifference he didn’t think it enough to have no reason to assign different probabilities: instead one had to have a reason to suppose thing equi-probable.
II Concerning Probability
ALL events, even those which on account of their insignificance do not seem to follow the great laws of nature, are a result of it just as necessarily as the revolutions of the sun.
This axiom, known by the name of the principle of sufficient reason, extends even to actions which are considered indifferent ; the freest will is unable without a determinative motive to give them birth ;
We ought then to regard the present state of the universe as the effect of its anterior state and as the cause of the one which is to follow. Given for one instant an intelligence which could comprehend all the forces by which nature is animated and the respective situation of the beings who compose it an intelligence sufficiently vast to submit these data to analysis it would embrace in the same formula the movements of the greatest bodies of the universe and those of the lightest atom ; for it, nothing would be uncertain and the future, as the past, would be present to its eyes.
The curve described by a simple molecule of air or vapor is regulated in a manner just as certain as the planetary orbits ; the only difference between them is that which comes from our ignorance.
It is to the influence of the opinion of those whom the multitude judges best informed and to whom it has been accustomed to give its confidence in regard to the most important matters of life that the propagation of those errors is due which in times of ignorance have covered the face of the earth.
This seems relevant to the financial crash of 2007/8. Laplace also gives a version of the Monty Hall problem, but using urns.
III The General Principles Of The Calculus Of Probabilities
The difference of opinions depends, however, upon the manner in which the influence of known data is determined. The theory of probabilities holds to considerations so delicate that it is not surprising that with the same data two persons arrive at different results, especially in very complicated questions. Let us examine now the general principles of this theory. [Some regard Bayes rule as overcoming the problem of different interpretations, but in fact it remains for some situations.]
[I]n the moral sciences, where each inference is deduced from that which precedes it only in a probable manner, however probable these deductions may be, the chance of error increases with their number and ultimately surpasses the chance of truth in the consequences very remote from the principle. [Boole later pointed this out, which implicitly yields comparative probabilities.]
First Principle. The first of these principles is the definition itself of probability, which, as has been seen, is the ratio of the number of favorable cases to that of all the cases possible.
Second Principle. But that supposes the various cases equally possible. If they are not so, we will determine first their respective possibilities, whose exact appreciation is one of the most delicate points of the theory of chance. Then the probability will be the sum of the possibilities of each favorable case.
Fourth Principle. When two events depend upon each other, the probability of the compound event is the product of the probability of the first event and the probability that, this event having occurred, the second will occur. … We see by this example the influence of past events upon the probability of future events.
Fifth Principle. If we calculate a priori the probability of the occurred event and the probability of an event composed of that one and a second one which is expected, the second probability divided by the first will be the probability of the event expected, drawn from the observed event. [Conditional probability, leading on to what English speakers call Bayes’ Theorem.]
Sixth Principle. Each of the causes to which an observed event may be attributed is indicated with just as much likelihood as there is probability that the event will take place, supposing the [cause] to be constant.
The probability of the existence of any one of these causes is then a fraction whose numerator is the probability of the event resulting from this cause and whose denominator is the sum of the similar probabilities relative to all the causes; if these various causes, considered a priori, are unequally probable, it is necessary, in place of the probability of the event resulting from each cause, to employ the product of this probability by the possibility of the cause itself. This is the fundamental principle of this branch of the analysis of chances which consists in passing from events to causes. [Bayes’ rule]
We arrange in our thought all possible events in various classes ; and we regard as extraordinary those classes which include a very small number. [The dependence of probability on classification can lead to two people with different classifying schema who correctly apply probability theory tending to contary results.]
Seventh Principle. The probability of a future event is the sum of the products of the probability of each cause, drawn from the event observed, by the probability that, this cause existing, the future event will occur.
This is the fundamental principle of this branch of the analysis of chances which consists in passing from events to causes.
[But] Placing the most ancient epoch of history at five thousand years ago … it is a bet of 1826214 to one that it will rise again to-morrow. But this number is incomparably greater for him who, recognizing in the totality of phenomena the principal regulator of days and seasons, sees that nothing at the present moment can arrest the course of it. [Thus one has imprecise comparative probabilities.]
IV Concerning Hope
Eighth Principle. When the advantage depends on several events it is obtained by taking the sum of the products of the probability of each event by the benefit attached to its occurrence.
Thus Laplace links probability to utility and rationality. But he discusses some of the limitations of such an approach:
[I]t is apparent that one franc has much greater value for him who possesses only a hundred than for a millionaire. We ought then to distinguish in the hoped-for benefit its absolute from its relative value.
Tenth Principle. The relative value of an infinitely small sum is equal to its absolute value divided by the total benefit of the person interested. [Laplace cites Bernouilli.]
Thus in the preceding question it is found that if the fortune of [a gambler] is two hundred francs, he ought not reasonably to stake more than nine francs.
It results similarly that at the fairest [gambling] game the loss is always greater than the gain.
The disadvantage of games of chance, the advantage of not exposing to the same danger the whole benefit that is expected, and all the similar results indicated by common sense, subsist, whatever may be the function of the physical fortune which for each individual expresses his moral fortune. It is enough that the proportion of the increase of this function to the increase of the physical fortune diminishes in the measure that the latter increases.
These remarks anticipate Ellsberg‘s risk aversion.
VII Concerning The Unknown Inequalities Which May Exist Among Chances Which Are Supposed Equal
But if there exist in the coin an inequality which causes one of the faces to appear rather than the other without knowing which side is favored by this inequality, the probability of throwing heads at the first throw will always be ½; because of our ignorance of which face is favored by the inequality the probability of the simple event is increased if this inequality is favorable to it, just so much is it diminished if the inequality is contrary to it. But in this same ignorance the probability of throwing heads twice in succession is increased.
We find thus generally that the constant and unknown causes which favor simple events which are judged equally possible always increase the probability of the repetition of the same simple event.
VIII Concerning The Laws Of Probability Which Result From The Indefinite Multiplication Of Events
One may draw from the preceding theorem this consequence which ought to be regarded as a general law, namely, that the ratios of the acts of nature are very nearly constant when these acts are considered in great number.
It follows again from this theorem that in a series of events indefinitely prolonged the action of regular and constant causes ought to prevail in the long run over that of irregular causes.
It is important then to the stability as well as to the happiness of empires not to extend them beyond those limits into which they are led again without cessation by the action of the causes; just as the waters of the seas raised by violent tempests fall again into their basins by the force of gravity. It is again a result of the calculus of probabilities confirmed by numerous and melancholy experiences. … Sometimes we attribute the inevitable results of these causes to the accidental circumstances which have produced their action.
When a simple event or one composed of several simple events … has been repeated a great number of times the possibilities of the simple events which render most probable that which has been observed are those that observation indicates with the greatest probability; in the measure that the observed event is repeated this probability increases and would end by amounting to certainty if the numbers of repetitions should become infinite.
There are two kinds of approximations: the one is relative to the limits taken on all sides of the possibilities which give to the past the greatest probability; the other approximation is related to the probability that these possibilities fall within these limits. The repetition of the compound event increases more and more this probability, the limits remaining the same; it reduces more and more the interval of these limits, the probability remaining the same ; in infinity this interval becomes zero and the probability changes to certainty.
Thus Laplace distinguishes between the states, behaviours and trends within ‘the system’, and the laws and limits that govern the system. On the one hand, a system will have common behaviours which one might reasonably expect to be repeated, as when current trends tend to be continued. But a system also has limits, which it will occasionally run into, leading to exceptional behaviour, as when a gradual boom leads to a sudden bust. The nature of causation is different for the different types of behaviour.
XV Concerning Illusions In The Estimation Of Probabilities
THE mind has its illusions as the sense of sight; and in the same manner that the sense of feeling corrects the latter, reflection and calculation correct the former. Probability based upon a daily experience, or exaggerated by fear and by hope, strikes us more than a superior probability but it is only a simple result of calculus. Thus we do not fear in return for small advantages to expose our life to dangers much less improbable than the drawing of a quint in the lottery of France; and yet no one would wish to procure for himself the same advantages with the certainty of losing his life if this quint should be drawn.
Thus the stability of actual order appears established at the same time by theory and by observations. But this order is effected by divers causes which an attentive examination reveals, and which it is impossible to submit to calculus. The actions of the ocean, of the atmosphere, and of meteors, of earthquakes, and the eruptions of volcanoes, agitate continually the surface of the earth and ought to effect in the long run great changes.
One may represent the successive states of the universe by a curve, of which time would be the abscissa and of which the ordinates are the divers states. Scarcely knowing an element of this curve we are far from being able to go back to its origin ; and if in order to satisfy the imagination, always restless from our ignorance of the cause of the phenomena which interest it, one ventures some conjectures it is wise to present them only with extreme reserve.
Thus probability can only be a guide to life in the short run. And perhaps only locally.
XVII Concerning The Various Means Of Approaching Certainty.
It is almost always impossible to submit to calculus the probability of the results obtained by these various means ; this is true likewise for historical facts. But the totality of the phenomena explained, or of the testimonies, is sometimes such that without being able to appreciate the probability we cannot reasonably permit ourselves any doubt in regard to them. In the other cases it is prudent to admit them only with great reserve.
XVIII Historical Notice Concerning The Calculus Of Probabilities.
The small uncertainty that the observations, when they are not numerous, leave in regard to the values of the constants of which I have just spoken, renders a little uncertain the probabilities determined by analysis. But it almost always suffices to know if the probability, that the errors of the results obtained are comprised within narrow limits, approaches closely to unity; and when it is not, it suffices to know up to what point the observations should be multiplied, in order to obtain a probability such that no reasonable doubt remains in regard to the correctness of the results.
Laplace has much of interest and use to say, both about calculating probabilities and interpreting the results. He largely covers what Bayes has to say about calculation, and says much more about interpretation.
Laplace, with Bernoulli, notes that (in modern terminology) utility is ‘sub-linear’ or ‘convex’ in nominal value. This implies that if probability of something good happening is either ‘p’ or ‘q’, with no reason for one to be the case rather than the other, then the effective probability is not the mid-point (p+q)/2, but is always strictly less, and is smaller the more significant the choice. A ‘neutral attitude to risk’ is the limiting case for a decision-maker who is so wealthy that the decision does not matter.
A Laplacean posterior probability, P(H|E), might best be interpreted as conditional, P(H|E:C) where C might include, for example, our current understanding of the laws of nature and an assumption that no hitherto unknown causes act upon it. Thus in applying any of his principles, including his version of Bayes’ rule, one has to be sure that the conditions are not challenged by the evidence. If they are, one needs to identify more general conditions (or weaker assumptions) that are not challenged. This may not always be possible.
A key distinction between Laplace and current convention is in terms of the temporal domain. Normally, one expects statistics to converge ‘in the long run’ but (as Keynes noted) ‘in the long run, we are all dead’. Laplace is not concerned with the very long term, or the short term, but with what Whitehead describes as ‘the current epoch’ or what we might call ‘for the time being’. It is thus important to identify what the appropriate epoch, to be used as a ‘frame of reference’ is, if any.