Constructive Theory: Keynes’ Treatise
Keynes’ Treatise on Probability discusses statistical inference at some length, cautioning against the unprincipled application of techniques with hidden assumptions. Here he seems to recant, noting that science often finds great homogeneity and regularity, and thus tends to justify the assumptions implicit in the statistical methods that it relies upon. Thus while in theory the conventional theory is ill-founded, Keynes invites us to suppose that in practice the methods that it advocates work well enough. In such cases, his detailed argumentation on complexity and uncertainty seem far from academic: For those faced challenging real-world situations, Keynes’ reasoning is more important than his conclusions.
Part V The Foundations of Statistical Inference
The final part of Keynes’ Treatise contains his ‘negative’ warnings about the limitations of conventional statistical inference, which are summarise here, leading to Keynes’ positive theory. In essence, the statistical inference is justified when one ‘knows’ that the implicit and explicit assumptions identified in the earlier chapters of this part do hold.
Ch. XXXIII Outline of a Constructive Theory
1. THERE is a great difference between the proposition “It is probable that every instance of this generalisation is true” and the proposition “It is probable of any instance of this generalisation taken at random that it is true.” The latter proposition may remain valid, even if it is certain that some instances of the generalisation are false. It is more likely than not, for example, that any number will be divisible either by two or by three, but it is not more likely than not that all numbers are divisible either by two or by three.
The first type of proposition has been discussed in Part III under the name of Universal Induction. The latter belongs to Inductive Correlation or Statistical Induction, an attempt at the logical analysis of which must be my final task.
2. ‘What advocates of the Frequency Theory of Probability wrongly believe to be characteristic of all probabilities, namely, that they are essentially concerned not with, single instances but with series of instances, is, I think, a true characteristic of statistical induction. A statistical induction either asserts the probability of an instance selected at random from a series of propositions, or else it assigns the probability of the assertion, that the truth frequency of a series of propositions (i.e. the proportion of true propositions in the series) is in the neighbourhood of a given value. In either case it is asserting a characteristic of a series of propositions, rather than of a particular proposition.
Whilst, therefore, our unit in the case of Universal Induction is a single instance which satisfies both the condition and the conclusion of our generalisation, our unit in the case of Statistical Induction is not a single instance, but a set or series of instances, all of which satisfy the condition of our generalisation but which satisfy the conclusion only in a certain proportion of cases. And whilst in Universal Induction we build up our argument by examining the known positive and negative Analogy shown in a series of single instances, the corresponding task in Statistical Induction consists in examining the Analogy shown in a series of series of instances.
3. We are presented, in problems of Statistical Induction, with a set of instances all of which satisfy the conditions of our generalisation, and a proportion / of which satisfy its conclusion; and we seek to generalise as to the probable proportion in which further instances will satisfy the conclusion.
Now it is useless merely to pay attention to the proportion (or frequency) / discovered in the aggregate of the instances. For any collection whatever, comprising a definite number of objects, must, if the objects be classified with reference to the presence or absence of any specified characteristic whatever, show some definite proportion or statistical frequency of occurrence ; so that a mere knowledge of what this frequency is can have no appreciable bearing on what the corresponding frequency will be for some other collection of objects, or on the probability of finding the characteristic in an object which does not belong to the original collection. We should be arguing in the same sort of way as if we were to base a universal induction as to the concurrence of two characteristics on a single observation of this concurrence, and without any analysis of the accompanying circumstances.
Let the reader be clear about this. To argue from the mere fact that a given event has occurred invariably in a thousand instances under observation, without any analysis of the circumstances accompanying the individual instances, that it is likely to occur invariably in future instances, is a feeble inductive argument, because it takes no account of the Analogy. Nevertheless an argument of this Mnd is not entirely worthless, as we have seen in Part III. …
4. Example One. – Let us investigate the generalisation that the proportion of male to female births is m. The fact that the aggregate statistics for England during the nineteenth century yield the proportion m would go no way at all towards justifying the statement that the proportion of male births in Cambridge next year is likely to approximate to m. Our argument would be no better if our statistics, instead of relating to England during the nineteenth century, covered all the descendants of Adam. But if we were able to break up our aggregate series of instances into a series of sub-series, classified according to a great variety of principles, as for example by date, by season, by locality, by the class of the parents, by the sex of previous children, and so forth, and if the proportion of male births throughout these sub-series showed a significant stability in the neighbourhood of m, then indeed we have an argument worth something. Otherwise we must either abandon our generalisation, amplify its conditions, or modify its conclusion.
5. … This is due to the fact that a statistical induction is not really about the particular instance at all, but has its subject, about which it generalises, a series ; and it is only applicable to the particular instance, in so far as the instance is relative to our knowledge, a random member of the series. If the acquisition of new knowledge affords us additional relevant information about the particular instance, so that it ceases to be a random member of the series, then the statistical induction ceases to be applicable; but the statistical induction does not for that reason become any less probable than it was it is simply no longer indicated by our data as being the statistical generalisation appropriate to the instance under inquiry. The point is illustrated by the familiar example that the probability of an unknown individual posting a letter unaddressed can be based on the statistics of the Post Office, but my expectation that I shall act thus, cannot be so determined.
Thus a statistical generalisation is always of the form : ‘ The probability, that an instance taken at random from the series S will have the characteristic Φ , is p ;’ or, more precisely, if a is a random member of S(x), the probability of Φ(a) is p.
It is not always the case that the evidence indicates any series at all as ‘ appropriate ‘ in the above sense. In particular, if evidence h indicates S as the appropriate series, and evidence h’ indicates S’ as the appropriate series, then relative to evidence hh’ (assuming these to be not incompatible), it may be the case that no determinate series is indicated as appropriate. In this case the method of statistical induction fails us as a means of determining the probability under inquiry.
6. … For, until a prima facie case has been established for the existence of a stable probable-frequency, we have but a flimsy basis for any statistical induction at all ; indeed we are limited to the class of case where the instance under inquiry is a member of identically the same series as that from which our samples were drawn, i.e. where S = S 1 , which in social and scientific inquiries is seldom the case.
… Lexis describes the stability as subnormal, normal, or supernormal according as Q is less than, equal to, or greater than 1. This is too precise, and it is better perhaps to say that the stability about the mean is normal if the dispersion is such as would not be improbable a priori, if we had assumed that the members of S1, S2 , etc., were obtained by random selection out of a single universe U, that it is sub-normal if the dispersion is less than one would have expected on the same hypothesis, and that it is supernormal if the dispersion is greater than one would have expected.
Let me recapitulate the two essential stages of the argument. We first find that the observed frequencies in a set of series are such as would have been not improbable a priori if, relative to our knowledge, these series had all been made up of random members of the same universe U ; and we next argue that the positive and negative analogies of this set of series furnish an inductive argument of some weight for supposing that a further unexamined series S resembles the former series in having a frequency for the characteristic under inquiry such as would have been not improbable a priori if, relative to our knowledge, S was also made up of random members of the hypothetical universe U.
7. … Now the field of statistical induction is the class of phenomena which are due to the combination of two sets of influences, one of them constant and the other liable to vary in accordance with the expectations of objective chance, Quetelet’s ‘ permanent causes ‘ modified by ‘ accidental causes.’ In social and physical statistics the ultimate alternatives are not as a rule so perfectly fixed, nor the selection from them so purely random, as in the ideal game of chance. But where, for example, we find stability in the statistics of crime, we could explain this by supposing that the population itself is stably constituted, that persons of different temperaments are alive in proportions more or less the same from year to year, that the motives for crime are similar, and that those who come to be influenced by these motives are selected from the population at large in the same kind of way. Thus we have stable causes at work leading to the several alternatives in fixed proportions, and these are modified by random influences. Generally speaking, for large classes of social statistics we have a more or less stable population including different kinds of persons in certain proportions and on the other hand sets of environments ; the proportions of the different kinds of persons, the proportions of the different kinds of environments, and the manner of allotting the environments to the persons vary in a random manner from year to year (or, it may be, from district to district). In all such cases as these, however, prediction beyond what has been observed is clearly open to sources of error which can be neglected in considering, for example, games of chance ; our so-called ‘ permanent ‘ causes are always changing a little and are liable at any moment to radical alteration.
Thus the more closely that we find the conditions in scientific examples assimilated to those in games of chance, the more confidently does common sense recommend this method. … If the sphere of influence of Mendelian considerations is wide, we have both an explanation in part of what we observe and also a large opportunity in future of using with profit the methods of
8. Two subsidiary questions remain to be mentioned. The first of these relates to the character of series which., in the terminology of Lexis, show a subnormal or supernormal stability ; for I have pressed on to the conclusion of the argument on the assumption that the stabilities are normal. Subnormal stability conceals two types : the one in which there is really no stability at all and the results are in fact chaotic ; and the other in which there is mutual dependence between the successive instances of such a kind that they tend to resemble one another so that any divergence from the normal tends to accentuate itself. Super-normal stability corresponds in the other direction to the second of these two types ; that is to say, there is mutual dependence of a regulative kind between the successive instances which tends to prevent the frequency from swinging away from its mean value.
The truth is that sensible investigators only employ the correlation coefficient to test or confirm conclusions at which they have arrived on other grounds. But that does not validate the crude way in which the argument is sometimes presented, or prevent it from misleading the unwary, since not all investigators are sensible.
… in taking leave of Probability, I should like to say that, in my judgment; the practical usefulness of those modes of inference, here termed Universal and Statistical Induction, on the validity of which the boasted knowledge of modern science depends, can only exist and I do not now pause to inquire again whether such an argument must be circular if the universe of phenomena does in fact present those peculiar characteristics of atomism and limited variety which appear more and more clearly as the ultimate result to which material science is tending :
materiem quoque finitis differre figuris.
The physicists of the nineteenth century have reduced matter to the collisions and arrangements of particles, between which the ultimate qualitative differences are very few ; and the Mendelian biologists are deriving the various qualities of men from the collisions and arrangements of chromosomes. In both cases the analogy with the perfect game of chance is really present ; and the validity of some current modes of inference may depend on the assumption that it is to material of this kind that we are applying them. Here, though I have complained sometimes at their want of logic, I am in fundamental sympathy with the deep underlying conceptions of the statistical theory of the day. If the contemporary doctrines of Biology and Physics remain tenable, we may have a remarkable, if undeserved, justification of some of the methods of the traditional Calculus of Probabilities. Professors of probability have been often and justly derided for arguing as if nature were an urn containing black and white balls in fixed proportions. Quetelet once declared in so many words “l’urne que nous interrogeons, c’est la nature.” But again in the history of science the methods of astrology may prove useful to the astronomer ; and it may turn out to be true reversing Quetelet’s expression that ” La nature que nous interrogeons, c’est une urne.”
This ‘constructive’ ending seems to beg a synthesis with Keynes’ earlier, negative, findings. Keynes constructs a positive theory of statistical inference in which he notes that science generally succeeds in explaining phenomena, and so retrospectively justifies the assumptions of its statistical methods. Thus, he notes, if the necessary homogeneity and regularity were universal then classical statistical inference could be applied without regard to the implicit assumptions, as is often the case.
But in the development of any science it is commonplace for significant factors to have been overlooked, and the regularity to be misleading, as when some people thought that all swans were white. Moreover, it seems inimical to the core concepts of science ever to suppose that everything is absolutely known about a subject. A synthesis with Keynes own negative findings would be to suppose that science (and other honest empirical endeavours) is justified when two things hold:
- Its findings are published in a form that allows others to check them and to look for additional factors.
- Its conclusions are expressed or invariably interpreted as being conditional on ‘all else remaining equal’, realising that this may not be the case when others have an incentive to change the situation (e.g. by identifying new factors) or the situation is otherwise subject to substantial evolution.
Roughly speaking, this is the position that Keynes’ developed for economics, for which the implicit assumptions of classical science clearly do not hold. That is, one can proceed ‘scientifically’, but with humility about the results. This contrasts with scientism, which applies the methods of science with no regard for their validity in the actual case.