What is ‘evidence’?

As a mathematician, I tend to worry about mathematical theories and how they are applied, particularly how they are misunderstood and misapplied. Hence my blog has mostly about uncertainty and how it can be understood (or misunderstood) by reference to relevant mathematical theories, and in particular why some regard ‘mathematical probability’ as the go-to theory, even when faced with – to take some topical examples – climate change, discrimination (such as ‘institutional racism’) and pandemics, when the theory is clearly not so useful, and has such potential to be misapplied.

But trying to make sense of the UK approach to pandemics, and its claim to be ‘following the science’ I now adjust my question:

What do those with power and influence think ‘evidence’ is?

There is a lot of ‘text’ available, much of it seems to me to assume some sort of implicit underlying theory of uncertainty, such as probability, without addressing the appropriateness of any such theory or even giving much hint of what such a theory might be or how to think about it logically or mathematically. If anyone can suggest something on this topic that I might begin to make sense of (other than as a dogma) and so usefully critique, I’d appreciate it. A related question is:

What kind of logic is appropriate to thinking about ‘evidence’, and how would one judge its appropriateness?

Dave Marsay

About Dave Marsay
Mathematician with an interest in avoiding 'bad' reasoning.

5 Responses to What is ‘evidence’?

  1. This is an excellent question. I don’t know what “evidence” is except to say, whatever it is, it is probabilistic in nature: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2271870

    • Dave Marsay says:

      Thanks. Interesting paper. You make some good points about the hypothetical case, and quite possibly the description of the case is adequate to support a judgment on the actual case to the standard appropriate for a civil case. Certainly your line of argument is sometimes ‘good enough’. But more generally …

      You quote Keynes, who used the term ‘probability’ in a much broader sense than is usual today, and his Treatise on the subject is full of arguments that in certain situations conventional probabilities are inadequate.

      In your bus example the presumption seems to be that the situation is stochastic. If it is, the notion of evidence seems uncontroversial. But, as Keynes points out, we should always question whether or not the situation really is stochastic.

      In the case in question, Mrs Smith saw a bus ‘coming towards her at high speed’. Was Mrs Smith driving reasonably? If so, presumably the bus was not. Why not? Does Mrs Smith have enemies … It is easy to imagine that the probabilities that seem to be assumed are completely bogus. At least, we need to consider the frequency with which the buses in question are driven so dangerously.

      Going slightly beyond Keynes, we might look at the situation as one with feedback: dangerous drivers are liable to be punished. In this case, my assumption would be that regular drivers for established companies would be less likely to drive dangerously than other drivers, so I would want to find out who was driving the suspect bus on that occasion. Unfortunately for Mrs Smith, it was only a civil case. But had it been criminal I would hope that he lawyers would consider all these subtleties, in which case – hopefully – the probabilities might reasonably be regarded as sufficiently tested to establish them ‘beyond reasonable doubt’.(In the UK, though, this seems not always to be the case.)

      Does this help clarify my question? What do we mean by ‘evidence’ when there are ‘reasonable grounds’ to doubt the probability estimates? What do we regard as ‘reasonable grounds’?

      • Let me digest this and think it over; I will report back soon!

      • Speaking of Keynes, what about Frank Ramsey’s review and takedown of Keynes’s logical approach to probability? But that said, your point about dangerous drivers is well taken. As it turns out, I was a regular user of public buses when I lived in San Juan, Puerto Rico (1993 to 2009) and my experience was that the regular bus drivers were very aggressive and dangerous drivers (like most taxi drivers)!

  2. Dave Marsay says:

    Thanks again. I’m glad I don’t live in Puerto Rico!

    First, a confession. I tend to look for the logic in papers and comment on the bits I can get to grips with, rather than, for example ‘what has been demonstrated’ which appeals to some more general sense of ‘evidence’ which I don’t pretend to understand. As you point out, there is a lot of debate around probability theories which provide plenty of evidence that no-one really knows what they are talking about, which kind of supports my view. But taking your conclusion “In sum, legal scholars, lawyers, and judges should reconsider the potential usefulness of Bayesian methods at trial to test and evaluate the strength of probabilistic proof, instead of rejecting such proof altogether.” …

    If you replace ‘Bayesian methods’ by a either ‘modern Bayesian approach’ or perhaps even ‘Bayes’ methods’ I would be inclined to agree. But my feeling is that many have a somewhat dogmatic view of ‘Bayesian methods’ (rather like the difference between ‘Keynesian economics’ and ‘Keynes’ economics’), and that this dogma is not always appropriate. I would like to see more of a debate around probability that wasn’t limited to dogmatic Bayesianism. Our differing experiences of buses might provide an example. In the UK, I’m not sure that judges’ views on buses are at all well-founded.

    Moving on to your “To the extent law is nothing more than a prediction of what courts, juries, and legislators will do, the law of the future should be open to Bayesian methods.” Here I think you state very clearly the difference between my own view and that of many human scientists. I think there is ample evidence that Bayesian methods can be used to predict what courts etc will do, including when there will be miscarriages of justice. But surely the aspiration should be to do better than this?. I compare the situation with economics or psychology, where one has differing views on ‘rationality (and, at least implicitly, on ‘evidence’)’. Some use the term to describe behaviour that consistently leads to crises, and on a global scale seems to make the climate crisis inevitably disastrous. While such notions of ‘rationality’ and ‘evidence’ are important, maybe we need to consider alternatives? (E.g., In the bus case, surely it is better to consider the actual behaviour of bus drivers in that city, rather than make judgements based on experience of doubtful relevance?)

    Back to Keynes and Ramsey (and also Wittgenstein, Russell, Whitehead, Turing and Good). While there are clear differences in presentation and assumptions about their differing target audiences, which can distract, I find their underlying logics much easier to reconcile. For example, sometimes they are using terms like ‘probability’ in different senses, so one has to read them carefully, seeking common meaning rather than fixating on the differences in expression. Others take a different view, which brings me back to thinking about ‘evidence’.

    I would welcome any other statements about the use of evidence that are as clear as yours. Maybe the question should be ‘Is it possible to use ‘evidence’ in such a way that does not depend on questionable unacknowledged assumptions?’ ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: