CSPO’s Crisis in science?

Andrea Saltelli, Jerome Ravetz and Silvio Funtowicz Who will solve the crisis in science? Ch. 1 of “The Rightful Place of Science: Science on the Verge” Consortium for Science, Policy & Outcomes Arizona State University, 2016

Endangered science

The integrity of science has been the subject of increasing concern, among scientists and the media, over the past decade. Although science is still among the most trusted of public institutions, the crisis of quality within science is now threatening to erode that trust.
Fraud and misconduct may be only part of the greater problem of integrity in science, but they are the most visible signs of a crisis. …

Rethinking the problem

In this section we suggest that the crisis in science has not yet been accurately described or diagnosed and that real insight into the situation will require a deeper analysis of its causes.
Our conclusion thus far is that the problems in science will not be fixed by better training in statistics, better alignment of incentives with objectives, better regulation of copyright or the elimination of impact factors …we are facing what we could call a generalized crisis in the epistemic governance of science.

What can be done

The main impediment to a possible cure of the present disease is the belief that the system will straighten itself out: that the scientific community can use its own craft to mend itself. This is implausible because the assumptions, structures and practices out of which the crisis arose are not likely to produce its solution unaided. Some help from without is in order.
If purely technical or instrumental solutions are unlikely to be adequate to solve the crisis, then the intuitions and endeavours of concerned scientists are likely to benefit from the contributions of other voices, including reflective scholars, journalists and members of civil society. Scientists will certainly play a crucial role in the construction of the future of science. …
Beyond these initiatives, we urgently need a philosophy of science in which science’s imperfections and vulnerabilities are acknowledged and explored. …
A further philosophical development, taking into account complexity, ignorance, abuse of mathematics and corruption in its broad (not personal) sense, is now overdue.
One necessary step on this path will be to review the traditional assumption of the separation between science and society. …
The ideal of a disinterested scientific practice which manages to isolate itself from the messiness of everyday life and politics is, of course, an abstraction.
Nowhere is such a crisis more evident than in the inappropriate use of mathematical modelling and quantification of the world (Saltelli and Funtowicz, 2014), which undermines the use of such evidence for policy … . One of the most hopeful signs in the present crisis is the tacit abandonment of the traditional image of science as a truth-producing machine. We can therefore already point to some of what needs to be ‘unlearned’ in the prevailing model of science. Kuhn observed that education in science is “narrow and rigid” and comparable to orthodox theology … .
There is an ‘implicit scientific catechism’ that students learn by example but that working scientists must leave behind: chiefly, that every scientific problem has one and only one correct solution, precise to several significant digits; that quantitative data and mathematical techniques produce certainty; and that error in science is the result of stupidity or malevolence.
Education will therefore clearly be an important facet of the necessary reform of science. There are signs that science education may already be changing, under the influence of the new social media. The growth of ‘do-it-yourself (DIY) science’, showing only a minimum of deference to established science, will eventually influence science education to good effect. When students conceive of a scientific exercise as a ‘hack’ rather than a ‘proof’, a new consciousness is being created. Kuhn’s gloomy picture of science education may at last be on the way out.
In his quest for social solutions to the anticipated predicaments of science, Ravetz (1971) envisaged a new “critical science” [which] led to the concept of “post-normal science” (PNS), which is today relatively well known as an approach to deal with problems at the interface between science and policy. While PNS was designed to apply where facts are uncertain, values in dispute, stakes high and decisions urgent (Funtowicz and Ravetz, 1991, 1992, 1993), with the ecological movement as one of its driving forces, PNS is understood today as a system of epistemic governance of practical applicability to all domains in which science and society interact, i.e. by definition, to all settings where science operates, including reflexively to the operation of science itself. …In the “extended participation model” (Funtowicz, 2006), deliberation (on what to do) is extended across disciplines—in the acknowledgment that each discipline has its own lens—and across communities of experts and stakeholders. In adopting this model, one moves from ‘speaking truth to power’ towards “working deliberatively within imperfections” (Funtowicz, 2006f).
McQuillan (2014) has recently remarked that the movement known as ‘Citizen Science’ could seize the opportunity created by the crisis in science. PNS is singled out by McQuillan as a promising framework for the work to be done. The remainder of this section will focus on the possibility that the principles of PNS, foremost that of the participation of extended peer communities, may furnish some elements of a solution to the crisis.

Two wings can be identified within the citizen science movement: ‘amateur-citizen’ and ‘activist-citizen’, depending on how they interact with established science. …
This mainly external movement may also eventually lead to the emergence of a ‘scientist-citizen’ movement within established science itself. Scientist-citizens could engage both with the internal problems of science, such as trust and quality assurance, and with the externalchallenges relating to the use of science to solve practical(i.e. policy) problems. An eloquent argument for therecognition of such scientist-citizens has been made byJack Stilgoe (2009). …A scientist-citizen approach would commit scientists qua citizens to criticism, reflection and action.

Conclusion

Could citizen science and scientist-citizens together perform the rescue of quality and trust in science? It is much too early to say, and the evidence proposed here in support of the thesis is largely based on anecdote, metaphor and analogy and on the predictions of a book that is now more than forty years old (Ravetz, 1971 …). It is clear that action is being taken on many fronts, both in established science and in its new forms of practice. The restoration of quality in science, and the preservation of trust, will not be accomplished by ‘scientific’ means alone. We are therefore facing one of the greatest challenges for science of our times.

My Comments

Theory

Practitioners often like to remind academics that:

“In theory there is no difference between theory and practice. In practice there is.”
(Variously attributed.)

But what is the difference? And which is right? In logic, including mathematics, in so far as the theory is actually sound (and not in error), the difference is that in practice practitioners are liable to error. Logicians and mathematicians are well aware of this, and strive for correctness. Apart from logic, theories are empirical. In so far as these theories are sound, there surely ought to be no difference, and if there is practitioners are again human, and so one might reasonably expect the theory to be more trustworthy. Yet here we have a problem:

 “The great tragedy of science — the slaying of a beautiful hypothesis by an ugly fact.”
Thomas Huxley

This seems to get at the core of concerns about ‘normal science’: sufficiently educated people should realise that it follows from both logic and experience that normal science cannot be deserving of the kind of trust that people sometimes seem to look for or suppose. While some may say that a particular theory has been ‘well-confirmed‘, this shouldn’t be taken too literally: the appropriateness of an empirical theory should never be turned into a dogma.

This seems to me the main thing that people should know about ‘normal’ science: it may be of the highest quality, conducted by people of the highest probity, and yet only ‘well’ confirmed, never absolutely.

Practice

Post-normal science seems remarkably like that practiced by the boffins of Malvern, at least until they became ‘better managed’. Saltelli et al’s account resonates with the changes that the boffinery underwent, which might provide an interesting case study. Thus, contrary to Saltelli et al, it seems to me that what we need is not so much post-normal science but a return to science before it became ‘corrupted’ (in Slatelli et al’s sense).

But how did science get to be corrupted? Some managers, at least, take their reponsibilities seriously and suppose that in order to discharge them their staff (including scientists) need to be able to explain what they are doing in accessible terms. This may seem reasonable, but sometimes the reification fallacy seems to matters, and maybe those with influence over citizen-science collaborations (such as educators and politicians) need to be more aware of this? I’m not sure that Saltelli et al quite get this. Or maybe it just needs more explication?

Synthesis

Even the best ‘confirmed’ theory can only be really said to have survived various tests. In the case of mathematics this is a test of a supposed proof against some explicit logic. In the ‘hard’ sciences it is a repeatable experiment. We might say that a theory has not been falsified by various tests, but which tests?

It seems to me that there should be some definite tests that the theory hasn’t failed, together with some common reasoning for some definite community of practice (e.g., engineers) that hasn’t led to some false conclusion. Then we may reasonably regard commonplace deductions by the community as relatively reliable. Less so the result of exceptional reasonings, practitioners or circumstances. Thus the issue is not so much the trustworthiness of the theory (regarded as deductions about a narrow range of experience) as the trustworthiness of applying the theory, which depends on it being applied by similar people in a similar manner in similar circumstances to those in which the theory was previously been applied.

‘Normal’ science often overlooks this caveat, and perhaps should be regarded as ‘routine’ science, not to be trusted otherwise. ‘Post-normal science’ ought to always bear this  in mind, as the best scientists (including boffins) havce done, at least when working on non-routine problems.

One might think of developing, for any given theory, a meta-theory about when the given theory was trustworthy. But then you’d need a meta-meta-theory, and so on. Ultimately, any application of a theory has to rely on an assumption or judgment of similarity, which can only be as good as the recognition of relevant distinctions. In this sense the reliability of the appliance of science depends on there being  a track record of success and no reason to expect any emerging new relevant ‘differences that may make a difference’.

 

 

%d bloggers like this: