Consensus is what we say it is

But who are “we”?

HM CollinsA BartlettLI Reyes-Galindo,  The Ecology of Fringe Science and its Bearing on Policy, arXiv:1606.05786v1 [physics.soc-ph],  Sat, 18 Jun 2016.

 In this paper we illustrate the tension between mainstream ‘normal’, ‘unorthodox’ and ‘fringe’ science that is the focus of two ongoing projects that are analysing the full ecology of physics knowledge. The first project concentrates on empirically understanding the notion of consensus in physics by investigating the policing of boundaries that is carried out at the arXiv preprint server, a fundamental element of the contemporary physics publishing landscape. The second project looks at physics outside the mainstream and focuses on the set of organisations and publishing outlets that have mushroomed outside of mainstream physics to cover the needs of ‘alternative’, ‘independent’ and ‘unorthodox’ scientists. Consolidating both projects into the different images of science that characterise the mainstream (based on consensus) and the fringe (based on dissent), we draw out an explanation of why today’s social scientists ought to make the case that, for policy-making purposes, the mainstream’s consensus should be our main source of technical knowledge.

I immediately notice a series of assumptions: that the authors  know what “consensus in physics” is, or “the mainstream (based on consensus)”, and that this, whatever it is, should be our main source of “technical knowledge.” Who is it that is asking the question, to whom does “our” refer in the last sentence?

Legally, the proposed argument is bullshit. Courts, very interested in knowledge, fact and clear interpretation, do not determine what the “mainstream consensus” is on a topic, nor do review bodies, such as, with our special interest, the U.S. Department of Energy in its 1989 and 2004 reviews. Rather, they seek expert opinion, and, at best, in a process where testimony and evidence are gathered.

Expert opinion would mean the opinions of those with the training, experience, and knowledge adequate to understand a subject, and who have actually investigated the subject themselves, or who are familiar with the primary reports of those who have investigated. Those who rely on secondary and tertiary reports, even from academic sources, would not be “expert” in this meaning. Those who rely on news media  would simply be bystanders, with varying levels of understanding, and quite vulnerable to information cascades, the same as everyone with anything where personal familiarity is absent. The general opinions of people are not admissible as evidence in court, nor are they of much relevance in science.

But sociologists study human society. Where these students of the sociology of science wander astray is in creating a policy recommendation — vague though it is — without thoroughly exploring the foundations of the topic.

Are those terms defined in the paper?

Consensus is often used very loosely and sloppily. Most useful, I think, is the meaning of “the widespread agreement of experts,” and the general opinion of a general body is better described by “common opinion.” The paper is talking about “knowledge,” and especially “scientific knowledge,” which is a body of interpretation created through the “scientific method,” and which is distinct from the opinions of scientists, and in particular the opinions of those who have not studied the subject.

1ageneral agreement UNANIMITY

the consensus of their opinion, based on reports … from the border—John Hersey

bthe judgment arrived at by most of those concerned

the consensus was to go ahead

2group solidarity in sentiment and belief

Certainly, the paper is not talking about unanimity, indeed, the whole thrust of it is to define fringe as “minority,” So the second definition applies, but is it of “those concerned.” By the conditions of the usage, “most scientists” are not “concerned” with the fringe, they generally ignore it. But “consensus” is improperly used, when the meaning is mere majority.

And when we are talking about a “scientific consensus,” to make any sense, we must be talking about the consensus of experts, not the relatively ignorant. Yet the majority of humans like to be right and to think that their opinions are the gold standard of truth. And scientists are human.

The paper is attempting to create a policy definition of science, without considering the process of science, how “knowledge” is obtained. It is, more or less, assuming the infallibility of the majority, at some level of agreement, outside the processes of science. 

We know from many examples the danger of this. The example of Semmelweiss is often adduced. Semmelweiss’s research and his conclusions contradicted the common opinion of physicians who delivered babies. He studied the problem of “childbed fever” with epidemological techniques, and came to the conclusion that the primary cause of the greatly increased mortality among those attended by physicians over those attended by midwives, was the practice of doctors who performed autopsies (a common “scientific” practice of those days) and who left the autopsy and examined women invasively, without thorough antisepsis. Semmelweiss studied hospital records, and then introduced antiseptic practices, and saw a great decrease in mortality.

But Semmelweiss was, one of his biographers thinks, becoming demented, showing signs of “Alzheimer’s presenile dementia,” and Semmelweiss became erratic and oppositional (one of the characteristics of some fringe advocates, as the authors of our paper point out). He was ineffective in communicating his findings, but it is also true that he met with very strong opposition that was not based in science, but in the assumption of physicians that what Semmelweiss was proposing was impossible.

This was before germ theory was developed and tested by Pasteur. The error of the “mainstream” was in not paying attention to the evidence Semmelweiss found. If they had done so, it’s likely that many thousands of unnecessary deaths would have been avoided.

I ran into something a little bit analogous in my personal history. I delivered my own children, after our experience with the first, relying on an old obstetrics textbook (DeLee, 1933) and the encouragement of an obstetrician. Later, because my wife and I had experience, we created a midwifery organization, trained midwives, and got them licensed by the state, a long story. The point here is that some obstetricians were horrified, believing that what we were doing was unsafe, and that home birth was necessarily riskier than hospital birth. That belief was based on wishful thinking. “We do everything to make this as safe as possible” is not evidence of success.

An actual study was done, back then. It was found that home birth in the hands of skilled midwives, and with proper screening, i.e., not attempting to delivery difficult cases at home, was slightly safer than hospital birth, though the difference was not statistically significant. Why? Does it matter why? However, there is a theory, and I think the statistics supported it. A woman delivering at home is accustomed to and largely immune to microbes present in the home. Not so with the hospital. There are other risks where being at home could increase negative outcomes, but they are relatively rare, and it appears that the risks at least roughly balance. But a great deal would depend on the midwives and how they practice.

(There is a trend toward birthing centers, located adjacent to hospitals, to avoid the mixing of the patient population. This could ameliorate the problem, but not eliminate it. Public policy, though, if we are going to talk about “shoulds,” should not depend on wishful thinking, and too often it does.)

(The best obstetricians, though, professors of obstetrics, wanted to learn from the midwives: How do you avoid doing an episiotomy? And we could answer that from experience. Good scientists are curious, not reactive and protective “being right,” where anything different from what they think must be “wrong.” And that is, in fact, how the expertise of a real scientist grows.)

Does the paper actually address the definitional and procedural issues? From my first reading, I didn’t see it.

From the Introduction:

 Fringe science has been an important topic since the start of the revolution in the social studies of science that occurred in the early 1970s.2 As a softer-edged model of the sciences developed, fringe science was a ‘hard case’ on which to hammer out the idea that scientific truth was whatever came to count as scientific truth: scientific truth emerged from social closure. The job of those studying fringe science was to recapture the rationality of its proponents, showing how, in terms of the procedures of science, they could be right and the mainstream could be wrong and therefore the consensus position is formed by social agreement.

First of all, consensus in every context is formed by social agreement, outside of very specific contexts (which generally control the “agreement group” and the process). The conclusion stated does not follow from the premise that the fringe “could be right.” The entire discussion assumes that there is a clear meaning to “right” and “wrong,” it is ontologically unsophisticated. Both “right” and “wrong” are opinions, not fact, though there are cases where we would probably all agree that something was right or wrong, but when we look at this closely, they are situations where evidence is very strong, or the rightness and wrongness are based on fundamental human qualities. They are still a social agreement, even if written in our genes.

I do get a clue what they are about, though, in the next paragraph:

One outcome of this way of thinking is that sociologists of science informed by the perspective outlined above find themselves short of argumentative resources for demarcating science from non-science.

These are sociologists, yet they appear to classify an obvious sociological observation as “a way of thinking,” based on the effect, this being argument from consequences, having no bearing on the reality. So, for what purpose would we want to distinguish between science and non-science? The goal, apparently, is to be able to argue the distinction, but this is an issue which has been long studied. In a definitional question like this, my first inquiry is, “Who wants to know, and why?” because a sane answer will consider context.

There are classical ways of identifying the boundaries. Unfortunately, those ways require judgment. Whose judgment? Rather than judgment, the authors appear to be proposing the use of a vague concept of “scientific consensus,” that ignores the roots of that. “Scientific consensus” is not, properly, the general agreement of those called “scientists,” but of those with expertise, as I outline above. It is a consensus obtained through collective study of evidence. It can still be flawed, but my long-term position on genuine consensus is that it is the most reliable guide we have, and as long as we keep in mind the possibility that any idea can be defective, any interpretation may become obsolete, in the language of Islam, if we do not “close the gates of ijtihaad,” as some imagine happened over a thousand years ago, relying on social agreement, and especially the agreement of the informed, is our safest course.

They went on:

The distinction with traditional philosophy of science, which readily
demarcates fringe subjects such as parapsychology by referring to their ‘irrationality’ or some such, is marked.3
For the sociologist of scientific knowledge, that kind of demarcation comprises a retrospective drawing on what is found within the scientific community. In contrast, the sociological perspective explains why a multiplicity of conflicting views on the same topic, each with its own scientific justification, can coexist. A position that can emerge from this perspective is to argue for less authoritarian control of new scientific initiatives – for a loosening of the controls on the restrictive side of what Kuhn (1959, 1977) called ‘the essential tension’. The essential tension is between those who believe that science can only progress within consensual
‘ways of going on’ which restrict the range of questions that can be asked, the ways of asking and answering them and the kinds of criticism that it is legitimate to offer – this is sometime known as working within ‘paradigms’ – and those who believe that this kind of control is unacceptably  authoritarian and that good science is always maximally creative and has no bounds in these respects. This tension is central to what we argue here. We note only that a complete loosening of control would lead to the dissolution of science.

They note that, but adduce no evidence. Control over what? There are thousands upon thousands of institutions, making decisions which can affect the viability of scientific investigation. The alleged argument, stated as contrary “beliefs,” misses that there could be a consensus, rooted in reality. What is reality? And there we need more than the kind of shallow sociology that I see here. Socially, we get the closest to the investigation of reality in the legal system, where there are processes and procedures for finding “consensus,” as represented by the consensus of a jury, or the assessment of a judge, with procedures in place to assure neutrality, even though we know that those procedures sometimes fail, hence there are appeal procedures, etc.

In science, in theory, “closure” is obtained through the acceptance of authoritative reviews, published in refereed journals. Yet such process is not uncommonly bypassed in the formation of what is loosely called “scientific consensus.” In those areas, such reviews may be published, but are ignored, dismissed. It is the right of each individual to decide what information to follow, and what not, except when the individual, or the supervising organization, has a responsibility to consider it. Here, it appears, there is an attempt to advise organizations, as to what they should consider “science.”

Why do they need to decide that? What I see is that if one can dismiss claims coming under consideration, based on an alleged “consensus,” which means, in practice, I call up my friend, who is a physicist, say, and he says, “Oh, that’s bullshit, proven wrong long ago. Everybody knows.”

If someone has a responsibility, it is not discharged by receiving and acting on rumors.

The first question, about authoritarian control, is, “Does it exist?” Yes, it does. And the paper rather thoroughly documents it, as regards the arXiv community and library. However, if a “pseudoskeptic” is arguing with a “fringe believer,” — those are both stereotypical terms —  and the believer mentions the suppression, the skeptic will assert, “Aha! Conspiracy theory!” And, in fact, when suppression takes place, conspiracy theories do abound. This is particularly true if the suppression is systemic, rather than anecdotal. And with fringe science, it is systemic.

Anyone who researches the history of cold fusion will find examples, where authoritarian control is exerted with means that not openly acknowledged, and with cooperation and collaboration in this. Is that a “conspiracy”? Those engaged in it won’t think so. This is just sensible people cooperating with each other.

I would distinguish between this activity as a “natural conspiracy,” from “corrupt conspiracy,” as if, for example, the oil industry were conspiring to suppress cold fusion because of possible damage to their interests. In fact, I find corrupt conspiracy extremely unlikely in the case of cold fusion, and in many other cases where it is sometimes asserted.

The straw man argument, they set up, is between extreme and entrenched positions, depending on knee-jerk reactions. That is “authoritarian control” is Bad. Is it? Doesn’t that depend on context and purpose?

But primitive thinkers are looking for easy classifications, particularly into Good and Bad. The argument described is rooted in primitive thinking, and certainly not actual sociology (which must include linguistics and philosophy).

So I imagine a policy-maker, charged with setting research budgets, presented with a proposal for research that may be considered fringe. Should he or she approve the proposal? Now there are procedures, but this stands out: if the decider decides according to majority opinion among “scientists,” it’s safer. But it also shuts down the possibility of extending the boundaries of science, and that can sometimes cause enormous damage. Those women giving birth in hospitals in Europe in the 19th century. They died because of a defective medical practice, and because reality was too horrible to consider, for the experts. It meant that they were, by their hands, killing women. (One of Semmelweiss’s colleagues, who accepted his work, realized that he had caused the death of his niece, and committed suicide.)

What would be a more responsible approach? I’m not entirely sure I would ask sociologists, particularly those ontologically unsophisticated. But they would, by their profession, be able to document what actually exists, and these sociologists do that. But as to policy recommendations, they put their pants on one leg at a time. They may have no clue.

What drives this paper is a different question that arises out of the sociological perspective: What is the outside world to do with the new view?

Sociologists may have their own political opinions, and these clearly do. Science does not provide advice, rather it can, under the best circumstances, inform decisions, but decision-making is a matter of choices, and science does not determine choices. It may, sometimes, predict the consequences of choices. But these sociologists take it as their task to advise, it seems.

So who wants to know and for what purpose? They have this note:

1 This paper is joint work by researchers supported by two grants: ESRC to Harry Collins, (RES/K006401/1) £277,184, What is scientific consensus for policy? Heartlands and hinterlands of physics (2014-2016); British Academy Post-Doctoral Fellowship to Luis Reyes-Galindo, (PF130024) £223,732, The social boundaries of scientific knowledge: a case study of ‘green’ Open Access (2013-2016).

Searching for that, I first find a paper by these authors:

Collins, Harry & Bartlett, Andrew & Reyes-Galindo, Luis. (2017). “Demarcating Fringe Science for Policy.” Perspectives on Science. 25. 411-438. 10.1162/POSC_a_00248. Copy on ResearchGate.

This appears to be a published version of the arXiv preprint. The abstract:

Here we try to characterize the fringe of science as opposed to the mainstream. We want to do this in order to provide some theory of the difference that can be used by policy-makers and other decision-makers but without violating the principles of what has been called ‘Wave Two of Science Studies’. Therefore our demarcation criteria rest on differences in the forms of life of the two activities rather than questions of rationality or rightness; we try to show the ways in which the fringe differs from the mainstream in terms of the way they think about and practice the institution of science. Along the way we provide descriptions of fringe institutions and sciences and their outlets. We concentrate mostly on physics.

How would decision-makers use this “theory”? It seems fairly clear to me: find a collection of “scientists” and ask them to vote. If a majority of these people think that the topic is fringe, it’s fringe, and the decision-maker can reject a project to investigate it, and be safe. Yet people who are decision-makers are hopefully more sophisticated than CYA bureaucrats.

Collins has long written about similar issues. I might obtain and read his books.

As an advisor on science policy, though, what he’s advising isn’t science, it’s politics. The science involved would be management science, not the sociology of science. He’s outside his field. If there is a business proposal, it may entail risk. In fact, almost any potentially valuable course of action would entail risk. “Risky” and “fringe” are related.

However, with cold fusion, we know this: both U.S. Department of Energy reviews, which were an attempt to discover informed consensus, came up with a recommendation for more research. Yet if decision-makers reject research proposals, if journals reject papers without review — Collins talks about that process, is if reasonable, as it is under some conditions and not others — if a student’s dissertation is rejected because it was about “cold fusion,” — though not really, it was about finding tritium in electrolytic cells, which is only a piece of evidence, not a conclusion — then the research will be suppressed, which is not what the reviews purported to want.

“Fringe,” it appears, is to be decided based on opinion believed to be wide, without any regard for specific expertise and knowledge.

“Cold fusion” is commonly thought of as a physics topic, because if the cause of the observed effects is what it was first thought to be, deuterium-deuterium fusion, it would be of interest to nuclear physicists. But few nuclear physicists are expert in the fields involved in those reports. Yet physicists were not shy about giving opinions, too often. Replication failure — which was common with this worki — is not proof that the original report was false, it is properly called a “failure,” because that is what it usually is.

Too few pay attention to what actually happened with N-rays and polywater, which are commonly cited as precedent. Controlled experiment replicated the results! And then showed prosaic causes as being likely. With cold fusion, failure to replicate (i.e., absence of confirming evidence from some investigators, not others) was taken as evidence of absence, which it never is, unless the situation is so obvious and clear that results could not overlook notice. This was a very difficult experiment. It seemed simply to physicists, with no experience with electrochemistry.

I’ve been preparing a complete bibliography on cold fusion, listing and providing access information for over 1500 papers published in mainstream journals, with an additional 3000 papers published in other ways. I’d say that anyone who actually studies the history of cold fusion will recognize how much Bad Science there was, and it was on all sides, not just the so-called “believer” side.

So much information was generated by this research, which went all over the map, that approaching the field is forbidding there is too much. There have been reviews, which is how the mainstream seeks closure, normally, not by some vague social phenomenon, an information cascade. The reviews conclude that there is a real effect. Most consider the mechanism as unknown, still. But it’s nuclear, that is heavily shown by the preponderance of evidence, the contrary view, that this is all artifact, has become untenable, actually unreasonable for those who know the literature. Most don’t know it. The latest major review was “Status of cold fusion, 2010,: Edmund Storms, Naturwissenschaften, preprint.

Decision-makers need to know if a topic is fringe, because they may need to be able to justify their decisions, and with a fringe topic, flak can be predicted.  The criteria that Collins et al seem to be proposing — my study isn’t thorough yet — use behavioral criteria, that may not, at all, apply to individuals making, say, a grant request, but rather to a communit. Yet if the topic is such as to trigger the knee-jerk responses of pseudoskeptics, opposition can be expected.

A decision-maker should look for peer-reviewed reviews in the literature, in mainstream journals. Those can provide the cover a manager may need.

The general opinion of “scientists” may vary greatly from the responsible decisions of editors and reviewers who actually take a paper seriously, and who therefore study it and vet and check it.

A manager who depends on widespread but uninformed opinion is likely to make poor decisions, faced with an opportunity for something that could create a breakthrough. Such decisions, though, should not be naive, should not fail to recognize the risks.

 

ICCF-21 Detailed Agenda

IICF-21 Detailed Agenda =  (original on ICCF-21 web site)

SHORT COURSE SPEAKERS (Sunday 3 June 2018)

  • 10:00 Introduction and Issues, David Nagel
  • 10:40 Electrochemical Loading, Michael McKubre
  • 11:20 Gas Loading, Jean-Paul Biberian
  • 12:00 Lunch
  • 13:30 Calorimetry and Heat Data, Dennis Letts
  • 14:10 Transmutation Data, Mahadeve (Chino) Srinivasan
  • 14:50 Break
  • 15:10 Materials Challenges, M. Ashraf Imam
  • 15:50 Theoretical Considerations, Peter Hagelstein
  • 16:30 Commercialization, Dana Seccombe & Steve Katinsky
  • 17:00 (end)

REGULAR CONFERENCE PROGRAM

18:00 Reception

20:00 Lounge