On Observation, Experiment, Theory, Trust and Belief

A discussion on lenr-forum led me to this musing. Eddington was quoted there.

“Never trust an experimental result until it has been confirmed by theory.”.

This led me to look at a 1978 New York times article, For a Nobel Math Prize, by Paul R. Chernoff, a mathematician, of course.

The case for a Nobel prize in mathematics rests on the fact, paradoxical yet undeniable, that research at the foundations of science ultimately yields far greater benefit to mankind than work on immediate practical problems.
…
Theory is always most fundamental, theoretical advances the most incisive. Often theory tells us which experimental questions to pose in the first place. Physicists recognize this. Einstein to Heisenberg: “It is the theory that determines what can be measured.” More flippantly, Sir Arthur Eddington: “Never trust an experimental result until it has been confirmed by theory.”

If theoretical science is fundamental, no science is more fundamental than mathematics. It is extraordinary that mathematical ideas, cultivated originally merely because they were beautiful, so often turn out to be useful — indeed essential — in understanding the physical world.

And then,

Nothing is more abstract than mathematical logic, and no abstract philosophical discovery is more fundamental than Kurt Gödel’s proof than any axiomatic system is essentially incomplete: No mere mechanical procedure can generate all mathematical truths.

These comments lead me to consider what is “fundamental,” what is “belief” or “trust,” and how do these operate? These questions are about epistemology and ontology, and are, in themselves, fundamental. We use these words — and that to which they refer — commonly, without much attention to their foundations, as if it is obvious. These are not merely about “science,” but about how we live our lives, day to day, constantly, almost without cease. Normally, anyway.

The question led me to the Wikiquote page for Eddington. (link to present version). This is a collection of nuggets from this scientist and philosopher.

There once was a brainy baboon,
Who always breathed down a bassoon,
For he said, “It appears
That in billions of years
I shall certainly hit on a tune”.

Eddington was having fun, and that’s how I lived my life. I actually played a keyboard like a “brainy baboon,” with no attention to what keys would be “proper” to press, I just played like a child (as a daughter later did with the violin, just whatever she felt like), though, to be fair, within an accompaniment that the keyboard set up and within rhythm. (It is very difficult to play outside of rhythm.) I learned this, of high practical import to a musician: if I repeated something, it sounded far better than in isolation, so “mistakes” can generate new music. The sense of freedom from just playing without regard to whether or not it was “good,” was worth all the time I spent doing this. My idea was that if I did this for ten years, I’d have created a new realm of musical mastery. However, I had other things to do …. so I never proved that, not with keyboard, anyway.

It is impossible to trap modern physics into predicting anything with perfect determinism because it deals with probabilities from the outset.
…
It is also a good rule not to put overmuch confidence in the observational results that are put forward until they are confirmed by theory.

this is the real quote, and it comes from this context:

But are we sure of our observational facts? Scientific men are rather fond of saying pontifically that one ought to be quite sure of observational facts before embarking on theory. Fortunately those who give us this advice do not practice what they preach. Observation and theory get on best when they are mixed together, both helping one another in the pursuit of truth. It is a good rule not to put overmuch confidence in a theory until it has been confirmed by observation. I hope I shall not shock the experimental physicists if I add that it is also a good rule not to put overmuch confidence in the observational results that are put forward until they have been confirmed by theory.

(Eddington’s emphasis.)

This is key here, and takes us to epistemological roots. The life of the mind begins with instinct, I assume, but develops through observation. However, “observation” is, first of all, sensory data, and then is what we remember (consciously or not) of it. Further, I understand, for efficiency we do not ordinarily remember much of the primary experience, but rather, we abstract it, interpreting it to create summaries, and over the years, these can shift. In addition, the interpretation process may be immature. We drew conclusions about life when we were very young, often emotionally reactive, and these conclusions then created modes of relationship to life. (This is basic to the training I have been through.) Taking this into science, what is Eddington talking about when he means “observational facts,” that we should not be “overmuch confident” in? I don’t think he means what I would use those words to mean. If I do an experiment, say I am measuring the deviation of a star from its normal position in the celestial sphere when the star appears close to the sun in an eclipse, my “observations facts” may consist of photographs I took. Okay, I’m referring to Eddington’s own famous work.

There are two aspects to that work. There is the experimental record, which generates data. That data is subject to measurement error. However, the collected data is the collected data.. Measurements are always approximate, never exact, so never completely “true” in some sense. But the recorded measurements are the recorded measurements (and photos taken are photos taken). This is primary observation. It is, all in itself, a kind of truth. Yes, we can imagine some mass delusion, so we could argue with this as “absolute truth.” It isn’t absolute, but it is what we live with and routinely trust, even if we are very cautious.

What is not so reliable is how we interpret this data. Eddington is referring to interpretation, not primary observation. He, of course, is also referring to the possibility of error in measurement, as well as in recording measurement.

And then we come to interpretation, and theory is interpretation. In my training, “the human being is a meaning-making machine.” Sometimes people take this as a claim that there is something wrong with this, but “wrong” is an interpretation. In fact, this human trait is a key to our survival, and it has obvious value for that. What has happened with science is that interpretation has become, for some, far more sophisticated, becoming highly successful, in many areas, at what is probably the evolutionary root, prediction, of what actually happens. Yet if we “believe” these interpretations, put “overmuch confidence in them,” we shut down the possibility of growth — and the recognition of possible interpretive errors, caused by a limited experiential data set.

This all has extensive application in life and science. At best, this is not merely an abstract idea, it can become a lived reality, and that is the point of the training I have been through. It’s not about theories, and in the training they say, first day, “what we are about to tell you is not the truth.” What is transmitted — much more by present example and observation than by declaration — is a set of tools, often expressable in words, but designed for function, not for “truth.” The point of these “distinctions,” as they are called, is not to create some new philosophy that is “truth,” — that is what is being set aside — but to create ways of looking at experience that have been found to open up new realms of possibility, in lives that have, very often, become otherwise stuck in what has been unquestioningly accepted from past experience. That past experience is not rejected or made wrong, but it may well be limited.

I will eventually get to cold fusion, but first, back to Eddington, more from Wikiquote.

Phys. Yours is a strange subject. You told us at the beginning that you are not concerned as to whether your propositions are true, and now you tell us you do not even care to know what you are talking about.
Math. That is an excellent description of Pure Mathematics, which has already been given by an eminent mathematician [Bertrand Russell].

Imagine this guy — a freakin’ genius — showing up on, say, lenr-forum.com, or, even funnier, ecat-fraud.com. On lenr-forum, he would have a few supporters. On ecat-fraud.com, a phalanx of pseudoskeptics would ridicule everything he wrote.

We used to think that if we knew one, we knew two, because one and one are two. We are finding that we must learn a great deal more about ‘and’.

In the training, new possibilities are discovered in the “unknown unknown,” what we don’t know that we don’t know.

We have found a strange foot-print on the shores of the unknown. We have devised profound theories, one after another, to account for its origins. At last, we have succeeded in reconstructing the creature that made the footprint. And lo! It is our own.

Walt Kelley, from another perspective: “We have met the enemy and he is us…”

Science aims at constructing a world which shall be symbolic of the world of commonplace experience. It is not at all necessary that every individual symbol that is used should represent something in common experience or even something explicable in terms of common experience. The man in the street is always making this demand for concrete explanation of the things referred to in science; but of necessity he must be disappointed. It is like our experience in learning to read. That which is written in a book is symbolic of a story in real life. The whole intention of the book is that ultimately a reader will identify some symbol, say BREAD, with one of the conceptions of familiar life. But it is mischievous to attempt such identifications prematurely, before the letters are strung into words and the words into sentences. The symbol A is not the counterpart of anything in familiar life.
…

Never mind what two tons refers to. What is it? How has it entered in so definite a way into our experience? Two tons is the reading of the pointer when the elephant was placed on a weighing machine. Let us pass on. … And so we see that the poetry fades out of the problem, and by the time the serious application of exact science begins we are left only with pointer readings.

Pointer readings. When we write them in our lab notebook, or when they are written in our memories, what is written is what is written. That’s a “reality.”

On the general topic, I actually found this before the NYT article: Michael Crichton. Google thought it contained the quote, but it doesn’t. The cached page returned an error. It was probably edited out. Makes me want to get a copy of the book. Crichton and I live in the same universe, but have, to some degree, different sets of experiences. Like, duh!!!

I take a stand that there is One Reality. I think he’d agree, and so would Eddington. Ekam sat vipra bahudha vadanti. Distinguished here, is Reality or Truth, and its expression in language. The possible language is practically infinite in variety.., and none of it confines or captures reality, but it may point to it or describe some of it.

Now, cold fusion. At this point, “cold fusion” is a set of experimental observations. What would it mean to “put overmuch confidence” in them? These observations are part of a far larger set of observations.

The original observations were a result of an examination of heat evolved from palladium deuteride, loaded into a deuterium/palladium cathode at levels not previously explored, and even thought to be impossibly high. Pons and Fleischmann had suspected that predictions of fusion rate, based on approximations resulting from the use of 2-body analysis, might be imprecise, that much more sophisticated analysis of the multibody problem might be necessary. However, they thought that any deviation would be small, and probably below their detection capacity. Nevertheless, they decided to look, and designed calorimetry that was as sensitive as possible, the precision of their calorimetry is reputed to be 100 microwatts (which is rarely found in calorimetry). And then their experiment melted down. They then worked in secrecy for about five years, until circumstances forced them to reveal their findings prematurely, they were obviously not ready. They were not overconfident in their findings, I suspect, but found themselves defending them anyway, and defensive reactions have a way of taking over.

The resulting mess created a mass of observational anecdotes. Each one is, relatively speaking and as to most of this work, weak. The massive effort that was created, created ample opportunities for the file drawer effect to arise, which is then a ready skeptical argument. “There must be some mistake” is very common. Why? Well, supposedly, the results conflict with theory.

But experimental results do not, in themselves, conflict with theory, conflict arises in interpretation. We interpret results according to a set of theories we are using, and our assumed theories and our interpretation may differ from those of others. Mike McKubre has stated his interpretation of the mass of data, including much that he personally collected.
His ICCF-19 talk.

Given the power and energy densities of the heat effect claimed by Fleischmann and Pons, only one of two rational possibilities
existed in 1989:
• Fleischmann and Pons were wrong in their excess heat determinations.
• Nuclear reactions occur in metallic lattices by mechanisms and with product distributions different from similar reactions in free space.

However, “fusion” was a word heavily laden with expectations. Instead of focusing on the experiment, on the observations, the major focus from most scientists was on theory and then on what theory would predict. I.e., if it was “fusion,” the ready thinking was, assuming d-d fusion, then there would be copious neutrons, tritium, etc., and if helium, gammas. The new observation was stuffed into and interpreted through an existing model, and when it didn’t fit the model, we had “there must be some mistake,” which could be asserted about any result that doesn’t fit accepted interpretations of theory.

So Eddington’s first caution: “It is a good rule not to put overmuch confidence in a theory until it has been confirmed by observation.’ But wait, wasn’t existing theory not only confirmed by observation, but heavily confirmed? There was a subtle distinction that was overlooked. Theories are applied to circumstances, and may have been tested in many circumstances, and, in them, may have produced precise predictions which are confirmed. But does this apply to all possible circumstances? It is clear that it doesn’t. It may, it would not be surprising if it does, but it also would not be impossible that it won’t.

In any case, “fusion” was an interpretation of the Pons and Fleischmann results. It was not their actual observation. What they actually observed was a measurement of anomalous heat, heat without explanation that they could recognize as chemistry, and they were chemists.

The obvious and core question was their calorimetry. Was it accurate? Unfortunately, Pons and Fleischmann did not disclose all they knew with their announcement, much was secret, and much still is. It was not announced and was not known how difficult the experiment was, and even Pons and Fleischmann did not realize how sensitive what we will now call the Anomalous Heat Effect (AHE) was to precise and even unknown material conditions, very difficult to control. Further, additional confirmation, needed to reconcile an AHE with the basic laws of physics, in particular the laws of thermodynamics, did not exist until Miles in 1991 discovered the heat/helium correlation, creating a strong indication that the reaction product was helium, even though gammas were not seen. The energy from that conversion, from deuterium to helium, matched that transmutation theory.

Even though Miles was confirmed, this was not enough to turn the mainstream around. The finding of the heat/helium correlation allowed a kind of confirmation by theory, but the theory was not complete — and still is not complete. That energy release is mechanism-independent, any such transmutation must release that energy, but the mechanism for cold fusion is still unknown (as to anything accepted and confirmed through prediction and thorough testing, and at this point, theory mostly serves to confuse until there is a solid and widely-accepted bedrock of fundamental observation with clear (but partial) explanation.

Before that finding, those who accepted cold fusion could be accused of Eddington’s other caution: “not to put overmuch confidence in the observational results that are put forward until they have been confirmed by theory.”

However, the “theory” that helium is being produced at a certain and possibly significant ratio to heat is testable (and was tested, and is being tested currently with higher precision, I understand).

Eddington is not suggesting the rejection of experimental results based on lack of theoretical understanding. This is about “overmuch confidence,” which could show up in many different ways. Eddington is not suggesting that if there appears to be an heat anomaly, the experimentalist should toss out his or her temperature record and other recorded facts because of lack of theory. Rather, until they have been reconciled, both theory and experimental observations (and especially the interpretation of those observations) will best be, as it were, suspended, as avenues for possible further investigation. With cold fusion, much work was done that did not seriously address the basic issues. Pons and Fleischmann did extremely precise calorimetry. Few even attempted to match their precision. Miles was confirmed, and even with some level of increased precision, but the basic finding, then, of anomalous heat correlated with helium was not rigorously demonstrated by the kind of extensive work with many measurements — and many researchers — that might be needed to overwhelm the “must be making some mistake” argument. (Most reports are of few measurement, sometimes as few as one, and experiments varied wildly so that comparing results could be problematic. There are choices to be made, and choices are poison when looking for correlations. One can create them with choices.)

In any ordinary field, the level of confirmation of Miles (in spite of what I wrote above, it could be considered substantial) would have been adequate, but given the massive information cascade that Pons and Fleischmann had been disproven, it was not adequate. Some of the most important work was not published in scientific journals, some (Apicella et al, 2004) was not even published as such at all, being only a conference presentation.

(My paper, Current Science, 2015, on heat/helium as a reproducible and confirmed experiment.)

Eddington was not placing theory as superior to observation. This is clear: observation is a huge pile of data. By itself, it is of little use. “Theory” is the process by which we organize that data, and interpret it, discovering patterns that can be used for prediction of future data. Data is the source, not theory; reality creates data (when we are careful), but we create theory. Theory does not exist in reality, I will claim. This might appear to differ from Eddington:

It is difficult for the matter-of-fact physicist to accept the view that the substratum of everything is of mental character. But no one can deny that mind is the first and most direct thing in our experience, and all else is remote inference — inference either intuitive or deliberate.

I see Eddington as referring to experiential reality and interpretation, whereas when I refer to “reality,” I am referring to an assumed source that is truly universal and that exists unconditionally, that is a source of our experience, but that is not simply our experience. Our experience varies, whereas reality is single. (This is a stand, I do not assert it as provable, but do claim that accepting it has effects.) The difference in view may not be great, in practice.