Naïve Falsification vs Theory-to-Theory Comparison

Note: I have changed my mind about what I say in this post due to communicating with Danny Frederick. I am now convinced that Popper never intended a refutation/falsification to ever be about a single theory and was always intended to be about the system to theories, i.e. the theory plus the background knowledge. This nicely answers all my concerns in this post. However, I still consider that to be an idiosyncratic way to define ‘refutation’ and ‘falsification’ that will invariably mislead people. So this is still Popper’s fault that there are so many misunderstandings of Popper’s epistemology.

In my historical posts, I compared the epistemology of Kuhn and Popper and pointed out that they actually agree on a lot more than people realize. (See part 1, part 2)

When putting those posts up, I came across this challenge that Kuhn made to Popper [1]:

A very different approach to this whole network of problems has been developed by Karl R. Popper who denies the existence of any verification procedures at all. Instead he emphasizes the importance of falsification, i.e., of the test that, because its outcome is negative, necessitates the rejection of an established theory. Clearly, the role thus attributed to falsification is much like the one this essay assigns to anomalous experiences, i.e., to experiences that, by evoking crisis, prepare the way for a new theory. Nevertheless, anomalous experiences may not be identified with falsifying ones. Indeed, I doubt that the latter exist. As has repeatedly been emphasized before, no theory ever solves all the puzzles with which it is confronted at a given time; nor are the solutions already achieved often perfect. On the contrary, it is just the incompleteness of the imperfection of the existing data-theory fit that, at any time, define many of the puzzles that characterize normal science. If any and every failure to fit were ground for theory rejection, all theories ought to be rejected at all times.

Thomas Kuhn, The Structure of Scientific Revolutions, p. 146

Popper’s Theory Isn’t About Naïve Falsification!

Now everyone reading this probably already has the answer to Kuhn’s challenge already rising to their lips. “Not so!” I can hear you shout. “Kuhn is confusing Popper’s epistemology for ‘naïve falsification!’ Of course, you can’t reject a theory for merely having a single failed prediction. That would violate the Duhem-Quine Thesis. And Deutsch already resolved this by pointing out that refutation only happens when you are comparing two theories.”

You may even want to quote Deutsch to back yourself up:

Nothing about the unmet expectation dictates whether [Theory] T or any of those background-knowledge assumptions was at fault. Therefore there is no such thing as an experimental result logically contradicting T..

David Deusch, “Logic of experimental tests”, p. 27)

So there you go! Problem solved! Kuhn just misunderstood Popper. End of story, right?

But of course Kuhn didn’t have Deutsch’s paper available to him. So the question is: Did Popper agree that his falsificationism did not work except in theory-to-theory comparisons? Or is Deutsch actually adding something to Popper’s epistemology that changes it into something new?

Put another way, is the epistemology of Deutsch and Popper the same? Or did Deutsch (perhaps unintentionally) add something to Popper’s epistemology that improved it? Or, alternatively, did Deutsch add something to Popper’s epistemology that is false?

My Own Experience With Popper

Before I attempt to answer that question, let me just give my own honest experience here. After I read The Fabric of Reality – which did not make it clear Popper’s epistemology only worked when comparing two (or more) theories – I went on to read Myth of the Framework, Conjectures and Refutations, and The Open Society and Its Enemies. At no point in any of these books did Popper state the idea that falsificationism only applies to theory-to-theory comparison in such a way that I understood this concept clearly. In fact, it wasn’t until I read Kuhn (the quote above in particular) that it finally occurred to me that Popper’s epistemology simply did not work with naïve falsificationism when dealing with scientific theories. [1]

What is Naïve Falsification?

Naïve falsification is where you have a single theory and, when the theory fails to make even a single correct prediction, we consider the theory to be refuted or falsified based on observations.

The Duhem-Quine Thesis refutes naïve falsification because it points out that there is simply no way to know if a single failed prediction is due to a problem in the theory itself or a problem with implicit background knowledge.

Let’s consider a few quick examples. When the OPERA collaboration measured neutrinos moving faster than the speed of light, contrary to Special (and General) Relativity, it made a huge splash in the media. Was Relativity falsified by this observation? How would you actually know if it was or wasn’t at that point? It seemed likely that there was just some mistake in the measurements. And this was later found to be the case, though it took years to show that.

Another example is the precession of the perihelion of Mercury that we often claim was how we ‘falsified’ Newton’s theory of gravity. But did it really? The mere fact that Mercury’s orbit wasn’t what Netwon predicted didn’t really falsify Newton’s theory because, for all we knew at the time, there might have just been a currently undiscovered body of gravity throwing the prediction off.

In fact, if we’re going to be honest with ourselves, the precession of the perihelion of Mercury really verified Einstein’s General Relativity far more than it falsified Newton’s theory. We had to have an alternative theory that explained the observations better before we were willing to consider Newton’s theory wrong and Einstein’s theory more correct.

Falsification Only Makes Sense in Theory-To-Theory Comparison

When I had the eureka moment that Popper’s epistemology only worked with theory-to-theory comparison, I wrote the following:

Popper’s emphasis on falsification over verification makes perfect sense within an epistemology based on scientific realism. Popper’s main concern is not that there is no sense in which we can ‘verify’ theories as completely correct. Clearly, if we are talking about comparing two (or more) theories, we can certainly verify that one of them is the more correct. This is even true for Popper’s epistemology. What we cannot do is verify that there will never be a better theory. That is to say, we can never verify that a theory is one and the same as reality. (link)

It’s interesting that it is through Kuhn, not Popper, that I came to the realization that Popper’s Conjecture and Refutation philosophy only works properly – at least within the context of scientific theories — within context of having two (or more) competing theories. [1]

Deutsch made this more clear and formalized it in his seminal paper The Logic of Experimental Tests in 2016 – many years after I wrote the above. Today we accept that Conjecture and Refutation is specifically between competing theories. And this seems so obvious to us now, it is hard to even imagine what it would be like to read Popper without that insight. Yet that is exactly what I did and, frankly, I came away with the same “misunderstanding” that Kuhn did and it was Kuhn himself that helped me make the connection between Popper’s falsification and theory-to-theory comparison.

So now that big question: Did Popper understand that his falsification only applied to competing theories but was just less clear than Deutsch? Or did Popper not understand that falsification only applies to theory-to-theory comparison? (And so Deutsch actually added something new to Popper’s theory.)

Popper on Theory-To-Theory Comparison

I think it is beyond doubt that Popper understood how important competition between theories was to his epistemology. Consider the following quotes from Popper:

…a new theory should …conflict with its predecessor – …it should lead to a least some conflicting results. …it should contradict its predecessor…

…a new theory… must always be able to explain fully the success of its predecessor.”

…[this] allow[s] us to decide of any new theory… whether it will be better than the old one… But this means that… we have something like a criterion for judging the quality of a theory compared to its predecessor, and therefore a criterion of progress.  And so it means that progress in science can be assessed rationally.

Myth of the Framework, p. 12

It is clear that the question of preference [towards a theory] will arise mainly, and perhaps even solely, with respect to a set of competing theories; that is, theories which are offered as solution to the same problem.

Objective Knowledge, p. 13

I could easily multiply these, but this should suffice. So the answer to the question “Did Popper understand the importance of competing theories” we can answer a definitive ‘yes.’

But did he understand that falsification with respect to scientific theories specifically required this or it didn’t work?

Here the answer is far less clear. For example, consider this quote from Popper:

In so far as scientific statements refer to the world of experience, they must be refutable; and, in so far as they are irrefutable, they do not refer to the world of experience.

Open Society Vol 2, p. 13

This does not sound to me like a correct understanding of falsification and refutation as explicitly only applying to the comparison between two (or more) theories. In fact, I’m quite tempted to say that the above statement from Popper is just wrong. Scientific theories are not, all alone, refutable or irrefutable. And being ‘irrefutable’ does not imply that they do not refer to the world of experience. They might be ‘irrefutable’ (for the moment) because we don’t yet have a good theory to jump to as per Einstein’s Theory of General Relativity.

Popper Sometimes Intermixes the Language of Naïve Falsification and Theory-to-Theory Comparison

To make matters more confusing, at times Popper seems to intermix statements that seem like they are advocating for naïve falsificationism with language that suggests he wasn’t advocating for it. Consider these quotes:

Can the claim that an explanatory universal theory is true be justified by ‘empirical reasons’; that is, by assuming the truth of certain test statements or observation statements (which, it may be said, are ‘based on experience’)?

My answer to the problem is…: No, we cannot; no number of true test statements would justify the claim that an explanatory universal theory is true.

Can the claim that an explanatory universal theory is true or that it is false be justified by ‘empirical reasons’; that is, can the assumption of the truth of test statements justify either the claim that a universal theory is true or the claim that it is false? [Note: the only difference from the above question is that he is now adding the words “or that it is false.”]

To this problem, my answer is positive: Yes, the assumption of the truth of test statements sometimes allows us to justify the claim that an explanatory universal theory is false.

Objective Knowledge, p. 7

These quotes from Popper above, outside any further context, come dangerously close to being naïve falsificationism of the kind that Deutsch and Kuhn argue is impossible for theories. In fact, my initial read and impression is that he’s specifically teaching naïve falsificationism here. However, the next paragraph suddenly adds:

This reply becomes very important if we reflect on the problem situation in which we are faced with several explanatory theories which compete qua solutions of the same problem of explanation…

Objective Knowledge, p. 7

Before you claim this shows he realized falsification requires theory-to-theory comparison, note that he said ‘becomes very important’ which is the wrong thing to say if this is a critical requirement for your theory. So we’re left with statements that do not make it clear what Popper’s intent was.

Are Misunderstandings of Popper Due to Popper Himself?

It is quite common for people to ‘misunderstand’ Popper as advocating for naïve refutation and falsification. Consider this Youtube video as an example.

In this video, the instructor ‘refutes’ Popper’s epistemology basically by using the Duhem-Quine Thesis. David Deutsch tweeted this video as an example of how people ‘misunderstand’ Popper.

Deutsch, referring to this video, said that the video contains things that “Popper didn’t say but people think he did.” Deutsch goes on to say that “There aren’t straw man arguments but sincere, vast, misunderstandings.” Deutsch voices his opinion that this is, “Caused, I think, by the fact that Popper’s achievement is bigger than most people conceive is possible, so they guess at what he must have meant and arrive at something silly. I did when I first read him.”

Deutsch later says, “If you hear ‘falsificationist’ you’re probably hearing a misunderstanding of Popper’s philosophy, even though Popper and Miller used the term.”

But considering the above quotes, I feel I must disagree that this is merely a misunderstanding of otherwise clear teachings on Popper’s part. Popper did himself no favors in how he chose how to word things. Even Deutsch himself admits he failed to understand that falsification and refutation were solely in theory-to-theory comparisons the first time he read Popper.

The simple truth is that it’s not that hard to state things in such a way that you make it clear theory-to-theory comparison is necessary for falsification of scientific theories. But Popper never did state this unequivocally whereas Deutsch did. So I believe Popper himself is largely responsible for these sorts of misunderstandings.

In retrospect, this is neither that surprising nor really a huge problem. Popper knew he was on to something very important and he was working out the implications all throughout his life. He caught the importance of theory-to-theory comparison via competing theories, but he got there by noticing that under certain circumstances you could ‘falsify’ or ‘refute’ a theory. He never came out and clearly stated this connection clearly but he did leave a legacy of how both concepts (falsification and theory-to-theory comparison) are important. Deutsch and others later plugged the partial hole Popper left. That is now our best theory of knowledge.


[1] In my historical posts, I ended up agreeing with Kuhn on this and went on to claim that Popper’s epistemology only worked within theory-to-theory comparison. I went on to say:

Kuhn is correct that the only sort of refutation that makes sense in science is that of comparison between two (or more) alternatives. In other words, when dealing with theories, science is comparisons between two (or more) paradigms.

From this post

I would note that the first time I published this post was back in February 17, 2011. A month before David Deutsch published The Beginning of Infinity and long before he published his 2016 paper on the Logic of Experimental Tests.

One Reply to “Naïve Falsification vs Theory-to-Theory Comparison”

Leave a Reply

Your email address will not be published. Required fields are marked *