Tentative Verification vs Tentative Refutation

The most famous example of the asymmetry of refutation and verification is that of the black swan. There is no number of white swans you can find that verify that all swans are white but a single black swan refutes the theory that all swans are white.

But see if you can apply that logic to these proposed theories:

  • “There are black swans”
  • “Not all swans are black or white”
  • “There are purple swans”

How would you test each of these theories using refutation?

I’ve posed various versions of this question to many different Popperians and expressed my opinion that verification probably made sense in these cases. The single most common answer I get is “verification is impossible because it is about justifying something as true.

But this answer is, at a minimum, unsatisfying. For one thing, it doesn’t even answer the question being posed — I still have no idea how to refute any of these ‘theories.’ But there seems to be a more serious problem with this answer which I’m going to outline in this post.

Theory-to-Theory Comparison

In a previous post, I pointed out that Deutsch argued that all falsification/refutation must happen within the context of a theory-to-theory comparison. In a separate post, I challenged Deutsch’s idea in the case of “hypotheses” (when meant as distinct from well-tested “theories”) and didn’t come to any definitive conclusions for that particular case. But, so far, I agree with Deutsch at least in the case of well-tested scientific theories. I do not expect to ever see a refutation [1] of a scientific theory without having a second theory to compare it to. And finally, in this post, I gave an example of Roger Penrose challenging Popper’s falsification criteria for science using the theory of supersymmetry and asked what role “corroborating evidence” played in Popper’s epistemology.

So with one possible caveat — in the limited case of ‘hypotheses’ (i.e. early untested conjectures) — it would seem so far that Deutsch’s view of all refutations of a scientific theory happening within a theory-to-theory comparison seems to be correct. But this leads to the very reason why the ‘standard answer’ I get to my question above can’t be correct.

How It All Began

Recall from my past posts that Kuhn claimed that Popper was wrong that falsification existed.

Nevertheless, anomalous experiences may not be identified with falsifying ones. Indeed, I doubt that the latter exist. … If any and every failure to fit were ground for theory rejection, all theories ought to be rejected at all times.

THOMAS KUHN, THE STRUCTURE OF SCIENTIFIC REVOLUTIONS, P. 146

But if you accept Deutsch’s view that ‘refutation’ and ‘falsification’ solely apply to a situation with competing theories and are wrong when applied to a single stand-alone theory you realize that Kuhn actually made a stronger point against Popper than I have been letting on up to this point. I am now going to come to grips with the problem.

Tentative Refutations

Popperians today (at least the ones that came to Popper via Deutsch) easily understand that ‘refutation’ and ‘falsification’ can’t happen in a vacuum and require two or more theories. But they still uniquely use Popper’s language of refutation and falsification. But I’m now going to demonstrate that Kuhn was correct that this is not strictly necessary.

To understand this, let’s look at this particular quote from Kuhn from above: “Nevertheless, anomalous experiences may not be identified with falsifying ones. Indeed, I doubt that the latter exist.”

Kuhn actually has a significant insight here that – again today – is now considered accepted wisdom among Deutsch Popperians but that, as I showed, Popper shows only limited awareness of. It is the idea that there simply is no such thing as a clear-cut refutation or falsification. [2]

Consider what Deutsch says here:

Accordingly, all decisions to modify or reject theories are tentative: they may be reversed by further argument or experimental results. And no such event as ‘accepting’ a theory, distinct from conjecturing it in the first place, ever happens.

Logic of Experimental Tests, p. 7

A crucial test – the centrepiece of scientific experimentation – can, on this view, take place only when there are at least two good explanations of the same explicandum (good, that is, apart from the fact of each other’s existence). Ideally it is an experiment such that every possible result will make all but one of those theories problematic, in which case the others will have been (tentatively) refuted.

Logic of Experimental Tests, p. 9

Here Deutsch clearly states that what an experimental “refutation” really is is a test between theories where we make all competing theories problematic by comparison but never actually refutes a theory by itself. Deutsch makes this so clear that if you came to Popper via Deutsch there is a good chance that you simply think of Popper’s theory as implying that all refutations are in a theory-to-theory comparison and are always tentative.

But what you probably missed is that this is conceptually the same as saying refutation and falsification — as Kuhn uses the term! — never take place at all. Yes, you found a better theory that has fewer problems and you decided to tentatively adopt it. But for all you know you may have to revisit the other “refuted” theories later. That’s what we mean by ‘tentatively’ refuted and it’s what Kuhn means by problems (or anomalies) not being the same a falsifying a theory.

This happens in real life. One of the best examples was light wave theory vs light particle theory. Lightwave theory had been tested and survived the test and light particle theory was problematic by comparison. So it (nearly) died out as a theory. That is, until Einstein came up with a test (the photoelectric effect) that light particle theory could pass but was problematic for lightwave theory. At that point, light particle theory and lightwave theory were both problematic but both superior to the other in some ways. The theories were what Kuhn calls incommensurable. (Thus proving that Kuhn was correct that incommensurability does exist.) This tension was resolved by coming up with a brand new theory about light — quantum theory.

So Kuhn was correct that falsifying observations are not to be identified with problems (what Kuhn calls anomalies) and that, in fact, falsifying observations don’t exist at all in any absolute sense. This represents an honest-to-goodness problem that Kuhn found with Popper’s theory — or at least with the popular interpretation of Popper’s theory. The Deutschian version of falsification/refutation just makes more sense: you don’t really ever refute anything — you just prefer some theories over others because they have fewer problems. Problems aren’t refutations, they are just problems.

The Language of Verification

But even Deutsch’s improved formulation leaves a lingering new problem. Let me attempt to explain it succinctly. The problem is that when you are assuming your refutations/falsifications are all tentative and only within the comparison of competing theories (as Deutsch assumes) then there is no longer a reason to prefer the language of refutation at all.

Consider this quote from Deutsch to illustrate the point:

The asymmetry between refutation (tentative) and support (non-existent) in scientific methodology is better understood in this way, by regarding theories as explanations, than through Popper’s (op. cit.) own argument from the logic of predictions, appealing to what has been called the ‘arrow of modus ponens’. Scientific theories are only approximately modeled as propositions, but they are precisely explanations.

Logic of Experimental Tests, p. 8

Here Deutsch follows Popper’s use of language. Within a comparison of competing theories, we can ‘tentatively refute’ a theory by having a better theory with fewer problems. But ‘support’ for a theory does not exist.

But is that correct? Say you have two theories – let’s say Newton’s Theory and Einstein’s General Relativity – and you perform Arthur Eddington’s experiment to discriminate between the two. One might just as rightly say that “Newton’s theory was refuted in favor of Einstein’s” as they might say “Einstein’s theory was verified as more correct than Newton’s.”

In a theory-to-theory comparison, the language of refutation and the language of verification are identical. It is a trivial to convert between the two.

As noted in the intro to this post, the objection that I usually get when bring this up is that “verification is impossible” because “it’s about justifying a theory as true.” But there are two problems with this claim. First, verification is only “impossible” in the same sense that refutation is, namely that it can’t be absolute. If you could non-tentatively refute a theory that would itself be a form of justificationism and would violate fallibility. Second, this claim is failing to compare like ideas. To understand what I mean, let’s imagine two forms of refutation as well as two forms of verification as I illustrate in this chart:

Note the symmetry in the above chart. To explain this a bit further we now have:

Absolute Refutation/Falsification – The ability to prove beyond doubt that a theory is refuted absent even the existence of a competing theory via some observation. This is a false idea.

Absolute Verification/Support – The ability to prove beyond doubt that a theory is true via some observation. This is a false idea.

Tentative Refutation/Falsification – The ability to show that one theory has more problems than another. This is a true idea.

Tentative Verification/Support – The ability to show that one theory has fewer problems than another. This is a true idea.

Deutsch’s claim (in the quote above) that “support” is “non-existent” is a comparison not of Tentative Refutation to Tentative Verification but between Tentative Refutation to Absolute Verification. So the comparison is unfair.

This is the real reason that support is non-existent; it’s non-existent in the same sense that refutation is non-existent. Both are non-existent in the absolute sense. But both are tentatively possible so long as we are talking about a theory-to-theory comparison. In actuality, they are exactly the same concept worded in a different way. All tentative refutations in a theory-to-theory comparison must of necessity be tentative support for the other theory.

In fact, the ‘standard response insisting verification is impossible confuses two concepts: Verification vs Verificiationism.

It is specifically ‘Verificationism’ and ‘Justificationism’ that are false philosophies that are about justifying something as true. “Tentative support” or verification in a theory-to-theory comparison claims neither of those — it only claims that of the available theories, one is better than the others.

In fact, Karl Popper himself claimed that “justification” (in some sense) is true in the setting of theory-to-theory comparison: [3]

“Scientific theories can never by ‘justified’, or verified. But in spite of this, a hypothesis A can under certain circumstances achieve more than a hypothesis B… I even stressed very early that questions of truth or validity, not excluding the logical justification of the preference for one theory over another (the only kind of ‘justification’ which I believe possible)

Objective Knowledge, p. 67, emphasis mine

Conclusion

If Deutsch is correct about theory-to-theory comparison being a requirement (and we’ve seen he is correct at least for well-accepted scientific theories), then the Critical Rationalist war on the language of verification is at least partially misguided. Even Popper admitted that we could think of “justification” (and by extension “verification” and “support”) as correct if we were thinking in terms of theory-to-theory comparisons and demonstrating that one theory achieves more than another.

So why do Popperians prefer the language of refutation over the language of verification? Why do we say the two are asymmetrical?

As it turns out, Popper actually was aware of the above issue I just expressed and even had a response to it. In a future post, I’ll lay out Popper’s full response to that question and why he felt there was a valid asymmetry between verification and falsification.

What’s interesting for this post, however, is that he did not resort to the ‘standard answer’ of claiming that verification is impossible because it is about justifying knowledge as absolutely correct! In fact, his writings are full of examples of how verifying things (particularly results of experiments) makes perfect sense.

Notes

[1] Danny Frederick (in a Facebook conversation with me) suggested that we think of “refutation” as always refutation of a theory + background knowledge — never a theory by itself. He then proposed using the word “rejection” for when we have a second theory available to replace the old one.

If by “refutation” we specifically always mean the theory and its (sometimes tacit) background knowledge then refutations again become possible. Without this special way of understanding the word “refutation,” as far as I can see, refutation may just be impossible among true scientific theories that are well-tested. So I approve of Danny’s solution to the problem in principle.

However, I don’t prefer his wording. I think the word “refutation” strongly implies refutation of a theory alone without its background knowledge and that is how people read and understand Popper today. I think coming up with a special technical understanding of the term ‘refutation’ like this will only lead to further confusion.

Instead, I propose we use Deutsch’s equivalent term: problem. If you have an observation that doesn’t match a theory you can never know if that problem is with the theory itself or not. All you can actually do is conjecture how to rethink either the background knowledge or the theory. Presumably, you’ll try to do both. You haven’t truly refuted anything at this point. What you have is a problem, not a refutation. (In the way people normally think of those terms.)

Later, when you have an alternative theory to consider that explains how to solve the problem, you can now (using Danny’s term) reject the previous theory — tentatively. But this is now exactly equivalent to saying we tentatively refuted the theory. Thus Danny’s ‘refutation’ = Deutsch’s ‘problem and Danny’s ‘rejection’ = Deutsch’s ‘refutation.’ Deutsch’s wording seems preferable to me, but I believe both are saying the same thing.

[2] Probably the strongest example of Popper admitting that naive falsification was impossible is this quote here. So we know Popper was aware that absolute refutations would violate fallibilism.

[3] Andrew Crawshaw argues that this is referring solely to logical probing (like we’re doing in this post) where we compare the consequences of theories. In context, this is probably a correct understanding of Popper’s intent here. But it does demonstrate that the language of justification makes sense in some cases and doesn’t have to refer only to absolute certainty. Logical probing doesn’t give us absolute certainty either, yet Popper had no problem referring to this as a form of ‘justificationism’ he agreed with if it showed one theory as preferable to another.

Leave a Reply

Your email address will not be published. Required fields are marked *