The Problems of Refutation

Though I’ve always agreed with Popper-Deutsch epistemology conceptually, I’ve always been bothered by some aspects of how it seemed to handled refutations. Particularly, I’ve struggled to reconcile some of what Deutsch says about refutations vs what I read in Popper. But I persisted in asking questions until I found some answers to my questions. So to be clear, the problems I’m about to lay out are all solved problems and in future posts, I’ll lay out how Popper solved each one. But at the time I didn’t know how Popper solved them.

Let me lay out what my main concerns used to be:

Kuhn’s Challenge to Popper: Anomalies Do Not Refute Anything

This first one, most Twit Rats know how to respond to. But let’s cover it because it feeds into the next one.

Kuhn pointed out in his book that Popper couldn’t be correct about falsification because all theories always had anomalies yet they never actually refuted any theory. So he felt anomalous experiences couldn’t be identified with falsifying ones:

Nevertheless, anomalous experiences may not be identified with falsifying ones. Indeed, I doubt that the latter exist. … If any and every failure to fit were ground for theory rejection, all theories ought to be rejected at all times.


The standard answer here is Deutsch’s from “The Logic of Experimental Tests” where he points out that, due to the Duhem-Quine thesis, an anomalous experience alone can’t refute a theory and that requires a second rival theory:

in the absence of a good rival explanation, an explanatory theory cannot be refuted by experiment: at most it can be made problematic. If only one good explanation is known, and an experimental result makes it problematic, that can motivate a research programme to replace it (or to replace some other theory). But so can a theoretical problem, a philosophical problem, a hunch, a wish – anything.

The Logic of Experimental Tests, p. 8

The Duhme-Quine thesis says that when you have a counterexample from an experimental prediction, you can’t know if the problem exists with the theory itself or with something in the background knowledge. It seems to follow that you can’t actually ‘refute’ a theory until you have an independently testable rival theory that explains whether the counterexamples were due to a problem with the theory or the background knowledge.

This certainly does handle Kuhn’s objection, though, in all honesty, I wonder how many Popperians before Deutsch even knew of this way to deal with Kuhn’s objection.

But there is a bigger problem now created by this solution.

Tentative Refutation vs Tentative Support

First, as I documented here, there is something unsatisfactory about how Deutsch lays out refutation vs support from within a theory-to-theory comparison. He declares:

The asymmetry between refutation (tentative) and support (non-existent) in scientific methodology is better understood in this way, by regarding theories as explanations, than through Popper’s… own argument from the logic of predictions, appealing to what has been called the ‘arrow of modus ponens’. Scientific theories are only approximately modeled as propositions, but they are precisely explanations.

Logic of Experimental Tests, p. 8

If you are not familiar with modus ponens, it’s just a fancy term in logic for the idea that if p implies q and p is true then q must also be true. It’s the logical “method of affirming” and is considered the opposite of modus tollens (the logical ‘method of denying’) where if p implies q and q is not true then we know p is also not true. [5]

Why is that relevant? Because Popper’s epistemology is rooted in the idea that explanations can be turned into logical statements and then regular logical inference can be used rather than the non-existent inductive inference as most scientists at the time assumed. (i.e. the problem of induction in a nutshell.) Deutsch is here challenging the idea that the best way to look at the asymmetry between refutation and support is due to Popper’s method of turning theories into logical propositions and is instead claiming that by looking at theories as explanations we can derive the same idea.

But remember, as quoted above, Deutsch defines a refutation as being solely within a theory-to-theory comparison with a better rival theory! There are no experimental refutations, according to Deutsch, absent a rival theory.

But if refutation takes place solely in a theory-to-theory comparison, then it’s no longer clear why we Popperians are so adamant (as Deutsch is above) that it’s impossible to find ‘support’ for or ‘verification’ of a theory. Thomas Kuhn was the first to notice this problem when he said:

But [Karl Popper’s] falsification, though it surely occurs, does not happen with, or simply because of, the emergence of an anomaly or falsifying instance. Instead, it is a subsequent and separate process that might equally well be called verification since it consists in the triumph of a new paradigm over the old one. Furthermore, it is in that joint verification-falsification process that the probabilist’s comparison of theories plays a central role.

Thomas Kuhn in The Structure of Scientific Revolutions, p. 147

To understand this issue concretely, consider the Arthur Eddington expedition to test between Newtonian physics and General Relativity. As you’ll recall, during an eclipse the stars near the sun move position due to the gravity of the sun bending the light. I found it hard to understand why this was specifically a ‘refutation’ of Newton and but for some reason not ‘support’ for General Relativity. Why isn’t it both? I (and apparently Kuhn) just couldn’t see anything wrong with saying something like “The Eddington experiment supported Einstein’s theory of relativity.” That seemed like a completely true statement. So was it really true that support was non-existent in scientific methodology?

Now, of course, I fully understand that you can’t ‘support’ or ‘verify’ a theory in any absolute sense. There will someday be a new theory (Quantum Gravity?) that makes predictions identical to General Relativity yet isn’t General Relativity. But when someone says “The Eddington expedition supports General Relativity” I really don’t believe that they secretly mean “We will never again find a new theory that makes this particular prediction because now we know that General Relativity is Truth with a capital T!” In fact, quite the contrary, I feel quite certain that they actually mean something more like “Of the two theories of gravity that actually exist (Newton and General Relativity) this experiment came out in favor of General Relativity.” If that’s all they mean, then support is possible in science precisely because it is nothing more than the logical inverse of refutations.

This fact bothered me enough that I started to ask other Twit Rats (and later FBook Rats) about this. And the answer I (at least at first) almost uniformly received was that it should be obvious to me that the word ‘refutation’ always implied a purely tentative resolution but that the words ‘support’ (or ‘verification’) always implied a claim to absolute certainty. Given fallibilism, we know absolute certainty is impossible thus both support and verification are impossible while refutation–which doesn’t imply absolute certainty (or so this argument claims)–is possible because it is always only tentative.

But this explanation is clearly word essentialism. There is no reason that I can see that person might use the word ‘verification’ or ‘support’ and mean only ‘tentative verification’ or ‘tentative support.’ In a theory-to-theory comparison (the only situation in which refutation happens according to Deutsch!), the language of refutation and the language of verification are identical. It is trivial to convert between the two.

Responding to this problem by claiming “verification is impossible” because “it’s about justifying a theory as true” just won’t work because it’s trivial to simply change the word ‘verification’ to no longer be about justifying a theory as true and, instead, making it merely about one theory being better than its current known rivals.

This is because verification (in a theory-to-theory comparison) is only “impossible” in the same sense that refutation is equally impossible, namely that it can’t be absolutely certain. If you could non-tentatively refute a theory that would itself be a form of justificationism and would violate fallibility too. So this Twit Rat argument is failing to compare like ideas. To understand what I mean, let’s imagine two forms of refutation as well as two forms of verification as I illustrate in this chart:

Note the symmetry in the above chart. To explain this a bit further we now have:

Absolute Refutation/Falsification – The ability to prove beyond doubt that a theory is refuted absent even the existence of a competing theory via some observation. This is a false idea.

Absolute Verification/Support – The ability to prove beyond doubt that a theory is true via some observation. This is a false idea.

Tentative Refutation/Falsification – The ability to show that one theory has more problems than another. This is a true idea.

Tentative Verification/Support – The ability to show that one theory has fewer problems than another. This is a true idea.

If we are going to accept that ‘refutations’ are always in a theory-to-theory comparison and are always tentative at best then I just can’t see why ‘support’ (or ‘verification’) can’t also be in a theory-to-theory comparison, be tentative, and effectively mean little more than ‘this theory is more correct than the competing theories that we currently know about.”

More to the point, I’m fairly certain that is a very common way in which people do really use those terms.

So at a minimum, there is something wrong with the idea that ‘refutations’ are always tentative while ‘support’ and ‘verifications’ are always absolute. Let’s give this fallacy a name to make it easier to refer to:

The Absolute Verification Fallacy: The idea that the asymmetry between refutation and verification is because the word ‘verification’ always means ‘verified with certainty’ whereas the word refutation always means ‘refuted tentatively’. Since one can never be certain, verification is impossible but refutation is possible.

Consider that Popper himself never makes any claim that the word ‘certainty’ always implies absolute certainty. And, in fact, he specifically argues that this is not the case:

There is a commonsense notion of certainty which means briefly, ‘certain enough for practical purposes’. When I look at my watch, which is very reliable, and it shows me that it is eight o’clock, and I can hear that it tickets (an indication that the watch has not stopped), then I am ‘reasonably certain’ or ‘certain for all practical purposes’ that it is fairly close to eight o’clock.

Objective Knowledge p. 78

Instead, Popper argues that certainty is about pragmatic belief. It simply means “I feel certain enough for this situation.”

It’s hard for me to accept that the words ‘support’ and ‘verification’ always imply absolute certainty but that the word ‘certainty’ doesn’t.

But the above argument, if correct, seemed to reintroduce the symmetry between refutation and support that Deutsch (and Popper) denied exists. So are Popper and Deutsch wrong that an asymmetry exists? Where is the misunderstanding in the above argument?

Corroboration and Support

What makes this even worse is that Karl Popper himself has a really clear view on what he calls “corroboration” of a theory — namely an attempt to falsify a theory by experiment that fails to refute the theory. And he didn’t mind using the term ‘support’ as a synonym for it:

…It should be noted that a positive decision [in a test] can only temporarily support the theory, for subsequent negative decisions [in future tests] may always overthrow it. So long as theory withstands detailed and severe tests… we may say that… it is ‘corroborated

LoSD, p. 10 (Italicized emphasis is Popper’s. Underlined is mine.)

And this isn’t just a simple choice of words either because Popper speaks of corroboration not as a binary but as a matter of degrees:

Instead of discussing the ‘probability’ of a hypothesis we should try to asses what tests, what trials, it has withstood; that is, we should try to assess how far it has been able to prove its fitness to survive by standing up to tests. In brief, we should try to assess how far it has been ‘corroborated’.

The Logic of Scientific Discovery, p. 248

Popper had no problem at all talking about ‘degrees of corroboration’ either. (See LoSD, p. 249)

In other words, I believe that most of the time when a person says something like “The Eddington expedition experiment supports (or even ‘increases support’ for) General Relativity” that (unless they are specifically inductivist philosophers) they almost always mean something identical to what Popper meant when he’d say something like “The Eddington expedition experiment corroborates (or even ‘increases the degree of corroboration’ for) General Relativity.”

In other words, Popper effectively treats corroboration as identical to how most people see “support” because it is seen as, in some sense, strengthening the theory in question by degree when it survives a sincere attempt to falsify it. In fact, ‘corroboration’ is so much like ‘confirmation’ and ‘support’ that Popper originally actually called it “degrees of confirmation” rather than “degrees of corroboration” and only changed the term when he found the Vienna Circle tended to misunderstand him as agreeing with them that you could make theories more probable.

…I myself used the term ‘confirmation’ for a time in a number of my publications. Yet… the association of the word ‘confirmation’ did matter, unfortunately, and made themselves felt: ‘degree of confirmation’ was soon used… as a synonym (or ‘explicans’) of ‘probability. I have thereefore abandoned it in favour of ‘degrees of corroboration.’

The Logic of Scientific Discovery, p. 249 in footnote 1.

But the key point here is that there is nothing magic about the word ‘corroboration’ as it is merely a synonym for confirmation. As with most synonyms, it carries with it slightly different connotations and was a better word for Popper’s purposes (in English anyhow) due to not being quite so strongly associated with probability and absolute certainty. But Popper clearly had no problem with referring to confirmation of a theory within his own epistemology at one point even if it wasn’t his preferred term. [1]

Many fans of Deutsch have claimed that Popper’s views of corroboration are mistaken. Brett Hall makes that claim here. Since Deutsch has declared the idea of ‘support’ non-existent and corroboration is really just support in disguise, I can understand where Brett is coming from. If support really is non-existent in science then it makes perfect sense that Popper must have been mistaken about the existence of corroboration.

But here, again, we bump into a new problem. Is it really true that you can refute a theory with a single observation–when within a comparison with a rival theory–and no observation after that matters? Consider the example of Newton vs Einstein again. Einstein only created his theory because there were known problems (in the Deutsch sense) with Newton’s theory. There was the perihelion of mercury and the Michelson–Morley experiment, for example. From the moment Einstein created his theory it already had these two observations that Newton couldn’t explain at the time (though widely believed at the time to be only problems with background knowledge) while Einstein’s theory could explain those observations. In fact, scientists (other than Einstein) so strongly believed that these observations were consistent with Newtonian physics (because they were believed to just be a problem in our background knowledge) that it wasn’t until the Eddington expedition experiment that a ‘crisis’ (to use the Kuhnian term) took place. [2]

But according to Deutsch’s view (above) a theory is ‘refuted’ the moment we have a single observation that counters Newton’s theory plus a rival theory (General Relativity) that explains the observation. So really Newton’s theory was (according to this view) refuted before any tests were run since we already had two observations that ran counter to Newton’s theory but were explained by Einstein’s theory.

But isn’t that how every theory must be as theories are only created (according to Popper) to solve known problems. So if Brett is correct that corroboration is unnecessary–as seems to follow from Deutsch’s interpretation of Popper–then we just proved that testing theories is an unnecessary thing to do as it can in no way further refute a theory nor give better support for the rival theory.

This makes the Eddington expedition experiment unnecessary for Einstein’s theory. Yet clearly people did not start to rally around Einstein’s theory (and indeed didn’t even think of it as a problem!) until it has been corroborated by an experiment that wasn’t already a known problem at the time Einstein’s theory was created. [2]

But it can’t be correct that testing is unnecessary as empirical tests are fundamental to Popper’s epistemology. In fact, empirical testing is required by Popper’s methodology for an explanation to avoid being considered an ad hoc explanation.

Worse yet, this would make String theory a superior theory to QM and GR because both of those theories have a refuting case (the need for quantum gravity) whereas String theory has no refuting cases. Yet it just seems intuitively obvious that String theory is an inferior theory due to its complete lack of testability and thus potential corroborations.

To put a finer point on it, if Brett’s view of corroboration is correct, then testing a theory with new experiments is completely unnecessary. Yet that must be incorrect because we can and should care about how many tests a theory has survived, just like Popper claimed. But then can we really say that something like support is a non-existent concept within Popper’s epistemology?

Theories that Require Verification

Another problem I struggled to resolve at first was theories that required verification. Let’s take the theory “Bigfoot exists” as an example. How would you go about ‘refuting’ this theory? In fact, you can’t because it’s a theory that is an existential statement rather than a universal statement. Popper himself admits that existential statements can only be verified–namely by producing bigfoot for scientists to study.

…a set of singular observation statements (‘basic statements, as I called them) may at times falsify or refute a universal law; but it cannot possibly verify a law, in the sense of establishing it. Precisely the same fact may be expressed by saying that [a basic statement] can verify an existential statement… but that it cannot falsify it.

Realism and the Aim of Science, p. 181

Make sure you read that carefully. Popper is here admitting that just as you can only refute/falsify a universal law (i.e. a logical statement of the form: ‘For all X…’) and not verify it, you can likewise only verify an existential statement (i.e. a logical statement of the form ‘There exists an X…’) and you can’t falsify/refute it. [3]

He goes on to say:

…if we can describe an empirical test as an attempted falsification or a search for a negative instance (of statement a), then we can also describe it as an attempted verification or as a search for a positive instance (of the statement non-a).

Realism and the Aim of Science, p. 182

This should put to rest the popular idea that the problem with verification is that it’s always understood as absolute whereas refutations never are understood to be absolute. Popper is openly admitting that all falsifications of universal law are logically identical to verifications of existential statements and vice versa. So Popper himself refutes the Absolute Verification Fallacy.

This is why a theory like ‘bigfoot exists’ can’t be falsified (as Popper points out above!) but only verified.

This all being the case, why do we emphasize refutation over verification then? Does it just depend on the nature of the theory being discussed?

Penrose’s Challenge To Popper

It’s tempting to claim that the bigfoot example isn’t a true scientific theory and thus outside of Popper’s original epistemology which was specifically about scientific theories. And I’ve had people argue this to me. [4] But it’s not like we can’t easily find equivalent examples in science. In fact, Penrose gave two such examples in this post here. To summarize one of the examples, supersymmetry is a theory in physics that is widely (though not universally) accepted. It makes a prediction that particles should all have superpartners. Penrose explains:

The theory predicts ‘superpartners’ for all the observed fundamental particles of Nature, but none of these has so far been observed. The reason that they have not, according to supersymmetry theorists, is that a symmetry-breaking mechanism (of unknown nature) causes the superpartners to be so massive that the energies needed to create them are still beyond the scope of present-day accelerators. With increased energy capabilities, the superpartners might be found, and a new landmark in physical theory would be thereby achieved, with important implications for the future. But suppose that still no superpartners are actually found. Would this disprove the supersymmetry idea? Not at all. It could (and probably would) be argued that there had simply been too much optimism about the smallness of the degree of symmetry breaking, and even higher energies would be needed to find the missing superpartners.

Roger Penrose in Road to Reality, p. 1020-1021

Penrose is correct: it is not possible to falsify this theory but it can be, at least in principle, verified. So Penrose offers this up as an example of how falsification is not the boundary for science after all. Is he correct? How would Popper look at this example? Was he just unaware of String theory and thus didn’t realize that some scientific theories require verification and not falsification?

The problem this poses for Popper is actually a bit more serious than it first appears. The idea is this: suppose we did find a superpartner that verified supersymmetry and String theory. In that particular case, the negation of the theory (“there are no superpartners”) isn’t a theory we’re particularly interested in. The true competitor was actually regular old physics without symmetry. But that theory didn’t actually predict there were no superpartners. It just said nothing at all about the subject. Does this then refute Popper’s view that science is only about refutations?

Best Theories Without Refuting the Competitor

And lastly, as I discuss in this post, there are many real-life cases of finding evidence that strengthens a theory without actually refuting its competitor. In my post, I give two examples.

In one example, the police decide to investigate a murder starting with the victim’s wife–even though they hadn’t ruled out (i.e. refuted) anyone else yet. This is because they knew the wife had a motive. At this point, the theory that someone else committed the murder isn’t refuted at all but the theory that the wife committed the murder has been strengthened due to the existence of a motive. So the police, rightly, start their investigation with the wife since she is the best theory they currently have.

In the second example, a DNA test is used to show that the child of a slave was actually the child of the slave’s owner. In this case, the argument is based on probability. It is unlikely–but not impossible–that the child is someone else’s child. But since the genetic marker is rare but known to be in the slave owner’s DNA people would rightly assume that this is now the best theory even though the rival theory hasn’t been refuted.

Resolving the Problems of Refutation

Based on my past posts on this subject, I’ve found that people tend to get upset when I raise questions like this. So let me be clear: I am NOT claiming these are actual problems with Popper’s epistemology. In fact, I know they are not problems for Popper’s epistemology because Popper covers all but one of them in his books. (And the one that he doesn’t cover–or at least I haven’t yet found where he does–is handled well by Deutch’s “The Logic of Experimental Tests.”)

So the problems I’ve raised above are known to be pseudo-problems. They are not real problems with Popper’s epistemology at all though some of the answers aren’t well known.

But do you know what the correct response is to each of the above? The simple truth is that most Popperians do not seem to be aware of how Popper responded to each of these challenges. And that is unfortunate because the above set of problems goes a long way towards explaining why Popper’s epistemology hasn’t caught on with many in the scientific community since they naturally have the same set of concerns. It is too easy to find problems like the above but it’s hard to find good explanations for how Popper dealt with each problem.

So my goal in the upcoming posts is to address each of these individually and explain how Popper (or Deutsch) handled these problems.

In the meantime, take a stab at answering each of these challenges and see how your answers compare to Popper’s.


[1] To keep the quote from Popper short and on point, I left out the part where he explains that he always preferred the term ‘corroboration’ over ‘confirmation’ and only went with ‘confirmation’ because Carnap translated it that way originally and Popper strongly believed that words did not matter, so it shouldn’t matter which term he used.

[2] Jagdish Hattiangadi, a former student of both Popper and Kuhn, in this episode of the Popperian Podcast discusses how Kuhn was wrong that a crisis always proceeds a paradigm change due to General Relativity. It was only after the Eddington expedition that a crisis was created.

I’ve had argued to me that if Einstein was unaware of, say, the Michelson–Morley theory at the time he created General Relativity that then this should be considered just as good a test as the Eddington expedition test. And since there is some evidence that Einstein was not aware of their theory at the time, this seems like a fair question. Yet clearly real scientists in the real world did not see it that way at the time. Were they merely mistaken? I’m going to argue they were not. Yes, in principle, if Einstein was unaware of the Michelson–Morley experiment at the time he created his theory, then the fact that his theory explained it could be thought of as a valid experiment corroborating his theory. But the key problem here is that it is literally impossible to know for sure that Einstein was unaware of the Michelson–Morley experiment and that it didn’t at least subconsciously inform his theory. So it makes sense that scientists at the time saw the Eddington experiment as a better corroboration of Einstein’s theory.

[3] This quote from Popper is a good example of how he does often equate refutation and falsification. However, see also “The distinction between falsification and refutation in the demarcation problem of Karl Popper” by Nicolae Sfetcu. So perhaps Popper does not always equate refutation and falsification together.

[4] This objection also ignores the fact that it is possible to generalize Popper’s epistemology to work outside of scientific theories. See footnote 1 on this post for discussion.

[5] I admit to being a bit confused here by Deutsch’s reference to modus pollens. Typically Popper referenced modus tollens, not modus pollens, when talking about how his epistemology used deductive logic. See, for example, LoSD, p. 19. I think what Deutsch is saying here is that the logic of modus pollens is used to make predictions. But then we use modus tollens to use the experimental result to refute the theory.

Leave a Reply

Your email address will not be published. Required fields are marked *