In the next few posts, I’m going to address what I call the ‘problem of refutation.’ I’ve hinted at this problem in past posts, particularly here and here. But see also Penrose’s example of the problem of refutation as well as these examples here. I should note that this has taken me quite a while to work out the problem but I now feel I have found the correct solutions within Popper’s epistemology that resolves all the problems. My intent is to share those solutions. But since most people don’t even recognize that a problem ever existed, I need to first layout the problems.
My introduction to Popper was through David Deutsch’s books. Probably Deutsch’s best treatment of Popper’s epistemology is his paper “The Logic of Experimental Tests, Particularly of Everettian Quantum Theory.” What a title!
In this paper, Deutsch lays out his interpretation of Popper’s epistemology — at least as it applies to empirical scientific theories  — like this:
Scientific Explanations: “I take a scientific theory to be a conjectured explanation (explanatory theory) of some aspects of the physical world – the explicanda of the theory – that is testable (I shall elaborate what that means below) by observation and experiment. A scientific explanation is a statement of what is there in reality, and how it behaves and how that accounts for the explicanda [the thing being explained]. (p. 5)
Bad Explanations vs False Explanations: “Let me distinguish here between a bad explanation and a false one. Which of a theory’s assertions about an explicandum are false and which are true (i.e. correspond with the physical facts) is an objective and unchanging property of the theory (to the extent that it is unambiguous). But how bad or good an explanation is depends on how it engages with its explicanda and with other knowledge that happens to exist at the time, such as other explanations and recorded results of past experiments. An explanation is better the more it is constrained by the explicanda and by other good explanations.” (p. 6)
“…an explanation is bad (or worse than a rival or variant explanation) to the extent that… (i) it seems not to account for its explicanda; or (ii) it seems to conflict with explanations that are otherwise good; or (iii) it could easily be adapted to account for anything (so it explains nothing).”
“It follows that sometimes a good explanation may be less true than a bad one (i.e. its true assertions about reality may be subset of the latter’s, and its false ones a superset). Since two theories may have overlapping explicanda, or be flawed in different respects, the relations ‘truer’ and ‘better’ are both only partial orderings of explanations.” (p. 7)
Problems vs Refutation: “When we, via arguments or experiments, find an apparent flaw, conflict or inadequacy in our theories, that constitutes a scientific problem and the theories are problematic (but not necessarily refuted yet…)” (p. 7)
“In this view a scientific theory is refuted if it is not a good explanation but has a rival that is a good explanation with the same (or more) explicanda. So another consequence is that in the absence of a good rival explanation, an explanatory theory cannot be refuted by experiment: at most it can be made problematic. If only one good explanation is known, and an experimental result makes it problematic, that can motivate a research programme to replace it (or to replace some other theory). But so can a theoretical problem, a philosophical problem, a hunch, a wish – anything.” (p. 8)
“But in any case, the existence of a problem with a theory has little import besides, as I said, informing research programmes – unless both the new and the old explicanda are well explained by a rival theory. In that case the problem becomes grounds for considering the problematic theory tentatively refuted.” (p. 10)
“That is known as the Duhem–Quine thesis (Quine 1960). It is true, and must be distinguished from the Duhem–Quine problem, which is the misconception that scientific progress is therefore impossible or problematic.” (p. 11)
Theories and Background Knowledge: “In any experiment designed to test a scientific theory T, the prediction of the result expected under T also depends on other theories: background knowledge, including explanations of what the preparation of the experiment achieves, how the apparatus works, and the sources of error. Nothing about the unmet expectation dictates whether T or any of those background-knowledge assumptions was at fault. Therefore there is no such thing as an experimental result logically contradicting1 [Theory] T, nor logically entailing a different ‘credence’ for T . But as I have said, an apparent failure of T’s prediction is merely a problem, so seeking an alternative to T is merely one possible approach to solving it. And although there are always countless logically consistent options for which theory to reject, the number of good explanations known for an explicandum is always small.”
“Note that even if [Theory] T is the culprit, merely replacing it by ~ T cannot solve the problem, because the negation of an explanation (e.g. ‘gravity is not due to the curvature of spacetime’) is not itself an explanation. Again, at most, finding a good explanation that contradicts T can become the aim of a research programme. (p. 11)
Purpose of Experiments: “An important consequence of this explanatory conception of science is that experimental results consistent with a theory T do not constitute support for T. That is because they are merely explicanda. A new explicandum may make a theory more problematic, but it can never solve existing problems involving a theory (except by making rival theories problematic…).” (p. 8)
Ad Hoc Explanations Are Not Allowed: “…to meet criterion (iii) above, it must not be protected from such a refutation by declaring it ad hoc to be unproblematic. Instead any claim that its apparent flaws are not real must be made via scientific theories and judged as explanations in the same way as other theories.” (p. 10) 
The Asymmetry of Refutation and Support: “The asymmetry between refutation (tentative) and support (non-existent) in scientific methodology is better understood in this way, by regarding theories as explanations, than through Popper’s (op. cit.) own argument from the logic of predictions, appealing to what has been called the ‘arrow of modus ponens’. Scientific theories are only approximately modelled as propositions, but they are precisely explanations.” (p. 8)
The Nature of Tests/Experiments: “I now define an objective notion, not referring to probabilities or ‘expectation values’, of what it means for a proposed experiment to be expected to have a result x under an explanatory theory T. It means that if the experiment were performed and did not result in x, T would become (more) problematic. Expectation is thus defined in terms of problems, and problems in terms of explanation, of which we shall need only the properties (i)-(iii)”
“A test of a theory is an experiment whose result could make the theory problematic. A crucial test – the centrepiece of scientific experimentation – can, on this view, take place only when there are at least two good explanations of the same explicandum (good, that is, apart from the fact of each other’s existence). Ideally it is an experiment such that every possible result will make all but one of those theories problematic, in which case the others will have been (tentatively) refuted.”
“It will suffice to confine attention to problems arising from tests of fundamental theories in physics. And of those problems, only the simplest will concern us, namely when an existing explanation apparently does not account for experimental results. This can happen when there seems either to be an unexplained regularity in the results (criterion (i) above), or an irregularity (i.e. an explanation’s prediction not being borne out – criterion (ii) above). So, if the result of an experiment is predicted to be invariably a1 , but in successive trials it is actually a5 , a29, a1 , a3…, with no apparent pattern, that is an apparent irregularity. If it is a5 , a5, a5 , a5…, that is apparently both an apparent unexplained regularity and an irregularity.” (p. 9)
Refuting Theories By Their Failure to Explain: “Suppose for simplicity that two mutually inconsistent theories, D and E, are good explanations of a certain class of explicanda, including all known results of relevant experiments, with the only problematic thing about either of them being the other’s existence. Suppose also that in regard to a particular proposed experiment, E makes only the everything-possible-happens prediction (my discussion will also hold if it is a something-possible-happens prediction) for results a1 , a2 ,…, while D predicts a particular result a1 . If the experiment is performed and the result a2 is observed, then D (or more precisely, the combination of D and the background knowledge) becomes problematic, while neither E nor its combination with the same background knowledge is problematic any longer (provided that the explanation via experimental error would be bad – Section 6 below).
“Observing the result a1 , on the other hand, would be consistent with the predictions of both D and E. Even so, it would be a new explicandum which, by criterion (i) above, would raise a problem for the explanation E, since why the result a1 was observed but the others weren’t would be explained by D but unexplained by E. Note that if it were not for the existence of D, the result a1 would not make E problematic at all.” (p. 12)
“If the experiment is then repeated and the result a1 is obtained each time, that is an apparent regularity in nature. Again by criterion (i), E then becomes a bad explanation while D becomes the only known good explanation for all known results of experiments. That is to say, E is refuted (provided, again, that experimental error is a bad explanation). Although E has never made a false prediction, it cannot account for the new explicandum (i.e. the repeated results a1 ) that its rival D explains.” (p. 12)
“Thus it is possible for an explanatory theory to be refuted by experimental results that are consistent with its predictions.”
“It follows that under E, the string of repeated results a1 is expected not to happen, in the sense defined in Section 2, even though E asserts that, like every other sequence, it will happen (among other things). This is no contradiction. Being expected is a methodological attribute of a possible result (depending, for instance, on whether a good explanation for it exists) while happening is a factual one. What is at issue in this paper is not whether the properties ‘expected not to happen’ and ‘will happen’ are consistent but whether they can both follow from the same deterministic explanatory theory, in this case E, under a reasonable scientific methodology. And I have just shown that they can.” (p. 13)
Let me now summarize Deutsch’s interpretation of Popper’s epistemology:
Scientific theories are explanations about reality, and are not merely predictive. They should not be judged solely about their empirical content, but also by how good the explanation actually is; where “good explanation” means that the theory is strongly constrained by potential observations (i.e. the explicanda) and by other good explanations. To be constrained by potential observations is what we mean by empirical testability.
When dealing with empirical theories that are testable, we start with known problems. A problem may be 1) an experiment that contradicts what the theory predicted, 2) an experiment that has a non-explained regularity, 3) a contradiction to another good explanation. 
However, a problem is not the same as a refutation because it is unknown if the problem is a problem with the theory [Theory T] in question or something in the (possibly only implicit) background knowledge. (This is known as the Duhem-Quine thesis).
A refutation therefore only takes place once we have a second rival explanation (Theory R) available that potentially resolves the problem with theory T. For this reason, refutations only take place when a second theory exists and are only problems before that point.
Experiments are ways of creating problems for an existing theory and testing between rival theories to tentatively refute one of the competing theories.
We methodologically disallow ad hoc saves of a problem because, otherwise, all theories would be bad explanations because ad hoc saves can be had for the asking. An ad hoc save is defined as an explanation with no testable consequences other than the one problem it was meant to explain.
 This qualifier that we’re specifically talking about empirical scientific theories is necessary here. It is entirely possible to generalize Popper’s epistemology to work outside of empirical scientific theories. However, Popper’s original theory as laid out in his book, The Logic of Scientific Discovery, (similarity in name to Deutsch’s paper is not an accident!) was specifically about how empirical science works and thus is specifically about empirical content of a theory.
Deutsch’s paper is specifically about experimental tests. (i.e. “In this paper I shall be concerned with the part of scientific methodology that deals with experimental testing.” p.6) So he is also writing specifically about empirical science as well.
But both Popper and Deutsch acknowledge that it is possible to apply the concept of conjecture and criticism to non-empirical theories as well. Probably the best-known generalization of Popper’s theory was laid out by Donald Campbell in “Evolutionary Epistemology” which Popper strongly endorsed. However, this theory has a number of important flaws that need to be addressed still.
 By far and away, this is the most common mistake made by Twit Rats. They have a long history of not realizing that ad hoc explanations used to save a theory from a problem are not allowed under Popper’s methodology. This is unfortunate because Popper’s epistemology that excludes his rule against ad hoc explanations is exactly identical to not using Popper’s epistemology at all because then every theory becomes easy to vary because, as Popper put it, ad hoc explanations can be had for the asking. See discussion here.
The way you know if a “counter refutation” is ad hoc or not is, according to Popper, if the new explanation (being used as a counter refutation) has any testable consequences other than specifically the problem it’s meant to save. (As Popper explains here.)
The second most common mistake Twit Rats make is thinking that just because Popper’s epistemology can be applied to non-empirical theories that therefore it is okay to offer a non-empirical theory as a rival to an empirical theory. Under Popper’s epistemology, we always favour empirical theories over non-empirical theories.
 I have yet to find #2 (a non-explained regularity) anywhere in Popper and I think it might be unique to Deutsch, though I’m not sure. But as I’ll show in future posts, it’s necessary to make sense of some aspects of Popper’s epistemology so, regardless of if this is unique to Deutsch or not, it is correct.