Demarcation: What Does it Mean to Be Empirical?

Popper’s demarcation criterion is widely understood as an attempt to find the boundary between science and pseudo-science. But in his paper “Addressing Three Popular Philosophic Mythos about Karl Popper’s Demarcation Criteria” Nathan Oseroff points out that this is actually a misunderstanding of Popper’s theory. This misunderstanding is one of the main reasons Popper is often dismissed by others.

So what is Popper’s real intent then? The first thing to realize is that Popper’s criterion is not normative. It is not about deciding what gets the honor of being called science. In fact, Popper’s demarcation problem was not really about the demarcation of “science” and “non-science,” per se, but really about the demarcation between empirical and non-empirical theories:

This criterion of demarcation between empirical and non-empirical theories I have also called the criterion of falsifiability or the criterion of refutability. It does not imply that irrefutable theories are false. Nor does it imply that they are meaningless. But it does imply that, as long as we cannot describe what a possible refutation of a certain theory would be like, that theory may be regarded as laying outside the field of empirical science.

The Myth of the Framework, p. 88 (emphasis mine)

Yes, metaphysics is also (by definition) non-empirical. And, yes, typically pseudo-science is not empirical either (though it’s often dressed up as if it is.) But those are secondary purposes.

People have tried to make Popper’s epistemology into something it isn’t: a boundary between science and non-science. But they are making this far more complicated than it actually is. Popper’s point is actually far simpler — and far more profound — than people realize.

What Is an Empirical Theory?

What Popper’s actually saying is this: when we call something ’empirical’ it specifically means it is possible to build a test for that theory:

I… admit a system as empirical or scientific only if it is capable of being tested by experience.

LoSD, p. 18

Popper’s theory of demarcation does not define the boundary between ‘science’ and non-science per se but instead defines the boundary between empirical and non-empirical. Its relationship to science is that science strives for empirical theories. This is why Popper often (but not always) specifies that he means ’empirical science’ rather than just ‘science.’ Popper himself, in an interview with Scientific American, confirms that this is how he understood his own demarcation criteria:

[Popper’s] falsification concept, he said, is a criterion for distinguishing between empirical and non-empirical modes of knowledge. Falsification itself is “decidably unempirical”; it belongs not to science but to philosophy, or “meta-science,” and it does not even apply to all of science.

Scientific American, “The Paradox of Karl Popper,” August 22, 2018

Most new theories start out their life as untestable and thus tautologically unempirical. Think String theory here. The goal of science is to turn that theory into a testable theory either by improving the theory itself or improving our ability to test things via better instruments. This–and only this!–is the sense in which Popper’s theory is about the demarcation of science and non-science. But real science spends a lot of time dealing with non-empirical theories–including String theory–trying to turn them empirical.

So it’s really technically incorrect to claim that falsification/refutation is the demarcation between science and non-science. The correct formulation of Popper’s epistemology is that falsification/refutation is the demarcation between empirical and non-empirical.

But is Popper correct about this? For example, why can’t a verification be considered empirical? Most people would think of verifications as empirical because you can theoretically test an existential statement. Consider the theory “Bigfoot exists” again. Can’t you test that theory by actually going out and finding Bigfoot?

And since we’ve now dispensed with the Absolute Verification Fallacy and we’ve seen that verifications are a necessary part of Popper’s epistemology (namely in doing experiments) perhaps you are wondering now why Popper insists so strongly that only falsifications/refutations should count as empirical. (Or maybe you already know why?)

Why is Refutation/Falsification Required to Be Empirical?

We’ve seen in a past post that Popper’s asymmetry between verification and refutation was really about a logical asymmetry between universal statements (‘For all x’) vs existential statements (‘There exists an X’). Scientists are more interested in universal statements due to their greater power and even when they are interested in theories that are existential statements, there is always an inverse universal statement of greater interest due to the fact that all existential statements, when negated, because universal statements. But many find this explanation unsatisfying. It makes Popper’s criterion of empirical theories seem a bit arbitrary. In fact, Popper admits that it is:

My Criterion of demarcation will… have to be regarded as a proposal for an agreement or convention. As to the suitability of any such convention opinions may differ… The choice of that purpose most, of course, be ultimately a matter of decision, going beyond rational argument. …Thus I freely admit that in arriving at my proposals I have been guided… by value judgements.

LoSD, p. 15

However, I sort of feel Popper is being modest here. Let’s look at his actual argument for why feels his demarcation criterion is the best one to essentially define what we mean by ’empirical.’ First, he explains how to go about analyzing his demarcation criterion–by looking at its ability to solve problems in the theory of knowledge:

There is only one way, as far as I can see, of arguing rationally in support of my proposals. This is to analyze their logical consequences: to point out their fertility–their power to elucidate the problems of the theory of knowledge.

LoSD, p. 15

Then he argues that this really comes down to the very nature of science itself happening to be about laws, which are modeled logically via universal statements that can, as we’ve seen, only be refuted:

That theories of natural science, and especially what we call natural laws, have the logical form of strictly universal statements, thus they can be expressed in the form of negations of strictly existential statements or, as we may say, in the form of non-existence statements or ‘there-is-not’ statements. … In this formulation we see that natural laws might be compared to ‘proscriptions’ or ‘prohibitions.’ They do not assert that something exsists or is the case; they deny it. They insist on the non-existence of certain things or states of affairs: they rule them out. And it is precisely because they do this that they are falsifiable.

LoSD, p. 48

Induction and Going from Existential Statements to Universal Laws

Remember that Popper’s purpose was to show that ‘inductive inference’ was unnecessary to science. Inductive inference is the non-existent logical rule that you can take existential statements and generalize to a universal statement. For example, that by seeing multiple white swans you can generalize to the rule ‘all swans are white.’ Yet science consists of what Popper calls basic statements which is really just a fancy word for observations. And basic statements take the form of singular statements in logic which are always a kind of existential statements. i.e. the observation of a white swan is equivalent to ‘there exists a white swan.’ So there is a need to go from existential statements to universal statements in some way or else empirical theories would be impossible. So Popper points out that it is logical refutations that take existential statements and tell us something about universal laws.

[Universal statements] are never derivable from singular statements, but can be contradicted by singular statements. Consequently it is possible by means of purely deductive inferences… to argue from the truth of singular statements to the falsity of universal statements. Such an argument to the falsity of universal statements is the only strictly deductive kind of inference that proceeds as it were, in the ‘inductive direction’; that is, from singular to universal statements.

LoSD, p. 19 (emphasis mine)

So Why Aren’t Existential Theories Empirical?

Given this thinking, Popper then explains why he, by convention, bans existential theories from being considered empirical:

Strictly existential statements, by contrast, cannot be falsified. No singular statement (that is to say, no ‘basic statement’, no statement of an observed event) can contradict the existential statement, ‘There are white ravens”. Only a universal statement could do this. On the basis of the criterion of demarcation here adopted I shall therefore have to treat strictly existential statements as non-empirical or metaphysical.

LoSD, p. 48

This convention by Popper has certainly been one of the main issues other philosophers and scientists have had with Popper’s epistemology. Popper admits this, at first, seems like a dubious convention:

This characterization may perhaps seem dubious at first sight and not quite in accordance with the practice of empirical science. By way of objection, it might be asserted (with justice) that there are theories even in physics which have the form of strictly existential statements.

LoSD, p. 48

If the above statement sounds familiar, it should. Penrose’s challenge to Popper contained two particularly good examples of how physics theories can and do have existential theories. Popper gives an example of his own how there are theories that predict the existence of elements of certain atomic numbers. But then he goes on to explain why this doesn’t pose a problem for his convention of refutation as the demarcation for empirical theories.

But if the hypothesis that an element of a certain atomic number exists is to be so formulated that it becomes testable, then much more is required than a purely existential statement. For example, the element with the atomic number 72 (Hafnium) was not discovered merely on the basis of an isolated purely existential statement. On the contrary, all attempts to find it were in vain until Bohr succeeded in predicting several of its properties by deducing them from his theory. But Bohr’s theory… are strictly universal statements.

LoSD, p. 49

Let’s now look again at the Bigfoot example I mentioned in a previous post.

The theory “There is a Bigfoot” can only be verified. Period. But that theory is not an empirical theory! Why? Because you can’t actually set up a program of tests for that theory.

Let’s say you decide to ‘test’ the theory that Bigfoot exists. So you go into the woods and try to capture one. You never even see one. What have you determined? Nothing at all. Bigfoot might exist or might not exist still. So there is no program of tests even in principle that could test this theory. That is why Popper says it is ‘non-empirical.’ [1]

Why By Convention We Must Exclude Purely Existential Theories

Popper gives another reason why we must exclude purely existential theories from counting as ’empirical.’ He uses the following example of a purely existential theory:

‘There exists a finite sequence of Latin elegiac couples such that, if it is pronounced in an appropriate manner at a certain time and place, this is immediately followed by the appearance of the Devil–that is to say, of a man-like creature with two small horns and one cloven hoof.’

Conjecture and Refutations, p. 337

Popper goes on to point out that “clearly, this untestable theory is, in principle, verifiable.” So why exclude it via his demarcation criterion? Why not just allow such theories to count as empirical by convention? (Since we’re only excluding them by convention.) Why not, as the positivist argued, just claim it is empirical but false?

…why… should anybody who takes it for empirical think that it is false? Empirically, it is irrefutable. No observation in the world can establish its falsity. There can be no empirical grounds for its falsity. Moreover, it can be easily shown to be highly probable: like all existential statements, it is in an infinite (or sufficiently large) universe almost logically true… Thus if we take it to be empirical, we have no reason to reject it, and every reason to accept it and believe in it–especially upon a subjective theory of probable belief.

Conjectures and Refutations, p. 337-338

To put this concretely, you either have to by convention exclude purely existential theories as being empirical or you have to accept an infinity of such theories–most of which are patently ridiculous–as empirical and likely true.

Penrose (Partially) Refuted

This explains why Penrose’s challenge to Popper is actually in agreement with Popper. As I discussed in this post, Penrose points out that String theory has a central ingredient called supersymmetry that includes a prediction about superpartners for particles. Because this theory (String Theory + supersymmetry) can only be verified because it merely predicts the existence of superpartners and it doesn’t explain how to test for them. In essence, the theory could be worded like this: “String Theory predicts the existence of superpartners.” Because this is an existential statement rather than a universal statement, it is not possible to falsify this theory.

Penrose’s point could be summarized as:

Science includes String theory and String theory includes supersymmetry which makes a prediction that can’t be falsified. Thus science is not demarcated by falsification like Popper thought it was.

But once you realize that Popper’s actual demarcation isn’t between science and non-science per se–but instead between empirical and non-empirical–then there is no disagreement here after all between Penrose and Popper. They both are agreeing that supersymmetry is not an empirically testable theory. Penrose merely fell into the common trap of misunderstanding Popper’s demarcation criteria. Nothing more. He has not created a problem for Popper’s theory. [1]

Spatio-Temporal Constraints Also Required

Don’t make the mistake here of thinking that therefore existential statements are non-empirical but universal statements are. Note that “Bigfoot does not exist” is just as much an unempirical theory as “Bigfoot exists” because they are in fact the same theory. Simply rewording it as a universal statement via negation does not change this fact. To be empirical is not merely to be a universal statement, it has to include reasonable spatio-temporal constraints that make a program for testing possible. As Popper puts it: [2]

Strict or pure statements, whether universal or existential, are not limited as to space and time. They do not refer to an individual, restricted, spatio-temporal region. This is the reason why strictly existential statements are not falsifiable. We cannot search the whole world in order to establish that something does not exist, has never existed and will never exist. It is for precisely the same reason that strictly universal statements are are not verifiable. Again, we cannot search the whole world in order to make sure that nothing exists which the law forbids. Nevertheless, both kinds of strict statements, strictly existential and strictly universal, are in principle decidable…

LoSD, p. 49

The Verifiability of Predictions: Inter-subjectivity

Now accepting by convention falsifiability as the demarcation of empirical vs non-empirical theories does lead to a bit of a paradox that many would see as contradictory. What I mean is that since observations themselves (basic statements [3]) take the form of existential statements and existential statements can only be verified don’t we still bump into the problem of science being non-objective because it’s really all about our subjective sense impressions (i.e. empiricism)? And since our senses (and everything) are uncertain and fallible, how can we be sure an observation actually contradicts a universal law?

Popper’s solution to this problem is what he calls inter-subjective testing which is nothing more than that a community of scientists can all reproduce and observation.

…the objectivity of scientific statements lies in the fact that they can be inter-subjectively tested. … Only when certain events recur in accordance with rules or regularities, as is the case with repeatable experiments, can our observations be tested–in principle–by anyone. We do not take even our own observations quite seriously, or accept them as scientific observations, until we have repeated and tested them. Only by such repetitions can we convince ourselves that we are not dealing with a mere isolated ‘coincidence’ by with events which, on account of their regularity and reproducibility, are in principle inter-subjectively testable.

LOSD, po. 22-23 (Emphasis Popper’s)

A “scientifically significant physical effect” must be one that “can be regularly reproduced by anyone who carries out the appropriate experiment in the way prescribed.” (LoSD, p. 23-24). This is why Bigfoot sightings are not scientific. If you personally see Bigfoot up close, you may personally now believe in Bigfoot. But the scientific community isn’t going to accept that observation until you manage to capture Bigfoot so that any scientist in the community can come to observe Bigfoot for themselves.

Popper does clarify, however, that observations (basic statements) might be indirect. For example, if you heard there was a Bigfoot at the zoo but it escaped or died, you could reasonably go talk to the caretakers or ask for videos, etc. This might–in some circumstances–be enough to tenativley convince a community that this is a valid basic statement.

But basic statements are never certain and may turn out to be incorrect–perhaps even the entire scientific community was making the same mistake and thus believed in a false basic statement. But that doesn’t really pose a problem for Popper’s epistemology. The mere fact that a community accepts a basic statement that is in contradiction to a theory is a problem that must be resolved even if the resolution turns out to be that the entire community had made a mistake. Thus reproducible basic statements are sufficient to constitute a problem and start the conjecture and criticism process.

But if basic statements aren’t certain either, how can we ever say they refute or falsify a theory? Popper explains that this is just a matter of convention. We simply introduce a special methodological rule as follows:

As to falsification, special rules must be introduced which will determine under what conditions a system is to be regarded as falsified. We say that a theory is falsified only if we have accepted basic statements which contradict it. This condition is necessary, but not sufficient; for we have seen that non-reproducible single occurrences are of no significance to science. Thus a few stray basic statements contradiction a theory will hardly induce us to reject it as falsified. We shall take it as falsified only if we discover a reproducible effect which refutes the theory.

LoSD, p. 66

Popper refers to a statement accepted by a community inter-subjectively as an ‘accepted’ basic statement. This is why we need to accept “a rule which tells us that we should not accept stray basic statements…” but instead only accept “basic statements in the course of testing theories of raising searching questions about these theories...” (LoSD, p. 88)

Put more simply, if a community accepts a basic statement (even if they do so wrongly) we now tentatively accept the ‘system’ (or ‘theory’) as falsified. [4] We now have a scientifically interesting problem to solve. It might later turn out to be, say, a problem with our instruments or the like, but that is still a legitimate problem to solve.


[1] There is another problem raised by Penrose’s example that I will cover in a future post. Namely, let’s say you really did find a superpartner particle, thus verifying Supersymmetry and String theory by an experimental test. Why would this not be considered an ’empirical theory’ then? And why would that be a problem for regular physics (absent supersymmetry) since it makes no specific predictions that superpartners do not exist? So finding a superpartner does not ‘refute’ regular physics, it only ‘verifies’ Supersymmetry and String theory. Is this a counterexample to Popper’s epistemology?

[2] Keep in mind that “Bigfoot exists” and “Bigfoot does not exist” are not representative of real scientific theories and are only marginally explanatory in the first place. It’s just an example of why you can’t test a purely existential statement, not an example of a scientific theory. For that reason, this is also an example of where an explanation and its negation are ‘equally’ explanations (or rather equally not very good explanations.)

[3] Popper says: “Basic statements are therefore… statements asserting that an observable event is occurring in a certain individual region of space and time.” LoSD, p. 85

[4] There is a problem here I’ll deal with in a future post. Namely that a ‘system’ and a ‘theory’ aren’t really the same thing in most people’s minds. A ‘system’ meaning all connected background theories. Which actually gets falsified? The ‘system’ or ‘the theory’ we are testing? The short answer is that it’s the system. But then why did Popper, all in one paragraph suddenly equate that with ‘the theory’?

Leave a Reply

Your email address will not be published. Required fields are marked *