Scientific Method

From Free Talk Live

The scientific method is a technique for developing theories. When performed properly, the technique is supposed to yeild theories that allow one to understand, predict, and control one's environment.

The most suprising aspect of the scientific method to many people is that the method is not used to prove claims, but rather to disprove claims. Theories developed using the scientific method are never known to be true, but rather as research is performed, the confidence one has in a theory may appoach 100%. Yet confidence that a theory is true never fully reaches 100%. This is due to the falsifiability requirement of the scientific method.

The falsifiability requirement comes from the premise of the scientific method that no theory (and a theory is merely a tested assertion) can be tested using the scientific method unless it can be disproven. This is the major distinction that practitioners denote between the scientific method, and other systems of knowing, for example religion. The assertion that "God exists" is not possible to disprove. Therefore, it is a question that the scientific method neither confirms nor denies. It is an issue the scientific method does not address, because is is not disprovable.

Using the Scientific Method

Here is a simple example. Suppose you want to set the temperature of your freezer just cold enough that your icecubes are ice and not liquid water, but you're trying to save electricity, so you don't want it any colder than necessary. Suppose further that you do not know the freezing temperature of water. Finally, suppose that we do not have a thermometer, but we can set the freezer temperature to whatever we want.

Step 1 in the scientific method is to list your possible true assertions. We call there hypotheses. Let's start with three hypotheses:

H1: Water freezes at 37 degrees.

H2: Water freezes at 27 degrees.

H3: Water freezes at 32 degrees.

We begin with three hypotheses, but only one can be true. Using the scientific method we will disprove as many hypotheses as we can, and any hypotheses left standing after the research will be entitled to be called theories.

Step 2 is for each hypothesis, to come up with a logically accurate statement of an observable fact that will be true if the hypothesis is true. (In this example we're going to do two sets of experiments, so we'll call the first set A, and these will be the A statements.

S1A: If water freezes at 37 degrees, then the H2O in the icecube tray will be solid when the freezer is set to 33 degrees.

S2A: If water freezes at 27 degrees, then the H2O in the icecube tray will be solid when the freezer is set to 23 degrees.

S3A: If water freezes at 32 degrees, then the H2O in the icecube tray will be solid when the freezer is set to 29 degrees.

Pause a moment and examine the nature of these testing statements. Each is a conditional sentence. The apodosis or main clause is the condition that the hypothesis to be tested is true. The protasis or subordinate clause is the state that must be observable if the hypothesis to be tested is true.

Step 3 is to perform the research, either an experiment or a field study. In this example we are performing an experiment. (More on this distinction below.)

Considering statement 1A: S1A regards the result if the freezer is set to 33 degrees. So we set the freezer to 33 degrees, and wait long enough for the temperature to stabilize. Then we observe.

O1A: with the temperature set to 33 degrees, we see in the ice-cube tray liquid, not ice.

Now this is news we can use. Since from S1A we know that if H1 were true, we would be seeing ice not liquid with the temperature set to 33 degrees, we know for sure that H1 is not a true statement. Thirty-seven degrees is not the freezing temperature of water. We have disproven hypothesis 1. We discard it and are no longer at risk of mistaking it for a true statement.

But we still have two hypotheses to consider, and either or both may be false. Considering statement 2A: S2A regards the results if the freezer is set to 23 degrees. So we set the freezer to 23 degrees, wait, and observe.

O2A: with the temperature set to 23 degrees, we see in the ice-cube tray ice, not liquid.

At first blush, this seems like a more useful observation than the first, because we have ice, but as you will see it is not so useful, because it does not disprove the hypothesis we're testing.

This is the important point: we still do not know whether H2 is true. All we can tell from this experimental trial is that H2 might be true. We have failed to disprove it, but it still might be false. However, it's still in the running, and we will have to continue our research. Without discarding H2, let's test our third hypothesis, using statement 3A.

Statement 3A considers the observation we will make if the temperature is set to 29 degrees. We set the temperature, wait, and observe.

O3A: with the temperature set to 29 degrees, we see in the ice-cube tray ice, not liquid.

Now we have two hypotheses that might be true. According to our second observation, the freezing point of water might be 27 degrees. According to our last observation, the freezing point might be 32 degrees.

Logically, we might stop here, since we can assume that if water is frozen at 29 degrees, the freezing point will not be below that. But for the sake of explaining the scientific method, let's continue until we have disproven all but one hypotheses.

We have used all our testing statements, and although we disproved one hypothesis, we still have two left, we must come up with another round of experimental trials. For these we will use the following two testing statements (numbered according to the hypothesis each tests):

S2B: If water freezes at 27 degrees, then the H2O in the icecube tray will be liquid when the freezer is set to 28 degrees.

S3B: If water freezes at 32 degrees, then the H2O in the icecube tray will be liquid when the freezer is set to 33 degrees.

Again we do two trials, and make two observations as follows:

O2B: With the temperature set to 28 degrees, we observe solid ice.

O3B: With the temperature set to 33 degrees, we observe liquid water.

Refering back to our testing statements, it is clear that the prediction of statement 2B is not observed. Therefore hypotheses 2 is disproven.

Meanwhile, the prediction of statement 3B is observed. It is not disproven.

In our second round of trials we have eliminated one of the hypotheses, and now we are left with only one, H3, water freezes at 32 degrees.

But before you go away thinking we have proved that water freezes at 32 degrees, think about what would happen if H3 stated that water freezes at 31 degrees, rather than 32. Review all our research, and you will see that H3 still would not have been disproven. If the freezing point of water actually were 31 degrees, our results would be the same, which just goes to show that we have not proven that water freezes at 32 degrees, rather we have proven that the freezing point is neither 27 nor 37 degrees. With those alternate hypotheses out of the running, our confidence in our remaining hypotheses is increased. In fact, we may now justifiably call it a theory rather than a mere hypothesis. The important point is that we are still not 100% certain of the truth of H3, we're just more certain than we were before we begain our use of the scientific method.

And that's the point: using the scientific method, we can never be 100% certain what is true, but we can be certain of what is false, and by logical deduction we can approach--yet never reach--full confidence in the truth of our scientific theories.

Weaknesses of the Scientific Method

Due to the nature of the scientific method, the theories that result from it will always be vulnerable to logical attack. The specific vulnerability will depend on whether the research on which the theory is based was an experiment or a field study.

An experiment refers to hypothesis testing in a laboratory, i.e. a controlled environment. The benefit of a lab experiment is that the researcher can control whatever factors she or he wants to. The weakness is that the results may not generalize to the real world, where the environment is not controlled and significant factors may be different.

The solution to this weakness is research that takes the form of a field study. The benefit of a field study is that the hypotheses are tested in the real world, and results should be generalizable. The weakness of a field study is that the factors affecting the outcome are what they are and cannot be controlled (though they may be observed).

In our example above, we performed a laboratory experiment regarding the freezing point of water. But our results may not be generalizable. For example, our temperature control may not be calibrated the same as other freezers. Or perhaps the amount of other food items in a freezer could affect results. We came up with a theory, but our theory will only work in the real world if our results can be generalized out of the laboratory, an assumption that should always be questioned.

To address this weakness, we might have performed a field study, making observations of freezers in the real world. This would introduce other weaknesses, specifically the fact that we cannot control all factors. For example, the room temperature where each freezer we look at might be different. Unless the hypotheses we are testing takes this factor into account or else we control that factor so it doesn't change from observation to observation, the results we see may be due to this uncontrolled variable, rather than the one we are interested in, the setting of the control in the freezer. And no matter how many factors we try to control for in a field study, there may always be others we are not controlling and possibly not even aware of.

These aspects of the research used to practice the scientific method indicate how to attack any claim that is a result of the method. Any claim whatsoever. First, find out whether the claim is based on experiments or field studies. If it's the result of an experiment, identify the differences between the laboratory environment and the real world. For example, "your claim that cannabis causes brain damage is invalid because the experiment was done on rats, and we are not rats but humans."

Alternately, if the claim is the result of a field study, identify some possible factors that explain the results other than those the theory attributes the cause to. For example, "your claim that cannabis causes brain damage is invalid, because the people you observed in your study all worked at a nuclear power plant where they were exposed to dangerous levels of radiation on a daily basis. The radiation, not the cannabis, may be what caused the observed brain damage."

Knowing this, you will see that the first step in attacking a supposedly scientific claim is to find out what research it is based on. Merely asking that question will often reveal that the claim is not scientific, but rather an argument based on authority. For example, "it's true because the National Institute of Standards and Technology says so," is not a scientific claim, but an argument from authority, which is a well-known logical fallacy.

Of course, using a combination of experiments and field studies, it may be possible to test theories in a variety of ways, and to consider more and more factors until we have an overwhelming degree of confidence in the theory that has not been disproven. But the bottom line is that though we may appreach certainty, we will never reach it, and every result of scientific research has vulnerabilities that are unavoidable as long as we accept the nature of the scientific method and the requirement of disprovability.

Further Reading

To learn more about the scientific method, see The Logic of Scientific Discovery by Karl Popper.

Johnson read this worthwhile article by Terence Corcoran about "concensus science" on the July 8, 2006 show. Also here. It was covered by Stephen Carson on the LewRockwell.com blog.