The Carl Sagan type

In a statistical test, we have to balance two types of error. They are called type I error and type II, but I can never remember which is which, so I just know them as false positives and false negatives. The balance between these errors is why we use a probability value of 0.05 for a test of significance, and I think different emphasis on these errors helps to define a skeptic from a true believer.

A false positive would be accepting a false hypothesis as true. A false negative is rejecting a true hypothesis. If you try to avoid one of these, you necessarily err in producing more of the other. Lets say you don't want to be a sucker so you definitely don't want to accept any false claim as true. You demand extremely high standards of evidence and large statistical differences. You can be sure no false hypothesis will get through. However, there are likely to be true hypothesis that you also throw out. Maybe you don't want to accept any drugs that might not be effective. You demand such high statistical results that you throw out some drugs that work but either to only a limited degree or in only a portion of the population.

On the other hand, if you want to make sure you don't throw out any true theories, you could accept things with less evidence. Now the risk is that you will accept false hypothesis. If you accept any drug that shows even the weakest improvement you can be sure you won't throw out any effective drugs, but you will accept some drugs that had an effect just by chance alone.

We all must balance these contrasting tendencies. In statistics, we usually use a value of P=0.05 as a cutoff that balances the two errors. We accept the hypothesis if there is a 1/20 chance or less that it is due to chance. I think a lot of the difference between skeptics and true believers is their emphasis on these errors. A believer would hate to throw out any theory that might be true, so they demand little evidence. Sometimes just the possibility of being true is enough for them. Of course, this means they will accept many false hypothesis. The skeptic on the other hand does not want to believe a theory that is not true and demands much more evidence. They risk possibly throwing out a theory that is true.

Going too far in either direction is wrong. We need a balance. But for any phenomenon, there are many more false hypothesis than true ones. There can only be one correct explanation, while there could be hundreds of incorrect hypothesis. This suggests we should err towards rejecting false hypothesis (as a P of 0.05 does). We certainly should be aware of the kinds of errors we are likely to make, and to look out for them. We should also make sure we make an effort to determine all possible explanations for something, and then determine the explanation that is most likely. If we stop creating hypothesis after the first few that occur to us, we are much more likely to only consider false hypothesis.

The risk with skepticism is going too far in rejecting hypothesis. I think you will find a significant number of skeptics who reject global warming, regardless of the overwhelming evidence for it. Often the rejection is just a general skepticism about all new theories and familiarity with other environmental exaggerations, rather than a consideration of the specific evidence (at TAM 6, Jillette Penn expressed such a view, and admitted it was just a gut feeling).

In science, we must avoid the low standards of the true believer, but we do need some scientists who tend more towards one error or the other. Some scientists are great at coming up with ideas, sometimes outside of the box, and we need them to advance science. Others need to look at these ideas critically. I can think of several scientists who have had a few brilliant ideas, but maybe also often accept some ideas that they shouldn't. Lynn Marguilus proposed the endosymbiont hypothesis and she got it right. She has also championed several less rigorous ideas such as the Gaia hypothesis. Robert Bakker revolutionized our view of dinosaurs with The Dinosaur Heresies and because of him we now think dinosaurs may have been warm blooded and more active, for example. But many of his heresies remain as just heresies. I am grateful for people like Bakker and Marguilus. Although they sometimes accept a few too many false hypothesis, they weren't afraid to be wrong and came up with some amazing true hypothesis. And we need the other scientists who criticized them when they were wrong and separated the good from the bad.

I have always admired Carl Sagan as an educator and popularizer of science and as a great skeptic. I think he epitomizes the balance between the two errors. In many ways he was the epitome of skepticism, attacking UFOs and Velikovsky and abductions and many other pseudosciences. But he wasn't afraid to consider wild hypothesis. At one time he suggested the moons of Mars might be artificial satellites. He rejected it when the evidence was clear, but he wasn't afraid to suggest it. Carl Sagan was a great skeptic, but he wasn't closed minded and was open to the wonder of the universe. A good way to balance type I and type II error is the Carl Sagan type.