The Scientific Method Explained

  In questions of science use reason

“If everyone's trying to trick everyone all the time, it's a lot of noise and confusion. It's better just to be straightforward and try to do useful things."

[Elon Musk]

In an interview for Rolling Stone by Neil Strauss#, Elon Musk explains the scientific method, a phrase he uses often when people ask him how he came up with an idea, solved a problem, or chose to start a business.

    For his purposes, in his words (mostly), he defines is as:

  1. Ask a question.
  2. Gather as much evidence as possible about it.
  3. Develop axioms based on the evidence, and try to assign a probability of truth to each one.
  4. Draw a conclusion based on cogency in order to determine: Are these axioms correct, are they relevant, do they necessarily lead to this conclusion, and with what probability?
  5. Attempt to disprove the conclusion.
  6. Seek refutation from others to further help break your conclusion. If nobody can invalidate your conclusion, then you're probably right, but you're not certainly right.

    “That's the scientific method,” says Musk. “It's really helpful for figuring out the tricky things.” The whole interview is worth a reading. It may shed some light on how Musk thinks and works and provide context to the media sound bites.

    Nobel Prize winner theoretical physicist Richard Feynman said something very similar. In a 1964 lecture#, he provided a one-minute explanation of the scientific method:

“Now I’m going to discuss how we would look for a new law. In general, we look for a new law by the following process.

First, we guess it (audience laughter), no, don’t laugh, that’s the truth.

Then we compute the consequences of the guess, to see what, if this is right, if this law we guess is right, to see what it would imply and then we compare the computation results to nature or we say compare to experiment or experience, compare it directly with observations to see if it works.

If it disagrees with experiment, it’s WRONG.

In that simple statement is the key to science. It doesn’t make any difference how beautiful your guess is, it doesn’t matter how smart you are who made the guess, or what his name is… If it disagrees with experiment, it’s wrong. That’s all there is to it.”

    “It is not unscientific to take a guess,” says Feynman, “although many people who are not in science believe that it is.” Because the scientific process starts with something that is likely vs. something is that unlikely. To explain the difference, he tells a personal story.

    In a conversation with someone who asks him is he believed in the sightings of UFOs, Feynman says:

“It is much more likely that the reports on flying saucers are the result of the known irrational characteristics of terrestrial intelligence, rather than the unknown rational efforts of extraterrestrial intelligence.

It's just more likely, that's all. And it's a good guess. We always try to guess the most likely explanation, keeping in the back of our mind that if it doesn't work, then we must discuss the other possibilities.”

    Our data changes over time and as we develop new experimental methods and techniques, so we must update what is right. Which means we should hold scientific concepts (or parts of concepts) to be right until the data no longer supports the hypothesis.

    To illustrate how right and wrong are not absolutes, but on a scale, Isaac Asimov uses a comparison. Asimov was a very prolific American author and professor of biochemistry at Boston University, best known for his works of science fiction and for his popular science books.

    He says:

“When people thought the earth was flat, they were wrong. When people thought the earth was spherical, they were wrong. But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat , then your view is wronger than both of them put together.”

    “It seems to me that right and wrong are fuzzy concepts,” he says. Certain ideas can be true in a sense, but still in need of further correction with new information. The difference between opinion and science is that science bases its correction on proof.

    How we go about collecting proof matters. The scientific method is a simple way to keep ourselves honest.

UPDATE: Michael Crichton helps us understand the scientific method by comparing it to consensus, which is opinion-based:

“I want to pause here and talk about this notion of consensus, and the rise of what has been called consensus science. I regard consensus science as an extremely pernicious development that ought to be stopped cold in its tracks. Historically, the claim of consensus has been the first refuge of scoundrels; it is a way to avoid debate by claiming that the matter is already settled. Whenever you hear the consensus of scientists agrees on something or other, reach for your wallet, because you're being had.

Let's be clear: the work of science has nothing whatever to do with consensus. Consensus is the business of politics. Science, on the contrary, requires only one investigator who happens to be right, which means that he or she has results that are verifiable by reference to the real world. In science consensus is irrelevant. What is relevant is reproducible results. The greatest scientists in history are great precisely because they broke with the consensus.

There is no such thing as consensus science. If it's consensus, it isn't science. If it's science, it isn't consensus. Period.”

    When in doubt, verify.