Your biggest misconception about science

Among the frantic madness of trying to finish a thesis, I've been thinking a lot about how we evaluate scientific evidence. This has brought me to realize what I see as the single greatest, most damaging, and potentially most widespread misconception held about science. I'm not talking about disbelief in evolution, or climate change, or the moon landing, or any of the other usual loony bin stories. The misconception I have in mind is at the root of, or at least facilitates, many of these more discussed fallacies.

Imagine if you will that you are an electrician. I ask you to install a solar power system, in which you have specific training, and then to run the power to batteries, outlets and appliances in various parts of a house I am building for my grandmother. You carefully chose the type, size and placement of the photovoltaic panels, figure out how to attach them securely to my roof, how to run the wires, how to ensure safety and consistency of power. You comply with various and sundry codes I've never heard of, use materials whose properties I'm not knowledgeable enough to appreciate, and otherwise exercise your expertise and skill.

I, as an intelligent well-educated person with some layman's knowledge of wiring should be able to ask you questions about your work, and if you know what you are doing, you should be able to explain your choices in ways that make sense to me. Why are you using the more expensive panels instead of these cheap ones my cousin bought? Why does the wire go through there instead of through here? But you would rightly think I was an idiot if I thought I could evaluate the soundness of the plan in all its intricacies. Other than getting independent opinions from other experts, assessing your reputation, or spotting huge obvious flaws like a lack of power to the kitchen, I would have no way of telling if you were really planning an ideal system, or screwing me because you secretly own stock in the company that sells that brand of battery. And if three electricians gave me three divergent opinions on the same design, I would not presume to know which was right without asking a fourth. The same would be true of work by almost any expert in almost any technical field. Non-experts should be able to understand, should be able to spot obvious flaws, should be able to ask multiple opinions, but should not assume they are qualified to assess the finest points. No one without training as an electrician can say if a complex wiring diagram is perfect or adjudicate a disagreement between master electricians.

You've probably spotted where the long analogy is headed. It is often said, quite rightly that any scientist who can't explain his work to an intelligent layman is a fraud (or at least very bad at explaining). I forget the original quote. This, and the general (and correct) view that scientists should feel obligated to explain our work to the public reinforce the view that any thinking person should be able to evaluate the correctness of a scientific conclusion. But if I, having worked as a carpenter and done some wiring, can't evaluate the optimality of a complex circuit diagram, why should I, as a biologist, expect to be able to assess whether the uncertainty is understated in a climate prediction model which took hundreds of person years to design and build, went through multiple iterations, been evaluated by independent experts and are so complex it would take me months of reading just to understand what all of the variables are? I shouldn't and neither should you, unless you have years of intense training in that sub-field. I can read the papers, get a sense of what the question and conclusion is, understand an outline of the methodology. But I have no hope of just stumbling upon a conceptual or methodological error. I have no hope of finding the climatologist's errors if what I read is not the primary paper, but an article written by a journalist who also has no training in climatology. This is the blind leading the blind in trying to find flaws in the color scheme of a digital drawing. If two climatologists with opposing viewpoints were to write a trade book in which they lay out their disagreements, present evidence for their views and question each other's evidence, I would understand the science much better than I do, but I would still not be qualified to say which one of them was right and why.

It is unfortunate that non-experts generally can't evaluate science. It leads to the view that science is esoteric, made up, snobbish, arbitrary, undemocratic, religious and most of the other negative things people believe about science. It is also unfortunate that people don't know, or won't admit, that they can't just sit down and by thinking hard decide whether a scientist's work is useful, novel and correct. But they can't. You can't. I can't either, except in fields where I have reviewed the literature, thought deeply about the issues for weeks or months, read more literature in tangentially related fields, discussed the issues with other scientists and then sat down and written out my objections and concerns. This is why PhDs take more than five years on average. Science is hard and requires masochistic attention to detail, not only for those making statements, but also for those evaluating it. If you could sit down read a few papers on a subject and make a novel, well reasoned and convincing argument showing that previous papers are substantially wrong, you could have a PhD in a few weeks. No one has ever done it (or if they did their was fraud involved.)

This misperception, that anyone can evaluate science is widespread. I know scientists who openly contradict conclusions which are long since the consensus in fields in which they have no particular expertise. I've talked to several extremely well educated non-scientists who claim to have evaluated the evidence for human-caused global warming and come to firm conclusions. My conclusion that humans are causing climate change is based not on my personal review of the arguments pro and con, but on the fact that the vast majority of people who are qualified to judge, including many who were originally skeptics, say that it is no longer reasonable to doubt. But many, perhaps most Americans feel qualified to personally cast judgment based on the evidence. I have seen no polling on this, but expect that a larger number of Americans would tell you they are capable of offering informed opinions of scientific controversies than on disagreements between electricians.

As an evolutionary biologist, I feel qualified to read a book by an "Intelligent Design" proponent and say exactly what is wrong with their argument. I can map the circular logic, find the flaws in their interpretation of the Second Law of Thermodynamics, and point out which references are being misquoted. I can describe exactly how their evidence fails to support their argument, and explain exactly why the argument fails to qualify as a science. A nonscientist, reading these books, and armed with the delusion that if there are flaws she will see them, is likely to believe or disbelieve based on religion, politics and predilection. The scientific soundness of the argument is tertiary. Before I was versed in evolutionary biology, I could see logical flaws in Intelligent Design. But had I been raised with as little knowledge of evolution as I have of climatology, I could have only trusted to expert opinion. Similarly, while conspiracy theories about the moon landing are grossly implausible, I am not qualified to judge the claim that the physics of the landing wouldn't work as NASA says they do. The fact that physicist say it works just as explained is good enough for me.

It is not the moral of this story that you shouldn't ask questions, be skeptical and challenge authority. It is not my intention to claim that laymen should never question the competence and motives of experts. Rather, I urge that we try to be realistic in judging our ability to evaluate complex arguments in fields we don't know much about. I am not an expert in whatever it is you studied, and I am not hubristic enough to think that I am.