Publication bias

Much of the culture and methodology of science has been specifically designed to help combat bias. The human brain takes sensory inputs, filters them through our biases before we even perceive them, assigns them whatever interpretation seems most reasonable given what we already think, stores them (or not) depending on how they fit into our preconceived notions, and then retrieves them when they seem to support the view we already have on the topic at hand. We are like perfect bias machines, and we have to bend ourselves into all sorts of intellectual contortions to achieve any measure of impartiality on almost anything. Most of the statistics scientists employ, rather than testing for support for our preferred hypotheses, are designed to test for support of the opposite of our hypothesis. Only if someone tries really hard to prove the opposite of their point and utterly fails to do so are we convinced that their data significantly support their point. The scientific review process, the sometimes painfully stilted manner in which science is written and even the reluctance of many scientists to have any polling based system of evaluating scientific consensus are all designed to help us keep our biases in check. The Spockian view that one should act as though emotions and unreason don't exist is clearly illogical.
Despite all this anti-bias fervor, scientists are still human, and still have biases. One such bias is that we want, as individuals, to be successful. We would rather study a topic that is going to win us praise, jobs and funding than something else that will take a long time to reach any conclusion, even if the payoff for society is much greater for the long term project. We are also far more likely to publish results that are going to advance our careers than those that won't. This leads to the title of a 2006 paper called, "Publication Bias: The Problem That Won't Go Away." To be clear, I am not referring to a reluctance to publish papers that challenge the dominant viewpoint. Name any truly successful scientist, and I will bet you that his or her career was built on a paper that challenged the dominant viewpoint. Scientists know this, and we are pretty much obsessed with finding holes in the dominant view. Publication bias rather is most commonly a tendency to publish clear positive results ("our data strongly support hypothesis X over hypothesis Y") over the less clear cut cases ("our data do not allow us to confidently support or refute the hypothesis we set out to test"). This kind of thing happens a lot.

Publication bias on the part of others, and by me, has been on my mind recently. In writing a paper on the evolution of paternal care in primates, I ran into the problem that almost nobody makes the statement, "in 72,000 hours of behavioral observation we did not observe males caring for their young." But if after 120,000 hours of observation they see a male pick a couple of burrs out of a juvenile's fur, they might well write a paper titled, "first observation of paternal care in species X." This results not only from the logical impossibility of proving the complete absence of a behavior, but because in many species where males don't care the assumption is that males don't care, and a failure to see counterexamples isn't that interesting to the people studying the species or the editors reviewing their papers. The result of this is that we have lots of publications stating the presence of care, even when that result is rare, but almost nobody stating the likelihood of its absence. I ended up having to make the rule that if multiple papers describe patterns of care in a species, and none of them say anything about male care, that counted as too little care to qualify. In my own studies of rotifers, I absolutely would have written a paper on parental care if I had observed any, but absolutely could not get a paper published in which I state that I didn't see any.
But my publication bias when it comes to the rotifers is much more profound than that. The results of one of my major experiments were negative, and as such not particularly interesting. I am writing it up anyway, because my dissertation committee wants me to, but I and they think it unlikely any journal will publish it. Of course if 20 different labs did similar experiments, and only one got a positive result, only the positive result would be published, creating a very false impression. I recognize this as a source of bias, I don't like it, and I don't see any way around it. The best I might be able to do is post the whole damn paper on my blog and hope that anyone interested in the topic stumbles upon it.

The other option, which I won't take but is far more common than you might think, is to simply analyze my data until I do find something interesting in there, then write up the paper as though that was my main question all along. This is something that advisers have specifically told me to do in the past, although not on this project. It is grudgingly accepted that this happens, and like publication bias, isn't going away.