Choosing peer reviewers

Several years back, I was unbearably excited to be, for the first time, submitting my own manuscript to a real scientific journal. I'd spent months polishing it, and was totally confident it should be published. Working through the various steps of submission (input author names and contact info, input title, input abstract, keywords, ect.) I was surprised to be asked to input names and contact info for three recommended reviewers. Defendants in courts don't get to recommend specific peers to serve on their juries, how could science be served by asking me to recommend peers to evaluate my work? Confused, I emailed one of my advisers, who happens to be an outspoken crank when it comes to all of officialdom, and came away with the impression that this was not to be taken too seriously. I hastily plugged my keywords into Google and chose three prominent names I had never heard of before, who went on to tell the journal my paper was technically sound but not of the greatest interest. I was extremely lucky in that the editors of the journal found it very interesting, and published it anyway. Failing to have learned my lesson, a few papers I submitted since then were sent to unfavorable reviewers that I had recommended.

Now, journal editors can decide to ignore these recommendations, and those invited to review can say no, but in a large portion of cases (I have no data, this is simply a strong impression), some or all of those recommended end up writing reviews. The reason journals ask for such recommendations is to help the journal editors quickly find people who are highly qualified to review. I already know who is very knowledgeable about my specific topic, while the journal editors may not. Many journals have editors who are not paid anything for their service, and all have editors whose time is limited. Asking for and using these recommendations saves time, and probably helps avoid unqualified reviewers.

Doing so, however, has some pretty clear corrupting influence. Those who are good at this game pay great attention to whom they recommend, not only carefully considering the knowledge, viewpoints and interests of those they will recommend, but crafting the paper to raise as few objections as possible from these individuals. Papers that defy much of what is well established in one field regularly are published based on the recommendations of reviewers whose knowledge comes from another branch, and this fact is not ignored in making these recommendations. If the paper is very likely to be sent to a particular reviewer, that person's terminology and definitions will be used and his or her papers referenced. At scientific meetings, people will say, "I like this idea, list me as a reviewer." At its worst, peer review is reduced to a popularity contest, with well established authors having their work (not only journal articles, but also grant applications) evaluated mostly by their friends and allies.

Mixing moral distaste, political naivity, and hurry, I have generally spent no more than a few minutes on the question of who my recommended reviewers will be. The latest iteration of this came with a paper on which two of my undergraduate students are lead authors. We submitted it to one journal with a recommended list of reviewers that was hastily thrown together, experts on the organism we were working on but not necessarily interested in our particular topic. They trashed it solely on the basis that no one cared. We submitted the same manuscript, with very few edits, to another journal, listing reviewers with some knowledge of the organism, but a strong interest in questions related to our own. We got back three extremely positive reviews, praising our highly original and relevant work, recommending several very minor changes and urging the editors to publish this paper. Yesterday morning we submitted these revisions, and yesterday evening this journal officially accepted our paper.

So I've not only learned my lesson, but decided to heed it. Whom I ask to review a paper is (depressingly) almost as important as the quality of the work. I see truly terrible papers come out in excellent journals, presumably approved by carefully chosen reviewers, and some very good papers get rejected by less selective journals, in part because of poorly considered recommendations. From now on, I will put my qualms aside and think carefully, early in the process, about whom I will recommend as reviewers. After all, everybody else is doing it.

Traditionally successful

I have had a hand in advising several doctoral students, but none of them have been my student in the sense that I am their major professor (what the Germans would call Doctorvater*) because I have not been a professor. Starting in January I will be an assistant professor, and starting in February I will have my first for-realsies doctoral student. She and I have already worked together for some years, and written papers together, and I am confident that she will be wonderfully successful.

That said, it would be irresponsible of me not to think through what barriers could arise that would potentially threaten the successful completion of her doctoral work. Of course there are all sorts of logistical and scientific questions to consider in designing her research, to make sure it will produce something that will be publishable and launch her on a successful career. But I have known enough doctoral students who stopped before getting a doctorate that I feel like I have a fair sense of what goes wrong, and the scientific difficulties (e.g., almost getting killed in Papua New Guinea) tend to cause delays, rather than make people stop altogether. I have not known any doctoral students who failed, in the sense that they produced dissertations that were simply indefensible. I have known many who didn't produce a dissertation or receive a PhD.

These nondissertators were in my experience no less bright, driven or well resourced than other doctoral students. Some of them suffered from health problems, scientific set-backs or family emergencies that exacerbated their situations. But the characteristic that unites them is that they were poorly matched with their Doctorvaters or Doctormutters. Sometimes it was a simple personality conflict, but more often the mismatch was in terms of the advising style and the need for advice. Students who need guidance and aid often end up with ultra-busy hands-off advisers. Students who need to be left alone to do what they already know how to do sometimes end up with micromanagers. I spent a fair bit of time as a doctoral student thrashing around, not getting as much advise as I needed, not even knowing what the rules of the university were. A good friend in another lab wasted a lot of time fighting with her adviser whose overzealous attempts to advise were often an impediment. Both of us at times considered dropping out, transferring to other labs, or committing felonies, but a combination of luck, pigheadedness and good burritos got us through. We had several peers who took other paths in response to adviser mismatch. It is a very rare student who will go to her adviser and say, "I want you to alter your approach to advising," and not all advisers would respond positively. I hope that my student will say this to me if necessary, and knowing her, I think she will. If she doesn't, I'll ask.

In the academic world, leaving the academic world is generally thought of, and often spoken of, as failing. If you are in a doctoral program and leave without your doctoral degree you fail. If you get your doctorate and then make a career in academia, you succeed. What I have done is clearly success, so to do otherwise is clearly failure. This is of course a definition born of bias and self-congratulation. My job as a Doctorfater is to simultaneously help my student to succeed in the traditional sense and to make it clear that other paths are not necessarily failures. Producing a garbage dissertation is failure. Finding an alternate route to a happy productive life is not. I hope that making this clear will help my student succeed, in the traditional sense.

 

*Vater in German is pronounced like 'Fah-tar' in English and means father. It has only just now occurred to me that Darth Vader's name was rather a large hint.