By J. Edward Wright, Ph.D.
Professor and Director
Arizona Center for Judaic Studies
University of Arizona
Associate Editor, Bible and Interpretation
I thank my colleague Bob Cargill for his usual thoughtful analysis of important issues. In addition to his scholarly work, I value and admire his expertise in IT and media. However, in this case I find I disagree with his basic thesis: the peer-review process is not beset by the problems he identifies.
Cargill states his concern concisely: “Traditional academic peer-review is an antiquated, inefficient, and oft-abused process that will soon be replaced by new technologies and the more transparent processes of academic discussion they empower.” There are two issues here: the nature of review process itself and the technologies used in that process.
Regarding the technologies used in the review process, I submit that today most publishers are already moving away from the “antiquated, inefficient” processes that Cargill identifies. Publishing is a business, and like any business it survives only if it produces revenue well beyond its expenses. Publishers are keen to utilize skills and technologies that expedite all phases of their work (online or print), thereby lowering their costs. True, not all of their authors or reviewers are able to exploit these technologies to their fullest, but that is merely an evolutionary hurdle that will be surmounted in due time.
The online and dynamic reviews described by Cargill do have an important role to play as scholars seek input on their work. It is in this field that Cargill is a recognized leader. These interactions with colleagues can contribute to the development of scholarly knowledge, and as we all know, feedback from peers in the course of developing one’s ideas is essential. Technology provides us with ever-newer mechanisms that can generate feedback that is detailed, international, and virtually immediate. Technology has enabled us to communicate with a wide and diverse audience, and it has especially served the needs of scholars who live in places at some distance from trusted colleagues. There is a need for caution, however, since such a wide-open access will create problems for scholars, not the least of which could involve compromising intellectual property rights, as Bob notes. Moreover, this could devolve true peer-review into mere wiki-review. Such a democratized version of peer-review is not the kind of rigorous peer-review that leading publishers and journals demand.
Cargill’s designation “oft-abused” seems to refer primarily to the nature of the peer-review process itself. True, publishers regularly enlist the assistance of at least two and regularly three outside, anonymous reviewers who are recognized as trustworthy authorities in their fields. It is natural to suspect that one negative review could determine the outcome. But, a single negative review does not by itself normally sink a proposal. In fact, that is why more than one person is asked to provide a review. Most publishers, cognizant of the danger of conflicting viewpoints among scholars, also allow an author to respond to negative criticisms prior to submitting the piece to the editorial board for the final decision. That board’s decision may even suggest or require some changes in the final draft. The process is typically dynamic and is not as strictly up-or-down as Cargill seems to suggest. Moreover, should an author’s work be declined by one publisher, he or she is free to find another venue for their work. The danger that the traditional peer-review process is undermining advances in the Academy is negligible.
Cargill is also concerned with the potential for abuse due to the anonymity used in the peer-review process. For my part, during the course of twenty-one years in the professorate I have participated in a wide variety of peer-reviews in a multitude of venues. In all of those cases I can honestly say that I know of only one where a single anonymous reviewer’s negative assessment largely determined the outcome, and that, interestingly, was a case in which I thought too many outside reviewers were included in the process in the first place. The anonymity demanded in the peer-review process serves the best interests of the institution or business, the applicant and the reviewers: it allows reviewers to provide an honest critique of the work. I think that the “transparency” Bob seeks would actually militate against reliable reviews. In any event, editorial committees carefully weigh all praise and criticism in an effort to detect if any personal, methodological or theoretical bias may lurk behind a particular review. Checks and balances are in place to ensure the publisher’s interest in quality is maintained.
I do not share Cargill’s enthusiasm for the review of scholarly products (or scholars themselves for that matter) by wider audiences, much like political candidates. The Academy is not concerned with popular support but with quality content as judged by nationally and internationally recognized leaders in the field who are free to express their honest assessments under the protection provided by anonymity. To be honest, Bob does not advocate a truly populist review process. But knowing the nature of academic interchange, I should think that engaging numerous scholars in a transparent review would only create more chaos, with a cacophony of voices slowing the process and introducing a host of potentially compromising variables into the review. Some journals and publishers will adopt procedures similar to those my friend and colleague Bob Cargill proposes. I am certain, however, that the names of premier journals and publishers will not soon be found in that cohort.