By Robert R. Cargill
http://Center for the Digital Humanities
Qumran Visualization Project
August 2010
An intriguing article by Patricia Cohen in the New York Times entitled, “Scholars Test Web Alternative to Peer Review,” looks at alternatives to the traditional academic peer-review process.1 Like it or not, current trends in online publication are all pointing toward a single reality: blogging, online forums, and wiki-technologies are the future of the academic peer-review.
Traditional academic peer-review is an antiquated, inefficient, and oft-abused process that will soon be replaced by new technologies and the more transparent processes of academic discussion they empower. Take for example the ANE-2 list.2 This moderated Yahoo group possesses a membership made up mostly of scholars working in the field of Ancient Near Eastern studies. A member or curious viewer asks a question or posts an item of interest on the message board. This action sends alerts to members’ email boxes, and the members respond with contributions gleaned from their specific areas of expertise. This process can be used to ask questions, refine an idea, or expose potential problems within a theory or article.
Comments on blogs and documents viewable by multiple viewers can similarly provide valuable feedback. An example of this is Google Docs.3 In fact, Google Docs may very well be the best solution to online peer-review available today, as it allows a select group of viewers (controlled by the document’s editor) to make corrections and suggestions to the same document. Each of the changes is logged so the author or editor can see who made what changes when, and the changes can be reverted as desired because Google Docs maintains the document’s revision history. This process is superior to the present peer-review practice of sending multiple copies of a Microsoft Word document to two peer-reviewers, which usually results in two new, highly edited copies of an article, which must then be integrated into the finished product without forgetting or overwriting changes made in the multiple versions of the original draft. Google Docs makes a single, password protected draft available to multiple designated reviewers. This process is simpler and provides feedback more quickly. The public version of this technology (a document that anyone can edit) is called a wiki,4and is the technology behind Wikipedia.5
Despite these advances, technology should not be adopted simply for technology’s sake. Rather, there should be a compelling reason to adopt these new solutions. Why should the academy make the transition to new forms of peer-review? What is so bad about the existing peer-review system? Earlier this year, I wrote an article that will soon appear in the Bulletin for the Study of Religion6 entitled “The Benefit of Blogging for Archaeology” that addresses this very question. In an excerpt from the forthcoming article, I argue:
Peer-review was developed to allow other credible scholars to review a new claim before its publication in a journal. The process is anonymous to protect against inappropriately influencing the reviewer. Likewise, the peer-review process protects the originality of the idea by not sharing it with the public, preserving the copyright for the publishing journal and the claim to a new idea for the article’s author.
Unfortunately, the peer-review process also has its drawbacks. For one, it is a slow and tedious process. Many peer reviewers are more interested in publishing their own research than reviewing another scholar’s work. Second, the peer-review process can be abused when scholars use peer-review for self-preservation. It is not unrealistic to imagine a scholar who rejects an article in peer-review not because it is poorly researched or written but because the scholar does not agree with the conclusion, which may or may not be a threat to the positive reception of the reviewer’s own work. With only two peer-reviewers needed for an article’s approval, the odds are greatly increased that one scholar, for whatever reason, might reject an article, causing the new idea to go unpublished.
Blogging should and will gradually replace the peer-review process as it now exists. By publishing an idea in a preliminary manner either publicly or in a selectively private manner via a blog, several scholars can read a new article, make comments, offer feedback, and publicly assess the article. Good articles will be received positively and poor articles will be received negatively. This process is superior to the “anonymous” process of peer-review limited to two, sometimes rival, scholars with their own agendas to push. Making this review process open to many reviewers increases the number of reviewers, therefore increasing the number of comments and providing a wider range of critique for the author and potential publisher. It is possible that with the decline of the publication of print journals and the rise of online publications, the entire process of professional print journal publication will be replaced by some combination of peer-review via blogs and online publication of original research.
Scholars…will be accountable to a wider circle of knowledgeable reviewers, and like modern political candidates making various claims, will be increasingly forced to provide evidence for the views and opinions they proclaim.
This is not to say that we should “vote” on truth, or that the most popular ideas are the best. However, if a greater number of credible, trained scholars can provide informed feedback to an article posted for review (privately or publicly), then the article can receive greater scrutiny and get to press more quickly. Granted, there are potential copyright concerns stemming from this process. For example, may an author cite another author’s preliminary draft that has been posted on a blog, message board, or wiki for public review if the final draft is not yet complete? Or, if someone copies a portion of text from another scholar's draft that has been posted for public review and publishes it elsewhere in his own article, did the scholar violate copyright? While these and other issues must be debated, they should not deter the academy from adopting online peer-review, as these same issues of copyright are already being debated in the discussion concerning online publication.
Peer-review is a valuable process and should not be abandoned. But like everything else in publishing, the migration toward online publication demands new thinking about the nature of peer-review, and part of the solution will inevitably include crowd-sourced solutions like blogs, online forums, and wikis. It is not a matter of if, but when this transition takes place. And, from the growing influence of the blogosphere and the ever-increasing number of scholars setting up blogs and publishing their findings online, it appears that answer is now. The first journals to adopt online peer-review and implement it successfully will lead the way in the new world of online publishing.
See J. Edward Wright's Response Here
Notes
1 http://www.nytimes.com/2010/08/24/arts/24peer.html
2 http://groups.yahoo.com/group/ANE-2/messages
3 http://docs.google.com/demo/?hl=en
4 http://robertcargill.com/2009/07/17/how-to-use-wikis-or-why-google-wave-will-eliminate-email-as-we-know-it/