A bunch of people have sent me Monday's New York Times article: "Scholars Test Web Alternative to Peer Review". This article describes a recent trial at Shakespeare Quarterly where they eschewed traditional peer review and instead opened the review process to a panel of experts and others on the web. The process garnered over 350 comments from 41 people which the editors evaluated. Ultimately they selected the four articles for publication in a special edition of the journal on Shakespeare and the new media.
Whenever a journal attempts a project like this it attracts attention and almost inevitably provoked headlines heralding the impending end of the traditional practice of academic peer review. Most articles envision traditional peer review to involve a journal circulating an article to an anonymous pair of experts who evaluate the article's suitability for publication in the particular journal and, in most cases, offer comments. This description of the peer review process is, of course, idealized. In reality, journals - particularly in Europe - have widely varying standards and practices for peer review with widely varying degrees of transparency. So the introduction of a new method of peer review which takes advantage of the increasing degree of connectivity on the web does not so much represent radical novelty amidst stodgy, ossified, practices of peer review, but another point along a continuum of practice.
Despite this reality, I know that any modifications to the traditional peer review practices are likely to create waves. I generally consider my colleagues across the disciplines to be fairly liberal minded folks, but it never ceases to amaze me how limited our perspectives become on matters like scholarly publishing. In fact, it befuddles me why academia struggles so mightily with the idea that "in the future" we could acknowledge the value of academic and intellectual work produced through a wide range of publishing paradigms ranging from the un-edited and un-reviewed blog to the highly polished peer reviewed journals. Of course, I can anticipate one response: with the explosion of new publications and formats, the "average scholar" struggles to keep abreast of developments in his or her field. Moreover, introducing a new layer of less rigorously reviewed material to the mix contributes to the massive quantity of material that scholars are expect to understand.
On the other hand, the rise of highly integrated and sophisticated social networking applications is making it easier to filter scholarship through a layer a kind of secondary review by colleagues. My friends and colleagues serve, in effect, as another layer of peer review ensuring the we as a group have access to "important" scholarly contributions even from obscure journals. While there is no guarantee that good scholarship will find its way through our social network, the economies of numerous eyes scanning the growing body of scholarly literature gives us a better chance of seeing things important to our common research interests.
The other traditional complaint against adjusting the peer review process is that it will ultimately undermine the quality of scholarship produced. It is as if the practice of circulating working papers, archaeological reports, pre-pubication drafts, informal reviews, has not existed for as long a peer-reviewed publications. For centuries scholars circulated manuscripts to colleagues and friends without the benefit of anonymous, exterior reviews. The major shift now is that we can democratize the process of circulating working papers by using the web rather than informal and private avenues of scholarly communication. In fact, the newly democratized practice of pre-publication circulation offers the potential to uphold the highest standards of peer review. The pressure will be on the peer review process to demonstrate the superiority of its product in relation to non-peer reviewed work. Such competition should make any benefit inherent to the traditional method of peer reviewed scholarship all the more visible.