[Previous message][Next message][Previous in topic][Next in topic] [Previous by same author][Next by same author][Previous page (1999)] [Back to main SEPTEMBER98-FORUM page] [Join or leave SEPTEMBER98-FORUM (or change settings)][Reply] [Post a new message][Proportional font][Non-proportional font] Date: Tue, 13 Jul 1999 13:13:39 +0100 Reply-To: September American Scientist Forum Sender: September American Scientist Forum From: Stevan Harnad Subject: Re: Journal Papers vs. Books: The Direct/Indirect Income Trade-off Comments: cc: Lib Serials list , VPIEJ-L@LISTSERV.VT.EDU, Elib List EJ , Bob Parks In-Reply-To: Content-Type: TEXT/PLAIN; charset=US-ASCII On Tue, 13 Jul 1999, Bob Parks wrote: > 1. xxx.lanl.gov has about 100,000 papers and that archive does not seem > to have reduced the number of journals in physics, nor the quality of > the scientific literature. Hence we have at least one strong piece of > evidence that 'free access archiving' will not lower the quality. I > don't know of any evidence showing that quality has been lowered in > physics or elsewhere. > > 3. xxx.lanl.gov seems to have conditioned its audience to 'filter' > relevent articles from the large number of submissions. I would guess > that works much like the usual filtering process that any academics use > for 'hard copy working papers'. The filter is even simpler than that: "R" (refereed journal, and perhaps journal name "JX") and "Author Name." The peer review proficiency of the journals in question takes care of the rest. > 4. When we have citation-linking for all scientific literature > http://www.princeton.edu/~harnad/citation.html it will be natural > and easy to 'value' writing - namely by the number of citations (and > possibly the 'quality' of citations). Such citation criteria are > already used in promotions and salary (at least in my small biased > sample). One can argue whether quality is better determined from > citations than from knowing that two or three referees and an associate > editor have passed judgement. Fallacy: Apart from the limitations of citation metrics, the validity they do have is COMPLETELY parasitic on the fact that the papers in question are published in peer reviewed journals. The invisible hand of peer review is behind them. Hence they would implode if that hand were withdrawn. Impact factors (like peer commentary) are a SUPPLEMENT TO, not a SUBSTITUTE FOR, peer review. (And peer review is not a go/no-go filter; it is an interactive, iterative, corrective process of submission, feedback, revision, resubmission, etc. between author and peer reviewers, adjudicated by a competent, answerable peer Editor: This is no passive numerical filter or box score on any blind metric.) Harnad, S. (1998) The invisible hand of peer review. Nature [online] (5 Nov. 1998) http://helix.nature.com/webmatters/invisible/invisible.html http://www.princeton.edu/~harnad/nature2.html > 5. As an economist, I would have to argue that the resources devoted > to refereeing are misallocated because they are not compensated > directly. In the current journal model, there may be too much > refereeing (or there may be too little). Peer review, like democracy, is not without its imperfections. And no doubt there exist ways to improve it. But those ways must first be found and tested and demonstrated to improve it -- especially the most radical one, of abandoning it entirely, in favour of self-archiving of unrefereed preprints alone. Note that I do NOT advocate the latter: I have always aimed (subversively) at the self-archiving of the REFEREED paper, not just the unrefereed one. That will be tantamount to freeing the current peer-reviewed literature, SUCH AS IT IS (warts and all). http://www.arl.org/scomm/subversive/toc.html We can worry about ways of fixing the warts independently; but the powerful and proven benefits of self-archiving should NOT be linked in any way to speculative and untested notions for improving peer review -- least of all abandoning it altogether. > If 'free access archiving' > means the end of journal refereeing as we know it, I am not sure > whether I (at least) could argue that there is a social gain or loss. There is every reason to believe it would be an enormous loss, throwing the baby (a reliable research literature) out with the bathwater (access barriers to that very literature). To put it another way, it's READER-ACCESS barriers that have to go; AUTHOR-ACCESS barriers (into the certified refereed corpus) must stay. > Referees might spend their time writing/reading rather than refereeing, > which could result in better scientific literature than what exists > with their time spent refereeing. I am not arguing that refereeing has > no value, only that we do not know what that value is, and that > whatever that value is, it is not compensated (directly at least). We know the value of the refereed literature, such as it is; every editor knows what raw submitted manuscripts look like (90% of which will be rejected, if it is a high quality journal; and most of the 10% that are finally accepted will look nothing like their initial drafts, for they will have gone through the iterative corrective feedback cycle of peer review mentioned above). It is the difference between these two literatures that is at issue (and the difference is even greater than that, because even those raw submissions are prepared with the PRESUMPTION of answerability to peer review -- yet another manifestation of its "invisible hand"). No, human nature being what it is, without answerability it quickly regresses toward the anarchic levels of the chat-groups NetNews, that Global Graffiti Board for Trivial Pursuit. (And neither the Pandemonium of post hoc "peer" commentary nor the still poster-hoc feedback from "citations" can provide that answerability. Pity the reader who has to navigate a chaotic corpus like that.) > IMHO, the only reason to sort it out is to determine, given the goals > of the esoteric author (a term I like), whether 'free access archiving' > will lower or raise the quality of scientific literature. IMHO, it would not be good empirical practise to test whether the FDA is really protecting our healths by scrapping it and seeing what happens! The goal of self-archiving is to free the refereed literature from access barriers, not to free it from refereeing! The way to test variants on or alternatives to peer review is locally, not globally: Do you know of any local experiments? I do, and as far as I know, it is not yet faring too well! http://www.bmj.com/cgi/shtml/misc/peer/index.shtml > Again, the point should be whether the quality of the scientific > literature is harmed by 'free access archiving'.... > In the NO-JOURNALS world of 'free access archiving' > we write to attract others attention, and citation. Rather than > writing for three people (two referees and an editor) we now have to > write for a larger audience and have to write to attract a readership > (rather than attract an editor/referee). I don't see that deters us > writing. The goals of fortune and fame remain, its just the journals > no longer have a Faustian GRIP on us. And it all becomes a vanity press, with no sign-posts for the poor reader and user as to what, in all this unregulated soup, is fit for consumption! > The current business 'model' for scientific literature is, well, > absurd. Editors are mostly not directly compensated, and those who are > are not compensated at the market value of their time. Referees are > not compensated ($35 or $50 is not compensation). Authors are not > compensated at all directly. Correct. But it is also what vouchsafes us our current refereed literature, such as it is. Let us free THAT before toying with any notional improvements -- INCLUDING referee payment, which is potentially corruptive: They referee for free now, and that's just part of the system, such as it is. The system, with its authors giving their papers away free, and its referees giving away their services for free, is best described as ANOMALOUS, not ABSURD. What is absurd is to continue treating it according to the access-blocking trade-model, instead of freeing it (from S/L/P) for the reader as well. Here is an argument that could be invoked against me, but I don't think it's valid: "Fine, I take you at your word. 'Don't tamper with the system, just free it.' Now I will show that that admonition is self-contradictory: I agree that altering or abandoning refereeing would be tampering, and would put quality at risk. I agree that paying referees would be tampering, and would put quality at risk. But then, isn't author self-archiving tampering too, and putting quality at risk? Might it not bring down the entire system, destroying the revenue base on which the quality control is built? Is it, therefore, not one of those untested "reforms" of peer review against which you always inveigh? Q.E.D." My reply is simple: Authors have always given away reprints of their papers for free. Self-archiving simply increases the scale of this. And the waters HAVE been tested, for close to a decade now, by Los Alamos, and no sign of diminished quality has emerged (as Bob notes above). http://xxx.lanl.gov/cgi-bin/show_monthly_submissions So the free waters are safe for peer review. (And if and when another revenue source must be found for continuing to fund it, the up-front redirection of 1/3 of S/L/P savings to institutional publication costs is ready as a natural source for it.) Hal Varian's worries about possible knock-on effects elsewhere in the quality control system are not entirely without basis, but there doesn't seem any compelling reason for alarm either. Let self-archiving proceed apace. > So the university pays us to > author/edit/referee and then buys our product back from a 'publisher'. > Resources must be misallocated in that model. If our current world was > a 'free access archiving' with citation valuations (rather than journal > valuations), proposing such a business model would, well, be absurd. > We need to unshackle ourselves from the current journal Faustian Grip, > from that mental model of the world, and proceed ahead. Nor should we > consider that scientific literature fits into other 'information' > products. This seems a bit garbled to me, because it conflates freeing the refereed literature form S/L/P with freeing the literature from refereeing. > Much of the discussion between Hal and Stevan side steps into business > models (ignoring any further words on motivations of authors). So what > is the business that requires a model? Production of (quality) > scientific literature. Actually, the "business" is making an impact on research with one's research. The literature is just a means, not an end. Hence my analogy with ads. > Must that be tied to the elsevier et al (I use > elsevier in lower case as a generic for profit and non-profit > presses)? elsevier does not pay the authors, nor the referees nor the > editors which is 95% to 99.9% of the real cost of producing the > literature. In the 'free access archiving' world, we do not need to > wory about whether elsevier survives. We do need to worry about the > quality of the scientific literature, and elsevier itself does not > provide that quality control. Editors and referees do. Citations do. Peer review does; citations do not! Elsevier (and others) implement peer review. That will always cost something -- but nothing like what the whole papyrocentric, S/L/P-based system costs now. > Whether universities are willing to compensate us for editing and > refereeing without the elsevier label is an open question (especially > if the citation linking proposal becomes fact). In fact it is a > question which should be asked - how much refereeing should be done? > If we have citations, do we need refereeing and editing? It is not > that refereeing and editing do not increase the value of an article, it > is whether the correct amount of resources are devoted to that > activity, and whether citations (or similar) would be a more cost > effective way to discern the quality (for promotion, tenure, etc.). Vide supra. It seems to me naive in the extreme to imagine that delayed post-hoc citations can substitute for the substantive, interactive, quality-control process of peer review. > Imagine a world with 'free access archiving' without journals. How > does one get promoted? Citations and review letters. Citation > analysis would be free, and universities would have to compensate for > outside review letters. Would that really change the quality of > scientific literature - for the worse? Not in my mind. Have a peek at Usenet/Netnews for a glimpse of where things would head, human nature being what it is... -------------------------------------------------------------------- Stevan Harnad harnad@cogsci.soton.ac.uk Professor of Cognitive Science harnad@princeton.edu Department of Electronics and phone: +44 2380 592-582 Computer Science fax: +44 2380 592-865 University of Southampton http://www.cogsci.soton.ac.uk/~harnad/ Highfield, Southampton http://www.princeton.edu/~harnad/ SO17 1BJ UNITED KINGDOM ftp://ftp.princeton.edu/pub/harnad/ ------------------------------------------------------------------------ Back to: Top of message | Previous page | Main SEPTEMBER98-FORUM page ------------------------------------------------------------------------ Back to the LISTSERV home page at LISTSERVER.SIGMAXI.ORG.