This is a reply to Arnold Relman's much-cited NEJM critique of the
NIH/E-biomed proposal.
> The NIH "E-biomed" Proposal -- A Potential Threat to the Evaluation and
> Orderly Dissemination of New Clinical Studies
>
> The New England Journal of Medicine -- June 10, 1999 -- Vol. 340, No. 23
> [EDITORIAL by Arnold Relman]
In summary, Dr. Relman has two objections to public archiving in
E-biomed, one of them justified, but easily accommodated (as has
already been pointed out repeatedly elsewhere), and the other neither
justified nor even, on the face of it, coherent.
The first objection is that public archiving of some unrefereed
biomedical papers (chiefly clinical ones, or those with clinical
implications) might pose a public health risk.
This is true, but there is no reason whatsoever why a safe and reliable
system cannot be designed and agreed upon that would (1) vet all
unrefereed E-biomed submissions to exclude what needs to be excluded to
protect public health and to maintain whatever special standards are
agreed upon for the unrefereed clinical literature in E-biomed, (2)
clearly tag all E-biomed contents as REFEREED (with journal citation)
vs. UNREFEREED, and even (3), if need be, prominently pre-pend a "health
warning" to all the unrefereed papers in E-biomed.
These rather obvious cautionary measures for the unrefereed clinical
literature, already partly sketched in the original E-biomed draft
proposal, should also be weighed in light of the fact that public
self-archiving on the Web is an option that is ALREADY open to authors
in a variety of ways, entirely apart from E-biomed. So unrefereed
clinical preprints can and do already appear on the Web, and are
retrieved for one and all by search engines. The unrefereed sector of
the E-biomed archive will have the virtue of filtering out substandard
preprints, tagging all preprints clearly as such, in contrast to
refereed reprints, and, if necessary, even flagging them with a warning
as potentially hazardous to health.
But the most important reply to this first objection is that it
presents no reason at all to delay the immediate implementation of
E-biomed as a means of freeing the biomedical literature for one and
all. For the self-archiving of REFEREED papers could begin at once,
with no attendant health risks, even as the safeguard systems for the
unrefereed clinical papers were being designed and tested. (Moreover,
the unrefereed NON-clinical preprint sector of the E-biomed could
likewise start at once, along the lines of the unrefereed preprint
sector of the Los Alamos Physics Archive.)
Dr. Relman's second objection is that the public archiving of even
REFEREED papers is unsafe too, in much the same way; they can only
be safely published by the journals, in the traditional way (through
print publication). Premature self-archiving, without the "benefit of
simultaneous expert commentary and interpretation" would cause
"confusion and misunderstanding."
Here I am afraid Dr. Relman is himself misunderstanding the new medium,
and perhaps confusing the public archiving of refereed findings with
premature press releases of unrefereed findings. The latter are indeed
dangers to public health, for the same reasons discussed above, but
there is no corresponding risk for REFEREED findings -- otherwise even
the traditional publication of refereed papers would have to be held
back until it had had the "benefit of simultaneous expert commentary
and interpretation"!
So the second objection has no basis whatsoever and can and should be
discounted completely. Refereed findings can and do elicit peer
commentary. They will certainly do this at least as effectively in the
online medium as in paper. But peer COMMENTARY never has been a reason
for holding up the PUBLICation of refereed findings: Peer REVIEW (i.e.,
refereeing) is supposed to have seen to it that they were ready to
appear once they met the quality standards for acceptance.
So there is nothing at all in Dr. Relman's critique that should hold
back the immediate implementation of E-biomed for either the refereed
sector or the nonclinical unrefereed sector; his concerns about the
vetting of the unrefereed clinical sector are legitimate, but were
already explicit in the first draft of the E-biomed proposal, and can
and will be satisfactorily accommodated.
I now proceed to quote/comment mode:
ar> The great majority of readers and users of the clinical literature are
ar> practicing clinicians, not working scientists. They often know little
ar> or nothing about the methods of published studies, and they depend on
ar> the accompanying editorials in clinical journals to help them interpret
ar> the data and place the studies in the context of their own practices.
If clinicians do not use published studies without consulting
concurrent editorials then there is no reason they should not continue
this practise when all studies are archived in E-biomed. They can
continue to first consult editorials, whether they appear only in the
print journals or are likewise archived in E-biomed. E-biomed provides
the added option of enhanced access to some or all of the literature
when and where it was lacking: So what?
ar> New clinical findings often attract wide public attention, and patients
ar> need advice from their physicians on the relevance of such findings to
ar> their own medical problems. The release of important new clinical
ar> findings can have an immediate social and economic impact and can
ar> affect public policy.
This is a non sequitur.
ar> The best way to protect the public interest is through the existing
ar> system of carefully monitored peer review, revision
Correct, and irrelevant, as peer review will proceed apace for the
refereed sector of E-biomed.
http://www.cogsci.soton.ac.uk/~harnad/nature2.html
ar> and editorial commentary in journals
Editorials too will continue to be written, presumably, by whoever wrote
them in the past, and will continue to be available to clinicians
requiring them in order to interpret refereed clinical studies. There is
no contingency here. (Perhaps refereed clinical articles too require a
warning label: "Do not use without consulting editorial"...)
ar> and by timing public disclosure to coincide with the date of journal
ar> publication.
This last point, unfortunately, has to be called the nonsense that it
is! If it is not merely a reflexive bid to safeguard journal primacy
and revenue (which would be deplorable, and one hopes that that is not
what motivates it), then it is merely an expression of a superstitious
adherence to completely irrelevant and obsolete features of the
print-on-paper era for journal publication.
There is no need whatsoever for an author to hold back his REFEREED
report once the final draft is done and accepted. The rest is just the
unfortunate retardation of a bygone papyrocentric era.
ar> Mistakes, inaccuracies, and misinterpretations in clinical research
ar> pose a far greater risk to health and the public welfare than do errors
ar> in basic-science research. A system that allowed immediate electronic
ar> publication of new clinical studies without the usual careful process
ar> of peer review and revision would be risky at best and might well fill
ar> the clinical data bases with misleading and inadequately evaluated
ar> information.
This is a false opposition (conflating the two objections into one): We
are talking about the REFEREED literature here.
ar> Even if E-biomed were to eliminate the second pathway and accept only
ar> work that had passed full peer review and revision by journal editorial
ar> boards, immediate posting on the Web site before publication would
ar> still be problematic, because the information would be made public
ar> without the benefit of simultaneous expert commentary and
ar> interpretation.
Repetition unfortunately does not strengthen this argument. Besides, for
those papers that would benefit from simultaneous editorial commentary,
there is no reason that synchronization cannot be arranged in the
Archive too. (In other words, this entire issue is trivial, and a red
herring.)
ar> The few weeks saved between acceptance and print publication would not
ar> justify the confusion and misunderstanding that would often attend the
ar> immediate electronic posting and subsequent publicizing of clinical
ar> studies.
Again conflates the two objections, and sounds rather hyperbolic and
alarmist as well.
(What proportion of the refereed clinical literature is accompanied by
a concurrent editorial? Why can't similar arrangements be made for this
subset in the new medium too? There could even be two different
formal acceptance criteria for referees if need be: R = "accept
unconditionally." RE = "accept only on condition of being accompanied by
editorial." Is this a failure of imagination or animus against the new
medium for some unstated reason?)
ar> And in any case, policies adopted by the Journal, and many other
ar> leading clinical journals, already allow any studies with urgent major
ar> implications for public health and safety to be released immediately
ar> after final editorial review and acceptance.
And so...?
ar> most clinical journals already have their own Web sites, which do much
ar> of what is proposed by E-biomed.
No doubt. But here I think we come to the heart of the matter (and
perhaps to those "unstated reasons"), for those Web sites happen to be
accessible only via Subscription/Site-License/Pay-Per-View (S/P/L), the
very access barriers from which self-archiving is meant to free the
literature.
In fact, the one thing "proposed by E-biomed" that no proprietary
journal Web site can or will do is to free the literature for one and
all, everywhere, forever.
ar> In addition, on-line data bases such as Medline are regularly used to
ar> search the published clinical literature. Electronic data bases should,
ar> and will, continue to grow in clinical medicine
Except that alas many of these secondary providers do this for a fee,
and none retrieve the full texts for free (because most are not yet
self-archived, hence not yet available free).
But it is a foregone conclusion that once the entire biomedical corpus
is self-archived in E-biomed for free, there will be (free) search and
navigational capabilities that will threaten the niche of commercial
secondary providers even more profoundly than they do the niche of
primary journal publishers.
(The latter will be able to downsize and restructure so as to continue
to provide the service of quality control -- peer review, editing,
certification -- paid for by the author-institution out of a small part
of the institutional S/L/P cancellation savings, but it is not at all
clear what the secondaries and tertiaries will be providing in this new
PostGutenberg world.)
http://www.cogsci.soton.ac.uk/~harnad/citation.html
ar> but they cannot replace the essential functions of peer-reviewed
ar> clinical journals.
E-biomed cannot and will not replace the journals' essential function of
peer review. But that is the ONLY function that journals will continue
to provide.
http://www.cogsci.soton.ac.uk/~harnad/nature.html
ar> I imagine that the proponents of E-biomed would reply that there is no
ar> intention to replace peer-reviewed journals. As long as accepted
ar> manuscripts were posted promptly on the E-biomed Web site, the NIH plan
ar> would not prevent the peer-reviewed clinical journals from continuing
ar> to review and publish original research articles as usual, together
ar> with whatever additional editorial and educational material they chose.
ar> Journals could also continue to maintain their own Web sites if they
ar> wished.
That would indeed be the quite obvious and correct response (indeed, it
has essentially been made above). Journals are free to continue to sell
an S/L/P version -- as long as dubious objections like the above ones
are not used as an excuse for delaying the optimal and inevitable
solution of public self-archiving the free online versions of their
(refereed) papers by their authors -- the same authors who have given
those same papers to the journals for free.
Only the improvement and certification added by peer review needs to be
paid for, and once S/L/P is no longer viable, institutional S/L/P
savings can amply cover those quality-control/certification (QC/C)
costs up-front on the author-institution end, instead of at the
access-blocking reader-institution end. But meanwhile E-biomed will also
allow authors to give away their refereed eprints to one and all for free.
ar> That response, while technically true, ignores the probably
ar> disastrous effects of E-biomed on journals. A flourishing E-biomed
ar> system that included clinical studies would very likely reduce the
ar> submissions, paid circulation, and income of most clinical journals
ar> enough to threaten their survival. Were this the price to be paid for a
ar> much better and less expensive clinical publishing system that would
ar> serve physicians and the public at least as well as the present
ar> arrangement and that would clearly facilitate the work of the clinical
ar> research community, I would take a more favorable view of the proposal.
ar> But I do not believe that most of the important functions of
ar> peer-reviewed clinical journals can be adequately replaced by E-biomed.
ar> And cost savings, noted by Varmus as one reason to adopt E-biomed, are
ar> not an issue with the leading clinical journals. In general, they are
ar> far less expensive than most basic-science journals.
Now it is becoming clear that concern about S/L/P revenue may indeed be
what is behind these objections.
As noted, of Dr. Relman's two sole substantive objections here, the
first only concerns unrefereed clinical preprints, and is a valid
objection, but it is also readily accommodated by E-biomed. The second
objection is invalid.
Is the revenue objection a third one? And is the fact that clinical
journals are "less expensive" a justification for continuing to hold
this literature hostage to S/L/P access tolls? Perhaps the simplest
answer comes from putting the question in principle to the authors of
all those clinical papers:
"Now that it is possible to make your refereed paper available
online to everyone and everyone everywhere for free by
self-archiving it in NIH's E-biomed, do you nevertheless prefer to
continue instead to restrict access to it to those individual and
institutional subscribers who can afford it -- even though you
don't get a penny of the revenue and have always given away the
reprints to all requesters for free, just as you gave the paper
itself for free to the journal (to sell), presumably because your
sole objective was to report your findings to one and all, once they
had been approved and accepted by peer review?"
No, I don't think that the goal of sustaining journals' current revenue
streams and modera operandi can or should persuade authors to keep
their research findings behind a financial firewall that is no longer
necessary -- particularly as there are obvious alternative ways for
journals to recover the surviving costs of their sole remaining
essential service -- QC/C -- out of S/L/P savings, once they have
downsized and restructured to provide that one essential service alone.
This "disastrous effect" sounds like a highly beneficial one for the
research community (both basic and clinical), and it sounds like
something the journals could successfully adapt to if they try, in the
interests of the research community they serve.
ar> A final worrisome aspect of E-biomed is that its proposed organization
ar> and management are so complicated as to raise doubts about its ability
ar> to function. It would be a huge conglomerate of different scientific
ar> fields, journals, editorial boards, and other "interested parties,"
ar> overseen by a necessarily very large and disparate governing board.
This was indeed a weakness of the original E-biomed, but it is all quite
easily and trivially remediable once it is recognized that E-biomed,
like Los Alamos, is merely a reliable, permanent infrastructure for the
SELF-ARCHIVING of all refereed and unrefereed papers by their authors
in the first instance, and eventually a facility for official journal
overlays, authenticated by the journals themselves.
http://www.nih.gov/welcome/director/ebiomed/com0509.htm#harn45
No new "complications" over and above the tried, tested and
spectacularly successful features of the 8-year-old Los Alamos Archive
are involved -- other than the special measures for the unrefereed
clinical sector called for by (among other things) Dr. Relman's own
sole valid objection!
ar> It takes a lot of work by editors to supervise a high-quality
ar> peer-review system.
Correct; and this is precisely the work that the QC/C charges will
cover out of the S/L/P savings.
ar> Even the simplified, two-reader system of approval envisioned
ar> as the alternative to editorial review by a journal would prove to be
ar> much more complicated than expected. How would differences between
ar> reviewers or between reviewers and authors be adjudicated? Suppose a
ar> reviewer's approval was conditional on suitable revision or correction
ar> of the manuscript. Who would oversee such negotiations? And who would
ar> be responsible for vetting the contentious issues that might arise
ar> later concerning corrections and commentary? All such functions are now
ar> carried out by the editors of peer-reviewed journals. I do not see how
ar> any system concerned about the quality of clinical data and their
ar> impact on the public health and medical practice could afford to ignore
ar> these questions.
This is conflating peer review (which will proceed exactly as before)
with the vetting of the unrefereed clinical preprint sector, a special
case that will be handled as the peers at NIH see fit, but should not
be confused with peer review, which will proceed apace.
ar> In clinical research, the best way to handle new data is to require
ar> rigorous peer review before their dissemination and, with few
ar> exceptions, to post the results in electronic data bases only after
ar> they have been published in carefully edited, peer-reviewed journals.
ar> That is because prepublication evaluation of the reliability of
ar> clinical studies and impartial assessment of their implications for
ar> health care are usually more important than the speed with which the
ar> data are made available.
I hope this has all been put in context and answered by now. Let me
close by saying that "speed" is only one of the virtues of E-biomed,
and by no means the foremost, which is the freeing of the literature
for one and all, everywhere, forever, to the eternal benefit of
research and researchers the world over, especially those whose access
to the research corpus was blocked or limited by its current system
of financial firewalls.
I hope these remarks will be accepted in the spirit of cautionary
editorial quote/commentary that sometimes needs to accompany even
the nonclinical papers that appear in refereed journals...
--------------------------------------------------------------------
Stevan Harnad harnad@cogsci.soton.ac.uk
Professor of Cognitive Science harnad@princeton.edu
Department of Electronics and phone: +44 2380 592-582
Computer Science fax: +44 2380 592-865
University of Southampton http://www.cogsci.soton.ac.uk/~harnad/
Highfield, Southampton http://www.princeton.edu/~harnad/
SO17 1BJ UNITED KINGDOM ftp://ftp.princeton.edu/pub/harnad/
This archive was generated by hypermail 2b30 : Tue Feb 13 2001 - 16:23:05 GMT