Here follows a rather lengthy discussion partially in broken english;
maybe you can wait for the summarizing until the dust settled.
Eberhard R. Hilf (ERH)
::::::::::::::::::::::::::::::::::::::
ERH:
> > Refereed means in an archive, that the paper has passed.
Stevan Harnad:
> This is incorrect. This is not what "refereed" means, and here, as a
> reply to the above query, it can only cause confusion. "Refereed" does
> not mean in an archive; and it is unclear what it would/should mean to
> "pass" in an archive.
Dear Stevan, not confusion but broken english on my side:
I meant: If an archive registers a paper with the tag 'refereed',
this paper has to have been accepted by a refereeing process,- not
necessarily of the archive itself, but of any publisher, journal,
institution, which has applied a professional refereeing system.
[Of course the copyright must allow the storage of such a paper in
the archive].
Clear advantage of the e-age: separation of archives and refereeing
process is made possible.
Stevan:
> "Refereed" means having successfully passed peer review by a
> refereed, journal, a refereed conference proceedings, or some other
> established and recognized form of peer review.
ERH: that is what I meant.
ERH:
> > Otherwise it is 'sent back' and the author can do what he wants.
> > As long as a journal has not accepted it.
Stevan:
> Unclear: Sent by by whom, from what, for what reason?
ERH: That is what most authors do: if not accepted by one journal, they
send it to another, and repeat this process until it is accepted somewhere,
going down the ladder of esteem of quality of the journals.
[In physics we start with PRL and end with ..]
Normally when a journal rejects the paper, 'there is no trace left'
(no time stamp of 'I am the first'; no public note 'rejected by').
The author is set back to the status quo ante.
Clear advantage of the e-age: time stamps independent of refereeing
are possible.
Stevan:
> If you mean a paper is unrefereed until/unless it has been accepted by
> a refereed journal, that is correct. But then please make it clear that
> what you mean here is that it has been submitted to and refereed by (and
> "sent back by") a journal (etc.), not an archive. The poster's question
> was about whether to archive a journal-rejected article in an archive
> as "refereed" or "unrefereed." (The rationale for the question had
> presumably been that in a sense it HAS been refereed, only it has
> failed to be accepted.) Hence the clear answer to the poster is: if the
> journal has rejected it, it is not "refereed," and should be archived
> as unrefereed.
ERH: that is what I meant.
ERH:
> > P.S.: But be aware: in an e-archive you can have many more subtle and
> > precise levels of certification. And they are an advantage and make
> > the e-print so much more powerful than just the refereed/unrefereed,
> > saying nothing about the quality of the act.
Stevan:
> It is important to point out that the above is not a fact, but merely a
> speculation by Ebs about a hypothetical future. The fact is that it is
> journal refereeing -- and especially the established quality level and
> standards (and impact factor) of the particular journal that has accepted
> the paper in question -- that provides the only official certification
> at the present time. Nor is there yet any evidence whatsoever of
> "more subtle and precise levels of certification." Unless Ebs can
> cite references indicating exactly what certifiers he is referring to
> (and what the evidence it that they are "more subtle and precise" than
> standard peer review, it is important that he make it clear that he is
> merely speculating at this time.
ERH: Stevan, not hypothecial future but personal vision and possibility.
The future is not 'coming over us' but we, the community of
professionals, are in charge of having visions, discussing them, and
actively influencing developments.
ERH:
> > So, create a field -- "certification" -- and give it a list of
> > possiblities, say c0 - c7.
> >
> > For example:
> >
> > c=0 author thinks paper should be archived
Stevan:
> This seems trivial. Would an author self-archive a paper that he did not
> think should be archived? The AUTHORNAME tag seems to cover this.
ERH: not trivial, but systematic. There are many authors, which do not
fulfill c=0: anonymae, pp.
ERH:
> > c=1 author is a professional by attached homepage showing his PhD in the
> > field or his prof. position in a profess. institution of the field.
Stevan:
> This is not the "certification" tag but the DEPARTMENT/INSTITUTION tag
> (and URL) (and perhaps an optional DEGREES tag). Again, re-interpreting
> this as "certification" is confusing and trivializing certification.
ERH:
No, I do not think so. Most sophisticated and established (you asked for
examples) is the system of CERN: they have several levels of
certification inside CERN, tagged by 'individual scientist, group, etc.
up to the final level of CERN-paper. (By chance it seems that their last
level is of higher quality than the refereeing of almost any refereed
journal (rejection rate zero for such CERN papers)
ERH
> > c=2 a technical check has been made (formats, metadata, etc. by
> > the archive)
Stevan:
> This seems superfluous. It is not "certification" of the content. It is
> merely an archive's own routine checking of metadata. Appearing in that
> archive is de facto attestation to the fact that the paper has met that
> archive's criteria (whatever those happen to be).
ERH: That's is what I meant. Not certification of content but
certification of form, and presentation. As a reader, I want to be assured
that the paper is readable. (That is not always the case: PhD.theses by
some University archives, one cannot read the mathematical formulae;
Elsevier at some time, did decide to override the wishes of the author
to correctly write differentials, 'because it is cheaper, not to care
for this'). In many disciplines, form and content are not so easy to
separate, and should not be.
The confusion here in the discussion comes possibly from my choosing the
word 'certification' and putting it into a sequence of label numbers.
But this is purely technical. I argue for more fully exploiting and
making explicit for an archive what they present to the reader and make
use fully of the possibilities.
ERH:
> > > c=3 a library expert has read the paper >
Stevan:
> So what? The relevant experts are specialists in the field of the paper,
> not librarians.
ERH: The reality is richer: e.g. in German University Libraries they
have for each learned field (say: Physics) a professional in that field,
one who is quite able to screen papers such that a minimum of garbage
does not pass their filter.
But they are not experts on all topical fields of physics. Which is
(apparently the only level of certification, you have in mind, and what
the traditional topical journals do).
ERH:
> > c=4 a loose screening has been done by an external expert on that
> > field (topical screening)
Stevan:
> What does this mean?
ERH: As an example: Not long ago, Elsevier planned a new e-service of
assuring that every single paper appearing in ArXiv as a preprint is
refereed by a topical expert within 24 hours. ('First look').
Indeed, in my own field, I would be able to do this and thus give some
first look screening (sorting out not only 'cold fusion', but comparing to
the status of the field, and knowing the groups).
But: for many papers for a thorough refereeing for a highly respected
journal, it may take me weeks, it's another level of quality given by the
same person..
ERH:
> > c=5 a thorough blind refereeing has been done by a real expert.
Stevan:
> Until further notice, to be able to say this with the kind of authority
> needed so that people can trust it, and can know what to do with it, it
> has to be said by a refereed journal, with known quality standards. Hence
> "c-5" is conventional peer review, the only form of certification so far.
> This is covered by the REFEREED and JOURNAL-NAME tags.
ERH: Yes, but here we separate: readers will highly value it if this
does not remain the only quality information on the paper given by the
source presenting it on the web. We should not just try to mirror the
existing services of traditional journals but to exploit the full richness
of the e-prints; and that means more responsibility to say clearly what
an archive presents.
ERH:
> > c=6 paper has been annotated, commented by other professionals openly.
> > and so forth.
Stevan:
> This is peer commentary, not peer review. A valuable supplement to peer
> review, but not a substitute for it, and covered by the COMMENTARY and
> CITATION links:
> Peer Review, Peer Commentary, and Eprint Archive Policy
> http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/1926.html
ERH:
> > Since you as an archive will store the paper right from the beginning,
> > you just keep changing the value of c and keep the paper.
Stevan:
> These are not changing values of a certification "c" but additional,
> parallel pieces of metadata: AUTHORNAME, DEPARTMENT/INSTITUTION, JOURNALNAME,
> CITING PAPERS, COMMENTARIES, etc.
ERH: I agree. You seem not to like the internal labelling of different
meta-tags. That is an internal archive workflow question.
ERH:
> > Nothing is rejected, but the user is told what status the paper has.
>
> > Further reading:
> > E.R.Hilf and H.-J.Waetjen:
> > Scientific Refereeing in a Distributed World
> > http://isn-oldenburg.de/~hilf/vortraege/cern01/
Stevan:
> Institutional archives are neither for accepting/rejecting nor for
> certifying. They are for archiving their own research output. They can
> have internal criteria for what may be archived therein (e.g., only research,
> only research by researchers affiliated with the institution, and
> any further internal criteria the institution may wish to impose on its
> archive), but this has nothing to do with certification, peer review,
> or other established external indicators of quality-level (chief of which
> is JOURNAL-NAME at the present time).
ERH: That is a clear formulation of your personal view. And here is mine:
You try to tell us that only Journal-name and their refereeing is giving
certification, peer review and quality levels. The reality is different:
Numerous research institutions and research groups have their own
paper-lines, for historic reasons called 'Preprint' often
(e.g., the BROWN BAG SERIES of Caltec Astrophysics Laboratory or the yellow
line of the T.University of Darmstadt, or the green one of the Small
Systems group of Oldenburg) where only those papers are added and
distributed that have undergone a refereeing of a topical expert.
For historic and technical reasons, topical journals with their blind
refereeing were the only quality filter in the past.
With the eprint age, we as scientific community have to improve the
professional information of our colleagues by exploiting and implementing
all means of filtering that will assist in finding the material we
seek.
Eberhard R. Hilf, Dr. Prof.i.R.;
CEO
Institute for Science Networking Oldenburg GmbH
an der Carl von Ossietzky Universitaet
Ammerlaender Heerstr.121; D-26129 Oldenburg
http://www.isn-oldenburg.de/
my homepage:
http://isn-oldenburg.de/~hilf
hilf_at_isn-oldenburg.de
tel/Fax: +49-(0)-441-798-2884/5851
Received on Wed Aug 07 2002 - 12:47:39 BST