Re: UK "RAE" Evaluations

From: Fytton Rowland <J.F.Rowland_at_LBORO.AC.UK>
Date: Wed, 22 Nov 2000 10:20:22 +0000

This issue may vary between disciplines. The UK Research Assessment
Exercise is carried out by subject panels (made up of senior academics in
the discipline), who have considerable independence in how they make their
assessments. Whatever the basic rules of the RAE may say, individual
subject panels may well apply different rules. The Library and Informatiom
Management panel has no problem with electronic journals without page
numbers, so far as I know; the quite well thought of Journal of Digital
Information (jointly edited from Stevan's department and mine, as it
happens) is certainly accepted by them, and it has no page numbers.

Chemistry is a deeply conservative discipline, and probably the most
obsessed with journal publication. But it should be possible to obtain a
statement from the Chair of the Chemistry Panel about the eligibility of
elctronic journals.

Fytton Rowland.

At 06:31 PM 11/21/00 +0000, Stevan Harnad wrote:
>On Tue, 21 Nov 2000, "Rzepa, Henry" < wrote:
>> The UK runs "Research assessment exercises" in subject disciplines
>> periodically. Ours (Chemistry) is coming up shortly.
>> I enquired what the status of my "e-only" work might be. The answer,
>> approximately, was that if it does not carry a page number, its not a
>> journal.
>> Publication in "journals" is of course one way in which my particular
>> discipline judges quality and international significance.
>> As the editor of four "e-print" conferences in chemistry, archived as
>> four CDROMs "without page numbers", I infer (to be confirmed) that
>> none of this counts towards the RAE, or at least not at the same level
>> as "page numbers".
>> As it happens, I have hedged my bets, and published, hopefully
>> adequately, in journals that DO carry page numbers. But if ones tenure,
>> or future, were to depend on "page numbers", it could be a worrying
>> period for some.
>> If push came to shove, and someone were "shoved" because their "page
>> numbers" were inadequate, I wonder if that could be tested legally, and
>> whether the "e-print" would prevail?
>I seriously doubt that page-numbers are or ever were a stumbling block
>with RAE. Here is the pertinent statement from the last RAE:
> 46. Material embodying research outcomes published by electronic
> means may be cited. Refereed journal articles published through
> electronic means will be treated on the same basis as those
> appearing in printed journals.
>This does not sound like any sort of double-standard to me.
>On the other hand, please bear and mind (and keep separate) the
>following further issues:
>(1) For the RAE or any other evaluation, not all refereed journals are
>on a par: There are journals of higher quality and lower quality; this
>is sometimes (but not always) correlated with the journal's ISI "impact
>factor," and sometimes also with its age (older, established journals
>may tend to be of higher quality and impact than newer start-ups, or
>may have established their eventual quality and prestige more fully).
>(2) These quality differences among journals can and and should be taken
>into account if such assessment exercises are to be conducted at all.
>It is then likely that newer start-ups will not be weighted as heavily
>as older established journals; it is also likely that the online-only
>journals will be among the newer start-ups, rather than the older,
>established journals, which will all be paper-based (although most will
>now also have an on-line version too). There are, however, exceptions
>to the rule that newer (hence on-line-only) start-up journals will have
>lower quality and impact, viz., The Journal of High Energy Physics
> "JHEP has proved to be the right answer to the need for a new path
> for scientific communication. Indeed, JHEP is an efficient and
> cheap alternative to conventional publishing that nevertheless
> maintains essential publishing features: quality control, easy
> retrieval, and archival responsibility for the future.
> "In order to illustrate the success of the experiment, launched in
> July 1997, let the figures do the talking. JHEP is by now a leading
> journal in the field. The number of consultations is outstanding
> and the impact factor as high as the best in the field."
>(3) It goes without saying that a refereed journal is a refereed
>journal, regardless of the medium, on-line or on-paper. In some fields,
>refereed conference proceedings are just about on a par with refereed
>journals (but again, they need time to establish their quality levels
>and impacts). What Dr. Rzepa neglects to indicate, however, is whether
>the "'e-print' conferences in chemistry" of which he writes are (i)
>refereed, and if so, (ii) whether they have yet established their
>refereeing quality standards (and if so, (iii) where they stand in the
>quality hierarchy).
>None of this is intended as advocacy of bean-counting. But if we are
>going to count beans, there are factors that will invariably be taken
>into account that prevent all publications from being treated on a par.
>The presence of page-numbers, however, will not be one of them.
>Stevan Harnad
>Professor of Cognitive Science
>Department of Electronics and phone: +44 23-80 592-582
> Computer Science fax: +44 23-80 592-865
>University of Southampton
>Highfield, Southampton
>NOTE: A complete archive of the ongoing discussion of providing free
>access to the refereed journal literature online is available at the
>American Scientist September Forum (98 & 99 & 00):
>You may join the list at the site above.
>Discussion can be posted to:

Fytton Rowland, M.A., Ph.D., F.I.Inf.Sc., Lecturer,
Deputy Director of Undergraduate Programmes and
Programme Tutor for Publishing with English,
Department of Information Science,
Loughborough University,
Loughborough, Leics LE11 3TU, UK.

Phone +44 (0) 1509 223039 Fax +44 (0) 1509 223053
Received on Mon Jan 24 2000 - 19:17:43 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:45:58 GMT