Re: UK Research Assessment Exercise (RAE) review

From: Stevan Harnad <>
Date: Thu, 12 Dec 2002 20:42:35 +0000

Below are excerpts from some articles that appeared in the Times Higher
Education Supplement on December 6 2002 about potential changes in the
UK Research Assessment Exercise. The articles are not online, and I have
only excerpted the passages pertinent to my own proposal for making
the RAE simpler and cheaper, and at the same time more explicit and
accurate. [The article doesn't mention an essential component of my
proposal -- the online standardized RAE Curriculum Vitae for every
researcher which
would include other measurable indicators of research impact, such as
numbers of research students and grants (which are themselves correlated
with citation impact): The scientometric research assessment tools are
described at: ]

    Excerpts from:
    RAE review by Mandy Garbner
    Times Higher Education Supplement 06 December 2002

    The research assessment exercise review panel wants to see radical
    change... Besides funding, the RAE has been criticised for the way
    assessment is carried out... Sir Gareth Roberts, who is chairing
    the Joint Funding Bodies' review of the RAE, says... that few people
    contributing to the consultation on its future have come up with
    radical ways of changing it... [T]he review wants something more
    radical than tinkering... It focuses on creating a "less burdensome
    assessment method", rather than on funding.

    One of the more radical proponents of change is Southampton
    University's Stevan Harnad. He would like to see UK research made
    "accessible and assessable continuously online" rather than in
    a four-yearly process, as happens now. He argues that research in
    every discipline can continue to be refereed, but can be made freely
    accessible to all academics if they archive their own refereed
    research online, bypassing the access tolls charged by research
    publications. He says software can also determine research impact,
    that is, how much other researchers cite research and build on it...

    Harnad adds that the software to do this is available. All it
    needs is for the RAE to encourage the process "by mandating that
    all UK universities self archive all their annual refereed research
    in their own e-print archives". The benefits, he says, include "a
    far more effective and sensitive measure of research productivity
    and impact at far less cost" to both universities and the RAE and a
    strengthening of the uptake and impact of UK research by increasing
    its accessibility. "The UK is uniquely placed to move ahead with
    this and lead the world," Harnad says, "because the RAE is already
    in place."

    The public consultation on the future of the RAE closed last week. The
    review panel is assessing the contributions and will report shortly.

    David Clark: "The time involved in the process is
    mind-boggling. Universities set up committees between each RAE to test
    scenarios, model possible outcomes and consider reorganisations to
    maximise potential income from the next exercise. The preparation of
    submissions dominates universities in the run-up to the RAE. Yet the
    outcome is so strongly correlated with research income from external
    funders that the grading could be achieved at the press of a button."


Prior articles on this topic (2001)

Harnad, S. (2001) "Research access, impact and assessment." Times Higher
Education Supplement 1487: p. 16.

Holmes, Alison & Oppenheim, Charles (2001) Use of citation analysis to
predict the outcome of the 2001 Research Assessment Exercise for Unit of
Assessment (UoA) 61: Library and Information Management

Jaffe, Sam (2002) Citing UK Science Quality: The next Research
Assessment Exercise will probably include citation analysis.
The Scientist 16(22): 54, Nov. 11, 2002

Lawrence, S. (2001) Free online availability substantially increases a
paper's impact. Nature Web Debates.

Oppenheim, C., (1997) The Correlation Between Citation Counts and
the 1992 Research Assessment Exercise Ratings for British Research in
Genetics, Anatomy and Archaeology, Journal of Documentation , 53(5),
1997, pp 477-487

Oppenheim, Charles (1996) "Do citations count? Citation indexing and
the research assessment exercise," Serials, 9:155-61, 1996.

Smith, A. & M. Eysenck, "The correlation between RAE ratings and citation
counts in psychology," June 2002, available online at
Received on Thu Dec 12 2002 - 20:42:35 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:46 GMT