Re: Future UK RAEs to be Metrics-Based

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Mon, 18 Sep 2006 17:14:21 +0100

On Mon, 18 Sep 2006, Richard Poynder wrote:

> >CO:
> >However, thank heavens for (4) because, as you rightly point out, a
> >metrics based system will sweep all this nonsense aside.
>
> Does not a metrics based system pose some problems for the
> humanities? I haven't followed the discussion closely, but I get the
> feeling that this is primarily intended as a fix for the sciences
> isn't it? How is it envisaged working in the humanities?

Charles Oppenheim has authorised me to post this on his behalf:

    "Research I have done indicates that the same correlations between
    RAE scores and citation counts already noted in the sciences and
    social sciences apply just as strongly (sometimes more strongly)
    in the humanities! But you are right, Richard, that metrics are
    PERCEIVED to be inappropriate for the humanities and a lot of
    educating is needed on this topic."

    Professor Charles Oppenheim
    Head
    Department of Information Science
    Loughborough University

I would also add the following to what Charles has said:

(1) Wherever anyone has checked the correlation between journal citation
counts and RAE outcome, the correlation has always been significant
and sizeable, hence predictive. Humanities disciplines have not been
exceptions.

(2) In book-based fields, what has likewise not been looked at is
supplementing the journal-article citation metric with a book-citation
metric. (That's still metrics!)

(3) And then there are all the other candidate metrics, most still
untested: downloads, co-citations, hubs/authorities, recursive CiteRank,
download/citation growth parameters (latency, slope, peak, longevity),
semantic metrics, etc.

An a-priori declaration, free of any supporting evidence -- by any
discipline today -- that its work is an exception, not assessable by
metrics, makes about as much sense as an a-priori declaration, without
any supporting evidence, that a discipline's work is not assessable by
any form of comparative performance evaluation at all. (Who's to say whether
subjective evaluations have any validity either?)

Stevan Harnad
Received on Mon Sep 18 2006 - 17:20:50 BST