Re: UK Research Assessment Exercise (RAE) review

From: Stevan Harnad <harnad_at_ecs.soton.ac.uk>
Date: Sat, 23 Nov 2002 03:28:27 +0000

On Fri, 22 Nov 2002, David Goodman wrote:

> I do not think that the comparison of the eventual value of the
> different specialties of scientific research can be judged at the
> time the research is being done.

"Can only be predicted with XX% reliability" is the statistically sound
way of putting it. And both XX and the time-span will vary (with time,
and field).

Assessors for research funding don't ask for 100% predictive accuracy. They
just want something like "Research/Researcher A is more likely than B"
(when funds are finite).

> That requires historical knowledge as well as scientometrics.

Yes, but hindsight is not a predictor (unless it picks out a predictive
pattern or index for the next time).

> This does imply a certain humility about the ability to use current
> knowledge as a valid basis for long term science policy.

I don't know about long-term science policy. The RAE just wants some
objective help in disbursing support for the next few years.
http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/2373.html

> Your second derivative technique, if the data are sufficiently
> accurate to support it, sounds like an exceeding nice way of measuring the
> potential short-term rise of a scientific field (or department). I would
> be reluctant to extrapolate very far into the future with such methods.

Extrapolate no further than your time-series correlations suggest you
have a statistical basis for extrapolating.

> For example, as judged by apparent current productivity, and its apparent
> valuation by the scientific world in general, scientometrics does not
> show very well. You and I know better, of course. :)

The time-line for the betting on scientometrics is still very
short, the field being new and its database growing. Its day is fast
coming, though, and open-access (along with the scientometric analzers
like http://citebase.eprints.org/ ) will help usher it in.

Stevan Harnad
Received on Sat Nov 23 2002 - 03:28:27 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:42 GMT