Re: A Note of Caution About "Reforming the System"

From: Stevan Harnad <harnad_at_COGPRINTS.SOTON.AC.UK>
Date: Sun, 18 Feb 2001 18:11:52 +0000

On Sun, 18 Feb 2001, Greg Kuperberg wrote:

> First, since you like to talk about "access barriers", refereeing
> itself is the single biggest access barrier to journal papers in math,
> simply because it takes so long.

You're quite right. But the fact that we are all overloaded -- and that
there is too much research that needs our refereeing to allow most of
us to immediately put it on the top of our stacks -- is unfortunately a
fact of current-scale life!

So I don't see any way around that human ergonomic "access-barrier" to
the final, refereed, revised draft. But certainly it is now possible to
make the pre-refereeing draft available on a scale as broad as
publication, and we should all do it (or at least those of us who are
not worrying about patents)!

I am hopeful, though, that as online implementation of refereeing gets
more streamlined, it will make it possible to re-distribute the refereeing
load in a way that will minimize turnaround times on people's stacks....

> Second, to the surprise of many new users, arXiv articles in the
> aggregate are about as good as published papers. Yes, there is the
> "invisible hand theory". You might be firmly convinced by this theory,
> but most of us are firmly undecided.

What we should all be firmly undecided about is its alternative. Since
absolutely nothing has changed insofar as universal answerability to
peer review is concerned, surely the null hypothesis is that the causal
factors remain as they have always been.

It is an empirical question, however -- and one that I hope our survey
and then some direct "DIFFing" on successive drafts, pre- and
post-refereeing, will help answer -- how much difference there really
is between pre- and post-refereeing drafts.

Note, though, that full empirical answer to this question will not even
give an epsilon of a clue as to what the size of that difference WOULD
BE if there were no longer the universal answerability, and the
peer-review and certification system simply vanished.

That would require some rather radical experimentation. You, alas, are
prejudging what the outcome would be...

> > Please ask your colleagues [to] participate in the survey:
> >
> >
> I filled it out. It has some interesting questions but it is twice
> too long.

Many thanks. We worried about that in constructing it. It's a trade-off between
getting an incomplete picture, by asking too few questions, and not getting
enough respondents, by asking too many questions.

We have not yet announced the survey far and wide. I hope that when we do, the
numbers will still prove substantial. If not, we may devise an alternative "short

> > Alas, it is hard to draw empirical conclusions from an N of 1...
> There are very few people who are hip enough to use the arXiv but who
> don't have to worry about employment or promotion.

It'll be interesting at least to hear what "Simon says," across disciplines.

> Other people in a similar position have told me that
> they think that journals are superficial, and that the only reason that
> they still publish is to please other people, either journal editors
> or administrators.

Could this be (and I am not asking this ironically), an elite
minoritarian opinion? the opinion of the top tail of the gaussian
distribution, of the ones who are almost peerless anyway, and can
police themselves as rigorously today as they did in Newton's day? How
many of the annual quota of at least 300K papers in math/phys/astro is
of that calibre?

Stevan Harnad
Professor of Cognitive Science
Department of Electronics and phone: +44 23-80 592-582
             Computer Science fax: +44 23-80 592-865
University of Southampton
Highfield, Southampton

NOTE: A complete archive of the ongoing discussion of providing free
access to the refereed journal literature online is available at the
American Scientist September Forum (98 & 99 & 00 & 01):

You may join the list at the site above.

Discussion can be posted to:
Received on Wed Jan 03 2001 - 19:17:43 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:03 GMT