Re: A Note of Caution About "Reforming the System"

From: Greg Kuperberg <greg_at_MATH.UCDAVIS.EDU>
Date: Mon, 19 Feb 2001 09:36:28 -0800

On Mon, Feb 19, 2001 at 12:34:20PM +0000, Stevan Harnad wrote:
> On Sun, 18 Feb 2001, Greg Kuperberg wrote:
> > On Sun, Feb 18, 2001 at 06:11:52PM +0000, Stevan Harnad wrote:
> > >
> > > Could this be (and I am not asking this ironically), an elite
> > > minoritarian opinion?
> >
> > ...all of their work is taken as interesting just because
> > of their names. But except for degree, this is true of most people
> > who contribute to the arXiv. Most of us have solid reputations.
> (1) Can this really be true of even (most of) the annual ~30K authors
> in arXiv?

I will only speak for mathematics since that is my discipline.
In mathematics peer review really serves two different purposes: To weed
out papers that are *wrong*, and to segregate work by how *important*
it is. It has really been true all along, since decades before the math
arXiv even began, that most mathematicians can trust each other to produce
correct work most of the time. This level of trust is hardly miraculous,
since mistakes are, relatively speaking, spelled out in black and white.
While making a mistake is okay, it is outrageous to refuse to admit that
your work has one. We don't divide into factions who endlessly dismiss
each others' work as nonsense.

You may wondering how we are sure that this trust exists when we do after
all have anonymous journal refereeing. The answer is that we see it in
informal seminars and other forums with the same rules of light
moderation as the math arXiv.

As far as how *interesting* work is, then we do divide into factions just
like everyone else. But this splintering is ultimately equivalent to
division into subdisciplines, since topologists are the ones interested in
topology, probabilists are interested in probability, etc. Although it
is a taboo topic, we silently understand that two subdisciplines can
be harder and easier sides of the same area. This structure can lead
to fights over hiring, but there is no point in quarreling over the
literature, because each subdiscipline can have its own space.

> Has the growing and time-consuming exercise of peer review for over a
> century been a waste of time, where NAME-RECOGNITION and SELF-POLICING
> would have vouchsafed the same quality without all that waste of time and
> effort?

I do wonder whether name recognition and self-policing would have
vouchsafed the same quality all along. For one thing, name recognition
and self-selection are already major factors in traditional refereeing.
However, that doesn't mean that journal refereeing has been a century's
waste of time. Its initial purpose was to ration a once scarce resource,
namely printing and postage. It is all well and good if you can trust
most of your colleagues, but if you can only afford to receive the papers
of some of them, referees have to decide which ones. The arXiv suggests
that this function of traditional refereeing in mathematics is obselete.
If so, we would need to reform peer review to restore it as a useful
filter for readers.

The other purpose of journal refereeing is credit for employment and
promotion. This function is still necessary in academia. But since it
is an adaptation of refereeing and not the raison d'etre, it too begs
for reform.
--
  /\  Greg Kuperberg (UC Davis)
 /  \
 \  / Visit the Math ArXiv Front at http://front.math.ucdavis.edu/
  \/  * All the math that's fit to e-print *
Received on Wed Jan 03 2001 - 19:17:43 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:46:03 GMT