On Wed, Dec 06, 2000 at 10:15:26AM +0000, Stevan Harnad wrote:
> Are mathematicians really so different from ordinary human beings that
> SC works for them, even though it doesn't work for anyone else? I'm
> prepared to believe it, but I'd prefer to see the objective evidence
> (personal experience being no such evidence).
First let me make two points about the implications of this question.
Even if your analogy between the anonymous referee system and meat
inspection were accurate, it would not necessarily justify the bias of
giving separate names to "preprints" and "postprints". The authorities
may inspect meat, but they do not dye it a different color and rename it.
And even when informal self-control is more important than formal quality
control, that does not mean that the latter is unimportant. Not only do
I believe that formal peer review is important, I think that it should
be improved and made more important.
That said, no, mathematicians are not all that different from other
people. Mathematics itself may be different from some disciplines.
In mathematics informal review by self-appointed experts --- what you
called vigilantism --- works pretty well. Most mathematicians are much
more worried that any other mathematician might find a devastating
error in one of their papers, published or not, than that the papers
will be rejected by journals. When I write a paper, I have absolutely
no control over who might read it and find a mistake, but I am free
to choose the journal. And if I had bad judgement I might well use
that freedom to my short-term advantage. At the same time, although
research in mathematics is not quite as rigorous as, for example, network
software development, it is rigorous enough that self-appointed critics
can quickly earn credibility.
So for my own work I worry a lot about finding all possible mistakes
before I contribute anything to the arXiv, for that matter even before I
give a seminar on unwritten new results. I have found that the anonymous
referee helps some for this purpose, but not all that much. About half
the time the referee report gives me the feeling that the referee didn't
really follow the crucial details of the arguments. And as a referee I
have sometimes had to trust the author quite a bit as well.
This state of affairs is certainly relevant to arXiv policy. Some of the
predecessors of the math arXiv deliberately set expiration dates for all
holdings in order to respect the dichotomy between ephemeral preprints and
immortal postprints. Indeed the physics part of the arXiv (then called
hep-th) initially had a 3-month expiration policy which, however, was
retracted after less than 3 months. Eventually the arXiv adopted the
diametrically opposite policy that every version of every submission
should be immortal. Part of the purpose of this is to sharpen the
author's incentive to guard his reputation, i.e., to further strengthen
the informal peer review system. Most mathematicians that I have
spoken to greatly appreciate this effect. One interesting consequence
of the policy is that you can search for all of the "withdrawn" papers,
meaning those in which the latest version begs the reader not to read
previous versions:
http://front.math.ucdavis.edu/search/withdrawn
One proposed name for this list is "The Avenue of Broken Dreams".
Mathematics is by no means unique in relying on informal review.
Whether or not formal or informal review works better depends on
the incentives of the people involved. For food safety I agree that
formal inspections work a lot better than vigilantist accusations.
But informal review is very important for detecting fraud in fine art,
for example. Another example is commercial versus open source software.
Most commercial operating systems are formally certified from time to
time, while open source systems such as Linux is mostly kept true by
informal peer review. In this case I trust the informal approach more.
In my opinion the success of the math arXiv in general, and the Avenue of
Broken Dreams in particular, is evidence that vigilantism in mathematics
and physics really works. Most arXiv papers that are ever withdrawn are
withdrawn before they are formally refereed, which in mathematics usually
takes three months to a year. In addition, most authors never do add the
journal references for the arXiv listings, although I would prefer that
they did. I know that your explanation of this evidence is the "invisible
hand theory". I grant you that the journals do extend an invisible
hand into the arXiv and that it does help some to uphold standards.
But I believe that it is a relatively weak hand. I think that the onus
is on the proponents of the invisible hand theory to prove its strength.
--
/\ Greg Kuperberg (UC Davis)
/ \
\ / Visit the Math ArXiv Front at http://front.math.ucdavis.edu/
\/ * All the math that's fit to e-print *
Received on Mon Jan 24 2000 - 19:17:43 GMT