Perils of Press-Release Journalism: NSF and Chronicle of Higher Education

From: Stevan Harnad <amsciforum_at_GMAIL.COM>
Date: Wed, 25 Feb 2009 18:36:12 -0500

In response to my critique of his Chronicle of Higher Education
posting on Evans and Reimer's (2009) Science article (which I
likewise critiqued, though much more mildly), I got an email from
Paul Basken asking me to explain what, if anything he had got wrong,
since his posting was based entirely on a press release from NSF.
Sure enough, the silly spin originated from the NSF Press release
(though the buck stops with E & R's vague and somewhat tendentious
description and interpretation of some of their findings). Here is
the NSF Press Release, enhanced with my comments, for your
delectation and verdict:

      If you offer something of value to people for free while
      someone else charges a hefty sum of money for the same
      type of product, one would logically assume that most
      people would choose the free option. According to new
      research in today's edition of the journal Science, if
      the product in question is access to scholarly papers and
      research, that logic might just be wrong. These findings
      provide new insight into the nature of scholarly
      discourse and the future of the open source publication
      movement[sic, emphasis added].

(1) If you offer something valuable for free, people will choose the
free option unless they've already paid for the paid option
(especially if they needed -- and could afford -- it earlier).

(2) Free access after an embargo of a year is not the same
"something" as immediate free access. Its "value" for a potential
user is lower. (That's one of the reasons institutions keep paying
for subscription/license access to journals.)

(3) Hence it is not in the least surprising that immediate
print-on-paper access + (paid) online access (IP + IO) generates more
citations than immediate (paid) print-on-paper access (IP) alone.

(4) Nor is it surprising that immediate (paid) print-on-paper access
+ online access + delayed free online access (IP +IO + DF) generates
more citations than just immediate (paid) print-on-paper + online
access (IO + IP) alone -- even if the free access is provided a year
later than the paid access.

(5) Why on earth would anyone conclude that the fact that the
increase in citations from IP to IP + IO is 12% and the increase in
citations from IP + IO to IP + IO + DF is a further 8% implies
anything whatsoever about people's preference for paid access over
free access? Especially when the free access is not even immediate
(IF) but delayed (DF)?
      Most research is published in scientific journals and
      reviews, and subscriptions to these outlets have
      traditionally cost money--in some cases a great deal of
      money. Publishers must cover the costs of producing
      peer-reviewed publications and in most cases also try to
      turn a profit. To access these publications, other
      scholars and researchers must either be able to afford
      subscriptions or work at institutions that can provide

      In recent years, as the Internet has helped lower the
      cost of publishing, more and more scientists have begun
      publishing their research in open source outlets online.
      Since these publications are free to anyone with an
      Internet connection, the belief has been that more
      interested readers will find them and potentially cite
      them. Earlier studies had postulated that being in an
      open source format could more than double the number of
      times a journal article is used by other researchers.

What on earth is an "open source outlet"? ("Open source" is a
software matter.) Let's assume what's meant is "open access"; but
then is this referring to (i) publishing in an open access journal,
to (ii) publishing in a subscription journal but also self-archiving
the published article to make it open access, or to (iii)
self-archiving an unpublished paper?

What (many) previous studies had measured (not "postulated") was that
(ii) publishing in a subscription journal (IP + IO) and also
self-archiving the published article to make it Open Access (IP + IO
+ OA) could more than double the citations, compared to IP + IO
      To test this theory, James A. Evans, an assistant
      professor of sociology at the University of Chicago, and
      Jacob Reimer, a student of neurobiology also at the
      University of Chicago, analyzed millions of articles
      available online, including those from open source
      publications and those that required payment to access.

No, they did nothing of the sort; and no "theory" was tested.

Evans & Reimer (E & R) only analyzed articles from subscription
access journals before and after they became accessible online (to
paid subscribers only) (i.e., IP vs IP + IO) as well as before and
after the online version was made accessible free for all (after a
paid-access-only embargo of up to a year or more: i.e., IP +IO vs IP
+ IO + DF). Their methodology was based on comparing citation counts
for articles within the same journals before and after being made
free online at various intervals.
      The results were surprising. On average, when a given
      publication was made available online after being in
      print for a year, being published in an open source
      format increased the use of that article by about 8
      percent. When articles are made available online in a
      commercial format a year after publication, however,
      usage increases by about 12 percent.

In other words, the citation count increase from just (paid) IP to
(paid) IP + IO was 12% and the citation count increase from just
(paid) IP + IO to (paid) IP + IO + DF was 8%. Not in the least
surprising: Making paid-access articles accessible online increases
their citations, and making them free online (even if only after a
delay of a year) increases them still more.

What is surprising is the rather absurd spin that this press release
appears to be trying to put on this unsurprising finding.
      "Across the scientific community," Evans said in an
      interview, "it turns out that open access does have a
      positive impact on the attention that's given to the
      journal articles, but it's a small impact."

We already knew that OA increased citations, as the many prior
published studies have shown.  Most of those studies, however, were
based on immediate OA (i.e., IF), not embargoed OA. What E & R do
show, interestingly, is that even delaying OA for a year still
increases citations, though not nearly as much as immediate OA (IF).
      Yet Evans and Reimer's research also points to one very
      positive impact of the open source movement that is
      sometimes overlooked in the debate about scholarly
      publications. Researchers in the developing world, where
      research funding and libraries are not as robust as they
      are in wealthier countries, were far more likely to read
      and cite open source articles.

A large portion of the citation increase from (delayed) OA turns out
to come from Developing Countries (refuting Frandsen's recent report
to the contrary). (A similar comparison, within the US, of citations
from the Have-Not Universities (with the smaller journal subscription
budgets) compared to the Harvards may well reveal the same effect
closer to home, though probably at a smaller scale.)
      The University of Chicago team concludes that outside the
      developed world, the open source movement "widens the
      global circle of those who can participate in science and
      benefit from it."

And it will be interesting to test for the same effect comparing the
Harvards and the Have-Nots in the US -- but a more realistic estimate
might come from looking at immediate OA (IF) rather than just
embargoed OA (DF).
      So while some scientists and scholars may chose to pay
      for scientific publications even when free publications
      are available, their colleagues in other parts of the
      world may find that going with open source works is the
      only choice they have.

It would be interesting to hear the authors of this NSF press release
-- or E & R, for that matter -- explain how this paradoxical
"preference" for paid access over free access was tested during the
access embargo period...

Stevan Harnad
American Scientist Open Access Forum
Received on Wed Feb 25 2009 - 23:39:01 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:49:43 GMT