Adminstrative info for SIGMETRICS (for example unsubscribe):
http://web.utk.edu/~gwhitney/sigmetrics.html On Thu, Nov 25, 2010 at 6:30 PM,
Ludo Waltman <ludo_at_ludowaltman.nl> wrote
There is an essential difference between what Stevan Harnad says and
what I am saying. Stevan states that the results of Phil Davis can
be interpreted as "simply the failure to detect any citation
advantage at all". He also states that "this failure to replicate is
almost certainly due to the small sample size as well as the short
time-span". We seem to agree that no OA citation advantage can be
detected in Phil's data. What we do not agree about is whether or
not this is likely to be caused by a small sample size. As indicated
in my previous post, I consider this to be unlikely. Phil's results
show not only that there is no OA citation advantage in his sample,
but also that an OA citation advantage of reasonable size is
unlikely to exist in the underlying population.
It depends on what you mean by "the underlying population." If you mean the 3
-year population of articles in the 36 (mostly APS) journals that Phil actually
tested, I agree.
But if you mean that general population of articles, in multiple different
fields, tested by multiple independent investigators, reporting a significant
(and sometimes quite sizeable) OA citation advantage (with the exception of a
very small number of negative or null outcomes), then Phil's study certainly has
*not* shown "that an OA citation advantage of reasonable size is unlikely to
exist" in *that* underlying population. It would take a null meta-analysis, not
just one null outcome, to be able to show that. (Otherwise any repeatedly
observed effect could be dismissed on the basis of one non-replication!)
What Phil set out to show was that the repeatedly observed OA citation advantage
was an artifact of author self-selection, and the way he hoped to show that --
and it's a good way -- was to show that randomly imposed OA eliminates the OA
citation advantage. So far so good. But what Phil failed to do was to replicate
the OA citation advantage in the case of author self-selection, and then show
that it is eliminated by randomization -- in the same journal population and
time interval.
Instead he just found no OA citation advantage at all, for that journal
population and interval.
That, to repeat, is one non-replication, for that journal population and time
interval. It is not a demonstration that the repeatedly observed OA citation
advantage is an artifact of author self-selection.
And -- not to focus only on negative results -- let me suggest that it is
pertinent to this question that we too did a test of the self-selection
hypothesis -- on a much larger sample across more fields and a longer time
interval -- and we not only found "an OA citation advantage of reasonable size"
for self-selected OA, but we found that the advantage was the same size for
mandated OA.
We accordingly conclude that "an OA citation advantage of reasonable size is
likely to exist in the underlying population" if you test for it, and your
sample is big enough and long enough -- except perhaps in Phil's sample of
(mostly APS) journals...
Stevan Harnad
On Wed, Nov 24, 2010 at 11:52 PM, Ludo Waltman
<ludo_at_ludowaltman.nl> wrote:
LW:
Phil Davis has published interesting results on the
question whether open
access leads to a citation advantage. In my
view, Stevan Harnad's criticism
of Phil misses the point...
In my view, Phil has convincingly shown that, at least
for the journals and
the time intervals he studied, there is no
meaningful OA citation advantage.
I don't understand Ludo Waltman's point, since this is
exactly what I said:
SH:
Phil Davis's dissertation results are welcome and
interesting, and include
some good theoretical insights, but insofar
as the OA Citation Advantage is
concerned, the empirical findings turn out
to be just a failure to replicate
the OA Citation Advantage in that particular
sample and time-span... it is
most definitely not a demonstration that the
OA Advantage is an artifact of
self-selection, since there is no control
group demonstrating the presence
of the citation advantage with self-selected
OA and the absence of the
citation advantage with randomized OA across
the same sample and time-span:
There is simply the failure to detect any
citation advantage at all.
Stevan Harnad
Received on Thu Nov 25 2010 - 21:46:14 GMT