Re: Use of Navigational Tools in a Repository

From: Stevan Harnad <>
Date: Thu, 9 Mar 2006 16:19:42 +0000

--From Peter Suber's Open Access News

    How well do search engines index the OA repositories?

    Frank McCown, Xiaoming Liu, Michael L. Nelson, Mohammad
    Zubair (2006) Search Engine Coverage of the OAI-PMH
    Corpus, IEEE Internet Computing, March/April 2006.

    Abstract: The major search engines are competing to index as much
    of the Web as possible. Having indexed much of the surface Web,
    search engines are now using a variety of approaches to index the
    deep Web. At the same time, institutional repositories and digital
    libraries are adopting the Open Archives Initiative Protocol for
    Metadata Harvesting (OAI-PMH) to expose their holdings, some of
    which are indexed by search engines and some of which are not. To
    determine how much of the current OAI-PMH corpus search engines
    index, we harvested nearly 10M records from 776 OAI-PMH repositories.
    From these records we extracted 3.3M unique resource identifiers
    and then conducted searches on samples from this collection. Of this
    OAI-PMH corpus, Yahoo indexed 65%, followed by Google (44%) and MSN
    (7%). Twenty-one percent of the resources were not indexed by any
    of the three search engines.

Posted in OAN by Peter Suber at 3/08/2006 10:15:00 PM.
Received on Thu Mar 09 2006 - 17:06:51 GMT

This archive was generated by hypermail 2.3.0 : Fri Dec 10 2010 - 19:48:14 GMT