 

RePEc is not a spider

We frequently get requests for inclusions in RePEc, and often these are complaints that some papers on a university department web page or a personal home page are not being picked up. RePEc is not Google. RePEc does not have a web spider that wanders the web and looks for research in Economics. I do not even think it would be possible to do so, as identifying research and Economics on an automatic basis is very difficult.

Material listed on RePEc is submitted, either by about 1200 participating archives, that each have followed our instructions, or by authors themselves at the Munich Personal RePEc Archive (MPRA). No need to send us links or papers. Just make sure your publisher participates, and if not upload your papers at MPRA (what you can in general also do for published material, see this previous blog entry).

And if you are really interested in a web spider for Economics, there is the Economic Search Engine (ESE), which uses in part RePEc data to search and index the subset of the web most likely related to Economics.

About these ads

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 194 other followers

%d bloggers like this:
http://blog.repec.org/2010/05/13/repec-is-not-a-spider/?like=1&source=post_flair&_wpnonce=c49b096ccc
blog.repec.org
crawl=CC-MAIN-2014-23
63864
application/http; msgtype=response
sha1:2G7AAULNIPOZ3RQUIEBLIZRRXSBOXLUA
<urn:uuid:1db889fa-bc79-4d88-80cd-20a301623bb0>
2014-07-23T01:17:42Z
66.155.11.238
sha1:D3NFFNDLI7NXQRNEZ5E7RQBB5V6PM42F
<urn:uuid:a41ca543-307b-477f-bdfa-8b835d9611fd>
http://blog.repec.org/2010/05/13/repec-is-not-a-spider/?like=1&source=post_flair&_wpnonce=c49b096ccc
length
response
<urn:uuid:35c00c26-d1a8-4832-8f33-cd41b8d5f34a>
robots: classic
hostname: ip-10-33-131-23.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-23
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for July 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf