W3C home > Mailing lists > Public > public-lod@w3.org > March 2010

Re: Resolving DC PURLs

From: Nathan <nathan@webr3.org>
Date: Tue, 30 Mar 2010 01:00:27 +0100
Message-ID: <4BB13F1B.3060505@webr3.org>
To: David Wood <david@zepheira.com>
CC: Thomas Baker <tbaker@tbaker.de>, DCMI Architecture Forum <dc-architecture@jiscmail.ac.uk>, public-lod <public-lod@w3.org>
David Wood wrote:
> Hi Tom,
> 
> On Friday, 26 March, Ian Davis (CTO of Talis) reported that purl.org was rejecting some requests to Dublin Core PURLs [1].  I asked OCLC to increase the number of threads used by their PURL server and the maximum number of concurrent connections.  They complied this afternoon.
> 
> At the time of the failures, OCLC reported that purl.org was rejecting between 20 and 60 requests per second for DC terms [2].
> 
> It would seem that Linked Data clients are becoming more prevalent and that some of them are not particularly well behaved.  Perhaps we need to raise awareness of the importance of caching.

This is a big issue in general with linked data (specifically when
generated by scripts). Caching is at an all time low and it is having
serious effects.

One particular suspect is PHP (and thus the scripts created with it)
which has little to no support of anything http related let alone caching.

Raised awareness all round would be most beneficial to all.

Regards!

> This incident points out the criticality of DC terms to the Linked Data community and the fragility of a single point of failure such as purl.org.  The PURL Federation development recently announced by NCBO and Zepheira may eventually serve to remove the single point of failure, but the criticality of service is likely to get worse with time.
> 
> This message is simply an advisory to the DC community of some practical issues arising from the use of DC terms by the Linked Data community and requires no immediate action.  I would ask, though, that awareness of these issues be kept in mind as Dublin Core considers the management of its identifiers.
> 
> [1]  http://twitter.com/IanD
> [2]  Personal correspondence from Tom Dehn at OCLC
> [3]  http://zepheira.com/publications/news/#PURLFederationDevelopment
> 
> 
> Regards,
> Dave
> --
> David Wood, Ph.D.
> Partner
> Zepheira - The Art of Data
> http://zepheira.com/team/dave/
Received on Tuesday, 30 March 2010 00:01:09 UTC

This archive was generated by hypermail 2.3.1 : Sunday, 31 March 2013 14:24:25 UTC
http://lists.w3.org/Archives/Public/public-lod/2010Mar/0294.html
lists.w3.org
crawl=CC-MAIN-2013-48
10129
application/http; msgtype=response
sha1:2EHDVH73HKIFTCGODHO5OX2WXBH4SU3W
<urn:uuid:20e83c91-b6f0-4e1f-9597-18f1da88587c>
2013-12-13T03:29:42Z
128.30.52.56
sha1:CYXVPMWGLDTQHXC4BA53V5GVPNMASV4X
<urn:uuid:021d8c20-4a0e-471b-b88c-c6f90c4c975a>
http://lists.w3.org/Archives/Public/public-lod/2010Mar/0294.html
response
<urn:uuid:cd495487-058f-4236-80a6-fe694291665f>
robots: classic
hostname: ip-10-33-133-15.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-48
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Winter 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf