371 MB XML file from Google full of bookish goodness

By Bryan Rasmussen
June 25, 2008

Trying to put the two opposing feelings into the title of the post, on the one hand Google has compiled a database of all U.S books published between 1923 and 1963 that have lost copyright protection due to not being renewed, which is wonderful, and on the other hand they have made it available as a 371 MB XML file (inside of a 56 MB zipped file containing a Readme and a PDF document showing communications with Carl Malamud of public.resource.org and the U.S Copyright office) which is not so wonderful, because hey 371 MB XML file!!

The zip file holding the XML is here, everybody, start your Streaming.

An example of the XML format is shown below


<Record>
<CopyrightYear>1923</CopyrightYear>
<RenewalYear>1952</RenewalYear>
<File>11806-8.txt</File>
<Recno>000000</Recno>
<Lines>8648-8654</Lines>
<MD5Sum>3108ea337254c046f204d01a4bbdcb4d</MD5Sum>
<Title>In silenzio</Title>
<Copyright>
<Date>1923-01-01</Date>
<Id>AF12792</Id>
</Copyright>
<Renewal>
<Date>1952-08-29</Date>
<Id>R99258</Id>
</Renewal>
<Contrib>
<Name>LUIGI PIRANDELLO</Name>
<Role></Role>
</Contrib>
<Holder>
<Name>Stefano Pirandello</Name>
<Type>C</Type>
</Holder>
<Holder>
<Name>Fausto Pirandello</Name>
<Type>C</Type>
</Holder>
<Holder>
<Name>Rosalia Lietta Aguirre Pirandello</Name>
<Type>C</Type>
</Holder>
<Published>His Novelle per un anno, v. 6</Published>
<Source>gutenberg</Source>
<Snippet>
PIRANDELLO, LUIGI.

In silenzio. (His Novelle per un
anno, v. 6) Â© 1Jan23, AF12792.
R99258, 29Aug52, Stefano Pirandello,
Fausto Pirandello & Rosalia (Lietta)
Aguirre Pirandello (C)
</Snippet>
</Record>


You might also be interested in:


Popular Topics

Archives

Or, visit our complete archives.

Recommended for You

Got a Question?
http://news.oreilly.com/2008/06/371-mb-xml-file-from-google-fu.html
news.oreilly.com
crawl=CC-MAIN-2018-05
56139
application/http; msgtype=response
sha1:6R3U7YRVP7HYD6CZJT5Q75GIHAV7MZ4M
<urn:uuid:260fbb2e-ad14-445a-9765-66735e3e1a5a>
2018-01-20T03:20:01Z
67.222.96.148
application/xhtml+xml
sha1:JSLO4GAX3JS2EJQ55TID26GGPB2ORGXD
<urn:uuid:1186dfd8-a2fd-4ce3-943f-79aaf6a4588b>
http://news.oreilly.com/2008/06/371-mb-xml-file-from-google-fu.html
response
<urn:uuid:c1696ddb-b50a-4cc0-b3ff-2209f663a524>
robots: classic
hostname: ip-10-233-143-46.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2018-05
operator: Common Crawl Admin
description: Wide crawl of the web for January 2018
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf