Project Gutenberg mirrors and Internet Archive server links

Comments about LibriVox? Suggestions to improve things? News?
Post Reply
ozdefir
Posts: 15
Joined: February 26th, 2016, 4:05 am
Contact:

Post by ozdefir » January 7th, 2017, 7:18 am

When there are connection issues with the main PG site (gutenberg.org), you can use the mirrors that are hosted by libraries and universities as listed here:
http://www.gutenberg.org/MIRRORS.ALL

They don't have catalogue search so you need to locate the book folders based on the PG number as described here:
ftp://aleph.gutenberg.org/README

The good thing about mirrors is that they don't nag the visitors with captchas etc, that's why in Readiance users are offered mirror links even when there's no connection issue with the main PG site.

I put together a form which converts the PG links to mirror links. It also takes care of some of the older links in LibriVox catalogue which are not valid anymore:
https://readiance.org/pg-fallback

Readiance also has a fallback mechanism for Internet Archive links. It keeps a local backup of metadata files in the Internet Archive's LibriVox collection. So whenever the main IA site is in maintenance, it uses the servers and directories found in the local metadata files which often helps. A user interface for it:
https://readiance.org/ia-fallback

TL;DR: If you have connection issues with the main Project Gutenberg site (gutenberg.org) or with Internet Archive you can try these:
https://readiance.org/pg-fallback
https://readiance.org/ia-fallback

Hope it helps.

Firat

Carolin
LibriVox Admin Team
Posts: 35667
Joined: May 26th, 2010, 8:54 am
Location: the Netherlands
Contact:

Post by Carolin » January 7th, 2017, 7:56 am

Thank you very much, that will be helpful :thumbs:
Carolin

Three things in human life are important: the first is to be kind; the second is to be kind; and the third is to be kind. Henry James

ozdefir
Posts: 15
Joined: February 26th, 2016, 4:05 am
Contact:

Post by ozdefir » January 7th, 2017, 10:17 am

David (dlolso21) asked me about the bad PG links, but it seems as a new user I'm not allowed to reply to PMs so I post it here.

The problem is with the PG links that have file names with letters. Like so:
http://www.gutenberg.org/dirs/etext02/1jrcl10.txt (from https://librivox.org/the-junior-classics-vol-1-by-william-patten)

gutenberg.org redirects some of these to the new number based URLs but it doesn't work for all books.

I will make a list with the LibriVox URLs and correct etext links and send to you.

knotyouraveragejo
LibriVox Admin Team
Posts: 14205
Joined: November 18th, 2006, 4:37 pm
Location: Pennsylvania

Post by knotyouraveragejo » January 7th, 2017, 11:06 am

A list with the current links would be great. These are probably going to be most of the early projects. I've fixed the link on the one you mentioned above.
Jo
My Librivox Solos
Books are the carriers of civilization. Without books, history is silent, literature dumb, science crippled, thought and speculation at a standstill. - Barbara Tuchman

dlolso21
LibriVox Admin Team
Posts: 3823
Joined: January 11th, 2011, 12:13 pm

Post by dlolso21 » January 7th, 2017, 11:57 am

Sorry, I forgot to check how many posts you had (there is a minimum number of posts required - a security feature to restrict robots and prevent spam).

Posting the links, a few at a time or the entire list, here would actually work better since more folks will see them and probably work on them.

David O

ozdefir
Posts: 15
Joined: February 26th, 2016, 4:05 am
Contact:

Post by ozdefir » January 8th, 2017, 8:39 am

Here is the list. As far as I remember I had checked them but a double check wouldn't hurt.

https://librivox.org/83 http://www.gutenberg.org/ebooks/3420 - Fixed
https://librivox.org/84 http://www.gutenberg.org/ebooks/3728 - Fixed
https://librivox.org/96 http://www.gutenberg.org/ebooks/3152 - Fixed
https://librivox.org/99 http://www.gutenberg.org/ebooks/9182 - Fixed
https://librivox.org/100 http://www.gutenberg.org/ebooks/2149 - Fixed
https://librivox.org/340 http://www.gutenberg.org/ebooks/518 - Fixed
https://librivox.org/610 http://www.gutenberg.org/ebooks/1429 - Fixed
https://librivox.org/857 http://www.gutenberg.org/ebooks/2870 - Fixed
https://librivox.org/875 http://www.gutenberg.org/ebooks/3207 - Fixed
https://librivox.org/951 http://www.gutenberg.org/ebooks/2515 - Fixed
https://librivox.org/1151 http://www.gutenberg.org/ebooks/8954 - Fixed
https://librivox.org/1177 http://www.gutenberg.org/ebooks/1281 - Fixed
https://librivox.org/1225 http://www.gutenberg.org/ebooks/1096 - Fixed
https://librivox.org/1235 http://www.gutenberg.org/ebooks/1493 - Fixed
https://librivox.org/1303 http://www.gutenberg.org/ebooks/78 - Fixed
https://librivox.org/1467 http://www.gutenberg.org/ebooks/284 - Fixed
https://librivox.org/1528 http://www.gutenberg.org/ebooks/9902 - Fixed
https://librivox.org/1544 http://www.gutenberg.org/ebooks/1717 - Fixed
https://librivox.org/1671 http://www.gutenberg.org/ebooks/6502 - Fixed
https://librivox.org/1708 http://www.gutenberg.org/ebooks/7166 - Fixed
https://librivox.org/1855 http://www.gutenberg.org/ebooks/121 - Fixed
https://librivox.org/2081 http://www.gutenberg.org/ebooks/2380 - Fixed
https://librivox.org/2224 http://www.gutenberg.org/ebooks/8319 - Fixed
https://librivox.org/2229 http://www.gutenberg.org/ebooks/419 - Fixed
https://librivox.org/2672 http://www.gutenberg.org/ebooks/6685 - Fixed
https://librivox.org/3933 http://www.gutenberg.org/ebooks/5116 - Fixed
https://librivox.org/3945 http://www.gutenberg.org/ebooks/3140 - Fixed
https://librivox.org/3960 http://www.gutenberg.org/ebooks/2397 - Fixed
https://librivox.org/5455 http://www.gutenberg.org/ebooks/8323 (should be http://www.gutenberg.org/ebooks/8021 - 8323 is a different Bible version) - Fixed

There are a couple of other situations which I will let you know.

dlolso21
LibriVox Admin Team
Posts: 3823
Joined: January 11th, 2011, 12:13 pm

Post by dlolso21 » January 8th, 2017, 10:04 am

Thank you for the list. I will get started on updating those.

David O

dlolso21
LibriVox Admin Team
Posts: 3823
Joined: January 11th, 2011, 12:13 pm

Post by dlolso21 » January 8th, 2017, 10:31 am

That list should be up to date now. I changed the link for the last one as your link pointed to The Bible, Douay-Rheims and not The Bible, King James version.

MaryAnnSpiegel
LibriVox Admin Team
Posts: 13779
Joined: February 23rd, 2009, 4:37 pm
Location: Chicago, IL
Contact:

Post by MaryAnnSpiegel » January 8th, 2017, 12:03 pm

ozdefir wrote:TL;DR: If you have connection issues with the main Project Gutenberg site (gutenberg.org) or with Internet Archive you can try these:
https://readiance.org/pg-fallback
https://readiance.org/ia-fallback

Hope it helps.

Firat
Thank you. I do have trouble accessing PG from home and as a result, also had trouble accessing PG to research mirror sites. I appreciate a loop that allows me to find the text without the need to access PG at all. I've used it twice already.
MaryAnn

ozdefir
Posts: 15
Joined: February 26th, 2016, 4:05 am
Contact:

Post by ozdefir » January 8th, 2017, 5:58 pm

Hi MaryAnn, I'm glad it's useful.
The advantage of gutenberg.org is that it's single click. If it's not a temporary problem you can email the PG support at "help2016 (at) pglaf.org". Even if they can't find an immediate solution, they will be able to identify the problem easier once there are multiple reports; because chances are other people in your neighborhood/using the same ISP are also affected. When I had a similar problem they were very responsive, so I think they take it seriously.

Post Reply
https://forum.librivox.org/viewtopic.php?f=24&t=63548&p=1311668
forum.librivox.org
crawl=CC-MAIN-2018-13
61778
application/http; msgtype=response
sha1:PJWROOMCUH43JDXNQ3GQDIOY27ZNVP67
<urn:uuid:d582ff9c-035c-4bd7-ba2c-1a23ed93b46d>
2018-03-21T07:21:24Z
208.70.31.70
text/html
sha1:3KC5RSPJOJXRURCCAPS426BL76KEU77Q
<urn:uuid:6008f834-7f38-474f-bcd5-874911fecb9a>
https://forum.librivox.org/viewtopic.php?f=24&t=63548&p=1311668
length
response
<urn:uuid:4194b6e6-c725-446a-b8c5-787fa31bfa3d>
robots: classic
hostname: ip-10-7-230-53.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2018-13
operator: Common Crawl Admin
description: Wide crawl of the web for March 2018
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf