List of websites excluded from the Wayback Machine

From Archiveteam
Jump to: navigation, search

This page collects sites that are manually excluded from the Wayback Machine. When a site is manually excluded, attempting to access it returns the error "This URL has been excluded from the Wayback Machine". This page does not track websites that disallow IA crawlers in their robots.txt file or block them. This list is not provided by the Internet Archive.

This list currently contains 1604 URLs.
https://www.archiveteam.org/index.php?title=List_of_websites_excluded_from_the_Wayback_Machine
www.archiveteam.org
crawl=CC-MAIN-2020-29
194345
application/http; msgtype=response
sha1:LK56MQ243B3QX4JHFUG4MYFSHD7DFRFA
<urn:uuid:af0a9901-b6ef-4853-9ff7-1fc4aa918728>
2020-07-15T12:41:07Z
204.109.60.27
text/html
sha1:IY6KW4X22D7RM64XCULJYXS74VK3UKYG
<urn:uuid:d16f1ebe-10c6-43ba-879a-e83e5cea006f>
https://www.archiveteam.org/index.php?title=List_of_websites_excluded_from_the_Wayback_Machine
response
<urn:uuid:8439ba50-7a15-4320-a0fa-54f41a179c6c>
isPartOf: CC-MAIN-2020-29
publisher: Common Crawl
description: Wide crawl of the web for July 2020
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-244.ec2.internal
software: Apache Nutch 1.17 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: http://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/