This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.Find sources: "Archive site" – news · newspapers · books · scholar · JSTOR (January 2016) (Learn how and when to remove this template message)

In web archiving, an archive site is a website that stores information on webpages from the past for anyone to view.

Common techniques

Two common techniques for archiving websites are using a web crawler or soliciting user submissions:

  1. Using a web crawler: By using a web crawler (e.g., the Internet Archive) the service will not depend on an active community for its content, and thereby can build a larger database faster. However, web crawlers are only able to index and archive information the public has chosen to post to the Internet, or that is available to be crawled, as website developers and system administrators have the ability to block web crawlers from accessing [certain] web pages (using a robots.txt).
  2. User submissions: While it can be difficult to start user submission services due to potentially low rates of user submissions, this system can yield some of the best results. By crawling web pages one is only able to obtain the information the public has chosen to post online; however, potential content providers may not bother to post certain information, assuming no one would be interested in it, because they lack a proper venue in which to post it, or because of copyright concerns.[1] However, users who see someone wants their information may be more apt to submit it.

Examples

Google Groups

On 12 February 2001, Google acquired the usenet discussion group archives from Deja.com and turned it into their Google Groups service.[2] They allow users to search old discussions with Google's search technology, while still allowing users to post to the mailing lists.

Internet Archive

The Internet Archive is building a compendium of websites and digital media. Starting in 1996, the Archive has been employing a web crawler to build up their database. It is one of the best known archive sites.

NBCUniversal Archives

NBCUniversal Archives offer access to exclusive content from NBCUniversal and its subsidiaries. Their NBCUniversal Archives website provides easy viewing of past and recent news clips, and it is a prime example of a news archive.[3]

Nextpoint

Nextpoint offers an automated cloud-based, SaaS for marketing, compliance, and litigation related needs including electronic discovery.

PANDORA Archive

PANDORA (Pandora Archive), founded in 1996 by the National Library of Australia, stands for Preserving and Accessing Networked Documentary Resources of Australia, which encapsulates their mission. They provide a long-term catalog of select online publications and web sites authored by Australians or that are of an Australian topic. They employ their PANDAS (PANDORA Digital Archiving System) when building their catalog.

textfiles.com

textfiles.com is a large library of old text files maintained by Jason Scott Sadofsky. Its mission is to archive the old documents that had floated around the bulletin board systems (BBS) of his youth and to document other people's experiences on the bulletin board systems.

See also

References

  1. ^ Jinfang Niu, University of South Florida (March–April 2012). "An Overview of Web Archiving". D-Lib Magazine. 18 (3/4). doi:10.1045/march2012-niu1.
  2. ^ "Google Acquires Usenet Discussion Service and Significant Assets from Deja.com". 12 February 2001.
  3. ^ NBCUniversal Archives
https://db0nus869y26v.cloudfront.net/en/Archive_site
db0nus869y26v.cloudfront.net
crawl=CC-MAIN-2021-21
18426
application/http; msgtype=response
sha1:5F4IJGYLZ7Z3SXERLWPYFGCL6WPCBY2M
<urn:uuid:959a845f-1b5c-49a0-bf88-17abd1478850>
2021-05-14T13:13:01Z
52.84.122.45
text/html
sha1:2UPGCIUPOCK5EB6I4CS5SOJC7ASZOMWG
<urn:uuid:ac7dfa2b-75f8-4447-a041-627deaf9a750>
https://db0nus869y26v.cloudfront.net/en/Archive_site
response
<urn:uuid:55439e49-f190-42f2-b300-710ab77185ae>
isPartOf: CC-MAIN-2021-21
publisher: Common Crawl
description: Wide crawl of the web for May 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-101.ec2.internal
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/