Starwars.yahoo.com

From Archiveteam
Revision as of 22:11, 20 December 2010 by Emijrp (talk | contribs)
Jump to navigation Jump to search
STARWARS.YAHOO.COM
Starwars.yahoo.com logo
Starwarszombies1.jpg
URL http://starwars.yahoo.com
Project status Offline
Archiving status Saved!
Project source Unknown
Project tracker Unknown
IRC channel #archiveteam (on EFnet)
Project lead Unknown

STARWARS.YAHOO.COM was a Star Wars themed user site, containing forums, user pages, and general movie information. Announced as a beta in October of 2007, it successfully suckered many users into posting, uploading, and sharing information on it before it was summarily shut off by Yahoo with a 30 day notice in December of 2009. On Dec 15, 2009, Yahoo shut the site down completely and redirected all queries to starwars.com.

Archive Team sprung into action and saved 99.9% of /users/, /forums/, /links/, and a smaller percentage of the other content on the site. (Most of the general star wars information was duplicated on starwars.com anyway - we focused our efforts on user-generated content.)

Problems encountered:

  • Yahoo issues an error 999 after about 30 minutes of fetching from a certain IP. We used two approaches to get around this.
    • TOR (slow as molasses, but worked) - collected using httrack
    • multiple IPs (fast, but needs large IP resources) - collected using wget

The tarballs in the archive reflect both archiving methods:

-rw-r--r--  1 root   root   228855239 Dec 15 13:35 starwars.yahoo.com-goekesmi-raw.tar.bz2
-rw-r--r--  1 root   root    36529217 Dec 20 15:53 starwars.yahoo.com-tor.tar.bz2

This archive of the collected remnants is located here.

Ywosw-200x200.jpg Just Kidding! Starwars.yahoo.com, 2007-2009
https://wiki.archiveteam.org/index.php?title=Starwars.yahoo.com&direction=prev&oldid=18721
wiki.archiveteam.org
crawl=CC-MAIN-2021-31
20439
application/http; msgtype=response
sha1:4SXIGFPDB5NASHVGOIB7ZHIELMPIKVNX
<urn:uuid:bb19f6c2-f57e-4041-b8a9-b135b8e692e1>
2021-08-02T16:34:51Z
213.184.85.58
text/html
sha1:UOJGR34JKKOQ744QU6IFZXCAMIPJM3FQ
<urn:uuid:2f3028b2-05e1-4838-9398-3fbe167bd2e6>
https://wiki.archiveteam.org/index.php?title=Starwars.yahoo.com&direction=prev&oldid=18721
disconnect
response
<urn:uuid:7f09ab83-d4f2-4f39-9753-83456b6b6277>
isPartOf: CC-MAIN-2021-31
publisher: Common Crawl
description: Wide crawl of the web for July/August 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-178.ec2.internal
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/