Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: kirstenanne Date: Jul 24, 2008 8:40am
Forum: faqs Subject: Re: PLEASE DELETE

I put the robots.txt file up. How long until my archives are deleted? I'm not worried about future archiving, as I'm canceling my domain.

Reply to this post
Reply [edit]

Poster: kustota Date: Jul 24, 2008 9:53am
Forum: faqs Subject: Re: PLEASE DELETE

after you put up robots.txt, you should submit your site for reindexing (see my previous post), otherwise it wouldn't disappear from web archives. after you submit, archived pages must disappear within a day.

Reply to this post
Reply [edit]

Poster: kirstenanne Date: Jul 24, 2008 10:04am
Forum: faqs Subject: Re: PLEASE DELETE

By submitting my site for reindexing, are you referring to the form that I can fill out to have Alexa crawl my site? If not, please clarify.

Reply to this post
Reply [edit]

Poster: kustota Date: Jul 24, 2008 10:06am
Forum: faqs Subject: Re: PLEASE DELETE

exactly that. when alexa sees robots.txt, all archived pages will disappear.

Reply to this post
Reply [edit]

Poster: kirstenanne Date: Jul 24, 2008 10:07am
Forum: faqs Subject: Re: PLEASE DELETE

Thank you for the useful information. How long does it typically take for the files to be deleted?

Also, once my hosting and domain cancel, will the archived files stay deleted?

Reply to this post
Reply [edit]

Poster: kustota Date: Jul 24, 2008 10:12am
Forum: faqs Subject: Re: PLEASE DELETE

i guess it would take about a day for archives to disappear.

after you take down your domain, no archived pages should appear.

This post was modified by kustota on 2008-07-24 17:12:54
http://archive.org/post/201270/please-delete
archive.org
crawl=CC-MAIN-2014-41
22387
application/http; msgtype=response
sha1:N2WU3A4SFUYSD3TDGUZOCOFIQ4HOIVHU
<urn:uuid:8c868a8f-d66b-4826-aee3-46be2932aa60>
2014-09-19T17:35:20Z
207.241.224.2
sha1:RN5CW5CGOEZ2EWGIPZNQTNFHWMZHR2PY
<urn:uuid:d4753f99-c4f0-4c49-a036-0309e70adcbb>
http://archive.org/post/201270/please-delete
length
response
<urn:uuid:8a5d15f3-25d9-4991-a43f-1dabcd45a793>
robots: classic
hostname: ip-10-196-40-205.us-west-1.compute.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2014-41
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for September 2014
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf