ArsTechnica

Law & Disorder / Civilization & Discontents

Internet Archive starts backing up digital books on paper

The Internet Archive is turning to a tried-and-true medium for backing up …

Modified shipping containers used by the Internet Archive

If you want real long-term backups of digitized ebooks, then look no further than dead tree. At least, that's the consensus of the Internet Archive project, which has announced an incredibly ambitious plan to store one physical copy of every published book in the world.

"Internet Archive is building a physical archive for the long term preservation of one copy of every book, record, and movie we are able to attract or acquire... The goal is to preserve one copy of every published work," writes IA's Brewster Kahle in a lengthy blog post about the plan.

Kahle cites a number of reasons for wanting to preserve physical copies of works that are being digitized; for instance, a dispute could arise about the fidelity of the digital version, and only access to a copy of the original would resolve it. Kahle also told Kevin Kelly that we'll eventually want to rescan these books at an even higher DPI, so the digital copies will be waiting when we do.

Another reason for keeping a physical copy, and one that Kahle doesn't mention, is that the problem of long-term digital storage still isn't completely solved. The cloud as a large-scale storage medium has only recently emerged, and it's definitely not perfect as a long-term archival medium. Digital archivists have long pointed out that given a sufficient length of time, data loss is a problem even for highly redundant, highly available, distributed storage systems. Apart from the well-known phenomenon of bit rot, bits can get flipped for any number of reasons as they traverse a network and multiple software stacks, and ECC and other failsafes don't catch 100 percent of these errors. So on a long enough time horizon with a large-enough, complex-enough system, these undetected, flipped bits will begin to accumulate.

Realistically, Kahle and Co. expect to preserve 10 million books, out of an estimated 100 million published. These will be packed into climate-controlled storage containers, and stored in a facility in Richmond, CA that opens this month.

Kahle describes the details of the physical preservation as follows:

  • Books are cataloged, and have acid free paper inserts with information about the book and its location,
  • Boxes store approximately 40 books with labeling on the outside,
  • Pallets hold 24 boxes each,
  • Modified 40′ shipping containers are used as secure and individually controllable environments of 50 or 60 degrees Fahrenheit and 30% relative humidity,
  • Buildings contain shipping containers and environmental systems,
  • Non-profit organizations own and protect the property and its contents.

In order to reach its goal of collecting one of every book ever made, the Internet Archive is soliciting contributions of books from everyone from libraries to individual collectors.

Expand full story

71 Reader Comments

  1. This is a fantastic initiative. I will contribute.
    5523 posts | registered
  2. It's always bugged me that these types of preservation efforts generally involve plain text. Would it be so hard to use some sort of easily readable format as well like barcode or QR code to complement it? A single QR code can store a little over 4k alphanumeric characters while a single sheet of 8.5" x 11" upper bounds near 3.8k characters. Or even just use the pure binary data storage 2.9kB if you're storing binary data. It's small amount of extra work without having to redigitize using computationally expensive OCR.
    851 posts | registered
  3. While I'm all for redundant copies, it seems quite silly to think that digital storage is somehow unsafe. Yes, hard drives can and do go bad, but that's why we have things like RAID and backups stored in multiple locations. I suppose that hard drive backups could die from some kind of electromagnetic-pulse type of weapon, but it is far likelier that these "paper backups" will be destroyed in a fire.

    I just don't see these paper backups being that useful other than in a Mad Max/Book of Eli scenario in which all computing technology has been destroyed.
    2052 posts | registered
  4. I have boxes of media that I no longer have readers for: floppies, zip disks, vhs tapes, cassette tapes, vinyl records. Long after the dvds, blu-rays, cds, hard drives, ipads, and nooks in my house have become unreadable and been recycled the books will linger on. Paper is not particularly efficient, but very robust.
    1863 posts | registered
  5. Stone tablets are better in the long term of course, but those aren't as feasible as a storage medium for a large number of works. Paper is a proven storage medium.

    Yes, fire is its enemy. But I'm talking about the ability to put a book into a controlled environment, let it sit for 50 years and, when you take it out, it's still instantly readable. Electronic media has long-term issues in that you need a way to access it - a cd-rom drive, a mainframe terrminal, a reel-to-reel tape device, ZIP drives etc. The media itself might degrade over time, but also the device needed to access the media.
    78 posts | registered
  6. fletc3her wrote:
    I have boxes of media that I no longer have readers for: floppies, zip disks, vhs tapes, cassette tapes, vinyl records. Long after the dvds, blu-rays, cds, hard drives, ipads, and nooks in my house have become unreadable and been recycled the books will linger on. Paper is not particularly efficient, but very robust.


    But what do you think is more expensive? Maintaining a huge warehouse of climate controlled storage, or maintaining a few servers which periodically verify the integrity of all their content, both individually and with each other, and upgrading them periodically to take advantage of new technology.

    Even if you want to be safe by leaving the digital repository disconnected from the internet, all you need to do is have each location power on and verify it's contents occasionally, and periodically swap redundant copies of data between locations to allow each location to verify it's data against what should be another identical copy.
    98 posts | registered
  7. Quote:
    But what do you think is more expensive? Maintaining a huge warehouse of climate controlled storage, or maintaining a few servers which periodically verify the integrity of all their content, both individually and with each other, and upgrading them periodically to take advantage of new technology.

    The latter.
    85478 posts | registered
  8. chaos215bar2 wrote:
    fletc3her wrote:
    I have boxes of media that I no longer have readers for: floppies, zip disks, vhs tapes, cassette tapes, vinyl records. Long after the dvds, blu-rays, cds, hard drives, ipads, and nooks in my house have become unreadable and been recycled the books will linger on. Paper is not particularly efficient, but very robust.


    But what do you think is more expensive? Maintaining a huge warehouse of climate controlled storage, or maintaining a few servers which periodically verify the integrity of all their content, both individually and with each other, and upgrading them periodically to take advantage of new technology.

    Even if you want to be safe by leaving the digital repository disconnected from the internet, all you need to do is have each location power on and verify it's contents occasionally, and periodically swap redundant copies of data between locations to allow each location to verify it's data against what should be another identical copy.

    For the amount of data you're talking about it, dead tree is cheaper and less volatile. No sane archival service uses HDDs anyways. Tapes or burned optical discs. Optical discs aren't great for long term storage though.

    Edit: CURSE YOU PIZZA!
    851 posts | registered
  9. DrPizza wrote:
    The latter.


    Justification? A digital repository could only require power occasionally, and if done right with sufficiently many copies, could just be scrapped if they crashed in a serious manner. If anything happens to one of these warehouses, any attempt to recover what remains of the books could cost quite a lot.

    thepasswordispassword wrote:
    For the amount of data you're talking about it, dead tree is cheaper and less volatile. No sane archival service uses HDDs anyways. Tapes or burned optical discs. Optical discs aren't great for long term storage though.


    How much data is 10,000,000 books, anyway? That doesn't sound like so much to me. I'm not saying a digital solution necessarily makes sense for film, since even a single film frame can contain quite a lot of data, but for text, I just don't see how this makes sense.
    98 posts | registered
  10. "Kahle also told Kevin Kelly that we'll eventually want to rescan these books at an even higher DPI, so the digital copies will be waiting when we do. "

    Should this "digital" be "physical"?
    1 post | registered
  11. Hmm - what everyone else is saying, and note that Richmond is right on the Hayward - Rogers Creek fault, which occasionally ruptures with an 8.0. The archive is also dependent upon climate control, which takes reasonable access to the power grid, and ongoing maintenance of the HVAC, physical security, etc. In other words, you need to fund an endowment to keep the darned thing going.

    I'd rather put money for physical book storage into preserving the collections of the UC library system, only a few miles away from Richmond - and also subject to earthquakes, but with a much better retrieval system, and trained archivists.
    9 posts | registered
  12. I think a suitable long term storage medium would be something akin to what Frank Herbert describes in GEoD -- Ridulian Crystal ~ Crystalline substance used to manufacture printing materials at the thickness of one molecule.

    Either choose or manufacture a suitably durable crystal material that can be micro-etched with all this data, plain text or otherwise. If plain text then sufficiently powerful optics are all that are necessary to retrieve the data. Such optics can be stored with the sheets.
    4957 posts | registered
  13. >How much data is 10,000,000 books, anyway? That doesn't sound like so much to me

    Good point. Assume that we're talking about just text rather than text and pictures. Suppose that, compressed, each book runs 1 meg or so (lots of Project Gutenberg are less).

    So, 10,000,000 books x 1,000,000 bytes/book= 10 TB?

    My local Costco is selling 1 TB WD Passport drives for $89. So, assuming that we need two sets for backup, that seems like we only need about $2k worth of hardware (including tax), and a grad student to copy the files.

    Even if we pay for the master's degree of the grad student in question ($50k?), we'd still come out ahead, I suspect.
    3935 posts | registered
  14. Quote:
    Justification? A digital repository could only require power occasionally,

    A digital repository would require the same HVAC as a paper one, even if powered down. The only advantage on that front is that it's less humidity sensitive.

    But powered down, there's no way to verify data, and hence no way to detect--and repair--errors. And that's killer. Hard disks are the most cost-effective, easy-to-use mass storage available, but they're designed to be spun up, not powered down and left on shelves for months at a time.
    85478 posts | registered
  15. DrPizza wrote:
    Quote:
    Justification? A digital repository could only require power occasionally,

    A digital repository would require the same HVAC as a paper one, even if powered down. The only advantage on that front is that it's less humidity sensitive.

    But powered down, there's no way to verify data, and hence no way to detect--and repair--errors. And that's killer. Hard disks are the most cost-effective, easy-to-use mass storage available, but they're designed to be spun up, not powered down and left on shelves for months at a time.


    Greater, actually, since paper doesn't generate heat.

    Given that the Archive can control the process from production to storage, it'll be cheaper than expected. They won't need to worry about acidic paper, outgassing (or exploding!) materials, crumbling material, etc. The lack of access makes it even easier - they can run off the printed matter, process, store securely, and not have to worry about it again for 50 years. I envy the archivist in charge of this; they have the easiest job in the field. ;)

    Digital materials scale much much better than paper, but for sheer longevity, physical material is what you want to use.
    81 posts | registered
  16. We need a large digital repository in several different "safe spots" in the world. Preferably places in a mountain or something. ZFS and back-ups.
    1803 posts | registered
  17. Bengie25 wrote:
    We need a large digital repository in several different "safe spots" in the world. Preferably places in a mountain or something. ZFS and back-ups.


    Already being done.

    http://lockss.stanford.edu/lockss/Home
    81 posts | registered
  18. Great initiative! People don't seem to realize how fragile electronic storage media are AND how fickle our formats have been (and will continue to be). Paper has a proven track record that's, what, at least two orders of magnitude greater than any proven electronic media?

    Is it perfect? No (fire, flood, difficulty of duplicating). But the smart money would bet on it.
    3028 posts | registered
  19. I think this is a great idea. How can you say anything is safe online with all the crap that happened this year?. Using digital books backed up by a physical copy is ideal in this situation.
    232 posts | registered
  20. Quote:
    Modified 40′ shipping containers are used as secure and individually controllable environments of 50 or 60 degrees Fahrenheit and 30% relative humidity


    What a waste of resources. The vast majority of media is not worth much; and the possibility that someone, sometime in the future may argue about the authenticity of scanned data is not worth the extra expenditure. The people behind this seem to have a major, major hoarding fetish.

    Quote:
    So on a long enough time horizon with a large-enough, complex-enough system, these undetected, flipped bits will begin to accumulate.


    For various reasons, the printed word also develops 'errors'. Moisture and oxygen can and does affect ink and paper and over long-enough time scales, will materially affect them. As will insects and microbes. Has anyone does a long-term (atleast decades long) study on the robustness of paper vs various digital media? Without that information, justifying the replication of a digital store with paper with various claims about digital errors is just hand-waving.
    7398 posts | registered
  21. Why not just invest in a storage format that does not degrade? Data doesn't need to be stored on computers, but it works fine as long as it can be read by computers. Pick a very robust substance and find a way to keep compact data on it.

    I remember a while ago when someone came up with a crystalline cube that could map data in a 3-dimensional space inside it, which could then be scanned and read in a digital medium. I think keeping an archive of those tiny things is a much more reliable format than paper at least.
    207 posts | registered
  22. grimlog wrote:
    Quote:
    Modified 40′ shipping containers are used as secure and individually controllable environments of 50 or 60 degrees Fahrenheit and 30% relative humidity


    What a waste of resources. The vast majority of media is not worth much; and the possibility that someone, sometime in the future may argue about the authenticity of scanned data is not worth the extra expenditure. The people behind this seem to have a major, major hoarding fetish.


    Except sometimes it is. Watches in the desert and all that.

    grimlog wrote:
    Quote:
    So on a long enough time horizon with a large-enough, complex-enough system, these undetected, flipped bits will begin to accumulate.


    For various reasons, the printed word also develops 'errors'. Moisture and oxygen can and does affect ink and paper and over long-enough time scales, will materially affect them. As will insects and microbes. Has anyone does a long-term (atleast decades long) study on the robustness of paper vs various digital media? Without that information, justifying the replication of a digital store with paper with various claims about digital errors is just hand-waving.


    Moisture won't under a certain threshold, oxygen only does on really bad paper, bugs are a solved problem, and no archive I can think of has had catastrophic microbacterial rot. Physical problems (at least, the ones the Archive would have to deal with - ref. my comment about their paper) have more or less been solved; what do you think us archivists have been doing the past century? ;)

    The problems we *can't* solve are the digital ones. Tapes on dead formats, EM surges blanking disks, and mechanical disk failures are really hard to deal with. The Archive knows they can make a 500+ year facility by de-digitizing material, and they certainly have the resources to make it, so there's no reason not to have a golden source, so to speak.
    81 posts | registered
  23. This reminds me of the Star Wars EU books, where the Chiss would only allow Luke access to their archives through the huge paper repository version. Even had to use paper indexes rather than computers to search for what he wanted.
    2626 posts | registered
  24. The world needs more archiving. It's not unthinkable that we could have the equivalent loss to that of the Library of Alexandria at some point, and the more secure caches we have the better.
    7816 posts | registered
  25. Archiving physical specimens of printed books, paintings, and other examples of our collective heritage is quite likely one of the best ways to preserve knowledge for future generations... digital medium be dammed. The most vital aspect of this project isn't in comparing how much energy usage will be required, or the size of the physical footprint it may require as compared to a bank of easily accessed servers - it's in the preservation of materials that can be read by human beings without the aid of a machine to translate it for them.

    If you want examples of even how the best of intentions can be made quickly obsolete by rapidly changing digital formats - and the devices that read and decode all this information, look no further than Britian's own failed Domesday Project ( http://en.wikipedia.org/wiki/BBC_Domesday_Project ), or those thousands of unreadable tapes in the NASA archives that represented decades of lost weather and climate recordings made during early satellite programs.
    Granted, such a project will entail storage measures unique to itself - But unlike the Domesday discs that were encoded, and as anyone who's suffered a catastrophic hard drive failure will appreciate - that 11th century book in Britians national archives is still available for viewing by the public, carefully preserved and easily readable at any time.

    The beauty is that the physical original of the book remains steadfast, while at the same time any number of digital copies or printed reproductions are in circulation world wide.
    82 posts | registered
  26. Forgive me to have the following comments. I just couldn't help myself from posting it here.

    It doesn't make any f*ckin sense to me 'til I got to the end of the article that says "Non-profit organizations" now it makes a better sense.

    We all know that what non-profit means. It's money, money, money that keep coming in endlessly forever like a hole in a bucket, donations from smart-ass rich who wanted tax exemption. Long as the project is on going and there's a single legitimate reason for it and there seems a good reason for this one, is to preserve books from paper to digital and back to paper.

    What a joke.

    But when we look at it from another angle. There's a merit to it is, people get paid for looking after these containers. The landlord gets paid. Janitors get paid to water the lawns. It means - jobs.

    If you come up with an idea something like this. You've got yourself make - for life.

    Okay, back to the debate on paper or DVD media. What happen to this 50GB blu-ray? Are they dead? Would one thousand of blu-rays disks do the job? If not, what's the costs to burn two thousand 50GB blu-rays? Another two thousands for back-up? Or burn four thousands for two back-up? Six thousands, and so on? It beats the hell of having these 40 footer containers. No?

    Oh I got it, stupid of me. Where's the beef if you could stored a couple of box of blu-rays disks in a single draw? Too easy.

    To many, they'd say what a fu*king moronic project. But this is what a democratic society is all about. You've come up with an idea on how to make yourself big money. Do it.

    How about this "Donate your cars for the kids?"

    "Donate your money for the missing children?"

    There're nights I couldn't fall to sleep. I was thinking... hmm..

    :-)

    </rant>
    4122 posts | registered
  27. Just to reiterate: If you want some data to be around and readable 500 years from now, the only proven technology is ink-on-paper. I have Public Enemy CDs that don't work and now I'm wondering how to write a score for Terminator X's compositions.
    10 posts | registered
  28. Evolution wrote:
    Forgive me to have the following comments. I just couldn't help myself from posting it here.

    It doesn't make any f*ckin sense to me 'til I got to the end of the article that says "Non-profit organizations" now it makes a better sense.

    We all know that what non-profit means. It's money, money, money that keep coming in endlessly forever like a hole in a bucket, donations from smart-ass rich who wanted tax exemption.

    Need some Zap-A-Gap for that little nick on your shoulder?

    Quote:
    Okay, back to the debate on paper or DVD media. What happen to this 50GB blu-ray? Are they dead?

    They very well can be. There are tapes that can't be read because nobody can dig up a working cassette or reel-to-reel and hook it up to something that can translate the ancient proprietary file format. Also, disks themselves are not invulnerable, even if kept under archival conditions. There's a lot of laser discs that can't be used anymore even though they were kept in the sleeve for years. I've known people lose printed, commercial CDs (not burned CR-Rs) even after keeping them in the original cases in a closet for a few years. And nevermind keeping a working Blu-Ray drive around and compatible with whatever computers exist in the future, indefinitely. It really is cheaper and more robust to store the books themselves under these conditions, especially since we've gotten pretty damn good at preserving paper over the lifetime of the printed page.

    But no, I'm sure this is really all a scam and you've figured it out. Archiving is easy and cheap, and don't let anybody tell you otherwise.
    7816 posts | registered
  29. Air-conditioned boxes of paper, a paradise for mice...
    Great idea, but isn't it out of scope for The Internet Archive? It's purely library archiving, and totally redundant with the Library of Congress, at least for American books. Like hell the Irish would give out The Book of Kells to be stored in a rats-infected container... ^-^

    At first sight it even seems to defeat the purpose of comparing to the original. First, it would take a lot of manual work to extract a book from its container, pallet and box (how many pallets per container, by the way, or how many books per container), moving many pallets and boxes before reaching the proper book. Then you would need to rescan the book. Even at high resolution, OCR generates a lot of errors, much more than the mentioned electronic bitrot. The whole captcha system is basically a way to have many users validate such badly scanned words.

    It would make much more sense to me to store the digital scans on a digital media, like many libraries do already, so the comparison with the original can be made instantly, although we're talking like a million times more storage space. Not as economical as paper, but the scans would be available worldwide instantly and forever, with enough redundancy and frequent disk replacement to beat the bitrot.

    And let's not forget the original printed book is not always the reference. More books will be available in electronic format only, and printed books do contain typos too. In which case, the corrected digital copy becomes more valuable than the original book or scan, if you have a system to manage such corrections.

    We need to embrace the digital age, and while printed books can be stored in containers, that wouldn't work for audio books. Or music. Or movies. It's all about our culture, so there is no reason to treat one particular media differently than the others. Or to let studios, labels and editors behave like harpies, but that's another debate.
    607 posts | registered
  30. Evolution wrote:
    Forgive me to have the following comments. I just couldn't help myself from posting it here.

    It doesn't make any f*ckin sense to me 'til I got to the end of the article that says "Non-profit organizations" now it makes a better sense.

    We all know that what non-profit means. It's money, money, money that keep coming in endlessly forever like a hole in a bucket, donations from smart-ass rich who wanted tax exemption. Long as the project is on going and there's a single legitimate reason for it and there seems a good reason for this one, is to preserve books from paper to digital and back to paper.

    What a joke.

    But when we look at it from another angle. There's a merit to it is, people get paid for looking after these containers. The landlord gets paid. Janitors get paid to water the lawns. It means - jobs.

    If you come up with an idea something like this. You've got yourself make - for life.


    Much of the Internet Archive is (or, at least for a long while, was) funded by one man's personal fortune, of which the monetary returns is minimal. If you'd like to contact him and let him know that his life's work is a method by which to fleece the public (and not a public service by being the only institution effectively archiving the Internet) feel free.

    Evolution wrote:
    Okay, back to the debate on paper or DVD media. What happen to this 50GB blu-ray? Are they dead? Would one thousand of blu-rays disks do the job? If not, what's the costs to burn two thousand 50GB blu-rays? Another two thousands for back-up? Or burn four thousands for two back-up? Six thousands, and so on? It beats the hell of having these 40 footer containers. No?

    Oh I got it, stupid of me. Where's the beef if you could stored a couple of box of blu-rays disks in a single draw? Too easy.


    I'll make you a gentleman's wager. I'll go out and buy 50 books; you can take a digital writable media of your choice and burn them to as many copies as you'd like. We'll meet again in 30 years, and whosever has survived better, wins.

    I suggest you bring a technician (likely called out of retirement) to get your decades-obsolete hardware running again. Assuming, of course, you can even find a computer with the proper supporting hardware, drivers, and file structure to read your stack of Blu-Rays.

    Evolution wrote:
    To many, they'd say what a fu*king moronic project. But this is what a democratic society is all about. You've come up with an idea on how to make yourself big money. Do it.


    If there's big money in archiving, please let me (and a couple thousand archivists - they'd like a raise) know.
    81 posts | registered
  31. Remember the George Pal movie of Wells' "The Time Machine" ? In it there was a scene in the far future where the paper books had turned to dust, but an intriguing little device consisting of a polished surface spoke to you if you spun a little metal ring on it. (It's worth looking up the movie just to see that bit.)

    My thought -- spinning a little ring on the flat surface was intuitive, something anyone might do.

    Would it be possible for someone to fund a graduate program to explore extremely simple user interfaces for digital media? Simpler than voice operation (dialects may shift), simpler than pressing a button (even buttons may become obsolete). Maybe spinning a ring on a little table...

    Or maybe something else. But it's something for fine minds somewhere to contemplate; this is our future, after all, and how we interact with our world is guaranteed to change.
    96 posts | registered
  32. there is something ironic about this... i think... I dont know, i just read the headline...
    1503 posts | registered
  33. Quote:
    I'll make you a gentleman's wager. I'll go out and buy 50 books; you can take a digital writable media of your choice and burn them to as many copies as you'd like. We'll meet again in 30 years, and whosever has survived better, wins.


    Been there done that. When were those first digital books out? Maps on CDs? Years ago one of my evening part time job after my 8 hours day job was scanning magazines using proprietary software especially tailor made for that company I worked, years (I would say) before you'd have seen some workable OCR scanning softwares that bundled with the OCR scanners. Do you still remember that terrible experiences you have a tiny dot on the paper would messed up the character that made it read like from out of space?

    That scanning software I worked with it not just did scanning, it also allow the users to altered the background colour to make the scanned images more outstanding. I remembered there were near 100 of us. If it were text base scannings it would have been done lot easier for us to do the job than scanning graphic I'd tell you.
    4122 posts | registered
  34. Quote:
    Okay, back to the debate on paper or DVD media. What happen to this 50GB blu-ray? Are they dead? Would one thousand of blu-rays disks do the job? If not, what's the costs to burn two thousand 50GB blu-rays? Another two thousands for back-up? Or burn four thousands for two back-up? Six thousands, and so on? It beats the hell of having these 40 footer containers. No?

    No, because they won't be usable in a century.

    A significant proportion probably won't be usable in a decade.

    Acid-free paper lasts phenomenally well, if kept in reasonable conditions.
    85478 posts | registered
  35. I have digital media that are thirty years old. Most I can still read, some I can't. And I am a zealot when it comes to being able to recover old media and data files. Even so, it is a challenge. Its not just media, its data formats, OSes, and interfaces to mention a few. Everyone who believes, newer is better and why doesn’t everyone just jump on the new bandwagon is just contributing to this issue. Data transfer is expensive. Format revision is expensive and error prone. In just 10 years the issues can become overwhelming. It does not matter what format you use. I did a bunch of archiving nearly 20 year ago, those files I can still read. But I used the very newest format of the time writable CD-ROM and oldest format in the books, text files. Everything is cataloged in the media and zipped. All programs are included on the CDs with CRCs for all files. Decoding instructions are included. It was work, 17 GB, 77k files, in 1993. There were no hard drives nearly so large. That took planning. But if I had started a year earlier, I would have had to use MO drives. Almost no one today even knows of MO drives. The change from MO to CD-ROM occurred in early 1993 when a new writing system became available and price dropped by a factor a 3 and would run on an ordinary PC (well sort of ordinary).

    Paper on the other hand really only suffers from revision in human language. As such Chaucer is hard to read, Shakespeare no so much. But the paper still survives without modern protection. I own one page of a manuscript (music) from the time of Guttenberg (or within a couple of decades). I have one 16th century book, a Latin dictionary, and two manuscripts. I have dozens of 17th century books, including two dictionaries. I can no longer count the number of 18th and 19th century books. But until ~around 1860, they are all very readable with little damage, except foxing. Why 1860s, well that is when the new “format” for paper making became to be used significantly for books. Then began the significant use of wood fiber and bleaching of the paper; and for low cost printing using mechanical pulping (think newsprint). Even if paper is flooded by water, the content is retrievable. The biggest issue here is maybe the paper copier, computer driven paper printer and scanner may only be with us another few decades after having only been invented a little more than some half century ago. Maybe digital cameras will have to be the scanner, since I suspect digital photography will survive the scanner, after all we live in an analog 3-D world and I don’t think that will change for sometime. But will the TIFF format survive?

    But my point is, even with careful planning digital formats are difficult to maintain for even a few decades and for paper centuries can pass. On top of that if they are reprinting the books onto special paper with specially designed fonts, OCR is very accurate. Even in 1980 I was using OCR fonts in the editing of large documents, where the changes were written in red by hand on to OCR computer printed pages and the computer, not have the originals because the insufficient storage capacity, would OCR in the prior page and the editor would incorporate the relined changes by hand and reprint the page again in an OCR font. A human readable computer code is very desirable from an archiving point of view. Also consider MICR stripe on the bottom of a check as an early variant. Look at OCR-A and OCR-B.


    http://en.wikipedia.org/wiki/Foxing
    http://en.wikipedia.org/wiki/Acid-free_paper (Alkaline paper has a life expectancy of over 1,000 years for the best paper and 500 years for average grades.)
    http://en.wikipedia.org/wiki/Papermaking
    http://en.wikipedia.org/wiki/Paper
    http://en.wikipedia.org/wiki/Bleaching_of_wood_pulp
    http://en.wikipedia.org/wiki/Magnetic_ink_character_recognition
    http://en.wikipedia.org/wiki/OCR-A_font
    http://en.wikipedia.org/wiki/OCR-B

    For digital formats what about PDF, that changes every year or two and still makes it difficult to recover the “text” vs. the page. HTML, they cannot stop changing that. Think of all the features they are depreciating. That would likely render a 2011 html file unreadable by 2021. So maybe that leaves txt files, but what ASCII or UNICODE?

    Examples of media to show just how difficult it can become.
    Floppies:
    8 inch hard and soft sectored: These I cannot read, however, most of mine were archived into 3.5 inch 1.44M floppies (FAT).
    5 1/4: 160k/180k, 320k/360k, 1.2 M these I can still read. I retain several drives. Most have at least one copy of any file on HDD.
    3.5: 720k, 800k, 1.44M 2.88M: I can still read and actively still use 1.44M*. I can read 720k. I cannot read 800k (Mac). I can read 1.4M Mac formatted disks, under XP. Never used 2.88 M (primarily used in Japan). * Lab equipment worth 10s of thousand of dollars and use floppies and no one is replacing these just because they have floppies.
    Never used any of the other floppy formats.
    http://en.wikipedia.org/wiki/Floppy_disk
    QIC 40, 80 Travan TR-1: I have one working drive still. Rarely used. There are many more formats.
    http://en.wikipedia.org/wiki/Quarter-inch_cartridge

    MO drives: I have one data base on MO drives (WORM), but it is backed up on CD ROM (c.a. 1993). (yes that is in 1993 with a JVC 2001 write drive [Archiver]driver under DOS 6 on to THATS media, $35 ea, 54 min ~480 MB, SCSI, 1x drive).

    http://en.wikipedia.org/wiki/Magneto-optical_drive:



    ST-506 MFM one, drive, 10 MB all moved on.
    ESDI work servers
    SCSI: still have
    ATA: of course
    SATA: of course (many TBs)

    http://en.wikipedia.org/wiki/Hard_disk_drive

    SyQuest / Iomega's(e.g. SyJet, Jaz drive/Zip/Bernoulli Box)
    SyJet drive: I still have one working drive.
    http://en.wikipedia.org/wiki/SyQuest_Technology
    http://en.wikipedia.org/wiki/Iomega

    7 track: 100 dpi - 800 dpi (Well you can hand read 100 dpi tapes with a simple tool, but why? It like reading punch tape by hand; and here is another format) (good luck reading)
    9 track: 800, 1600, 6250 dpi (almost a much luck)
    http://en.wikipedia.org/wiki/IBM_7_track
    http://en.wikipedia.org/wiki/9_track_tape

    For a broader prospective:

    http://en.wikipedia.org/wiki/Data_storage_device
    65 posts | registered
  36. Evolution wrote:
    When were those first digital books out? Maps on CDs?


    I wouldn't know; I wrote it down in a WordStar document and can't open it anymore.
    81 posts | registered
  37. Quote:
    No, because they won't be usable in a century.

    A significant proportion probably won't be usable in a decade.


    I still have some .wri files that XP won't read them. "decade?" No, it's merely just a period of 5 years away. I didn't convert the files to XP format and who's false was that? Me. You would never left those discs untouched for the next 100 years. Upgrade the discs to the latest format would be wise and that's what the pro do. And when you handle some 100 of millions copies of books from all over the world, you're a pro.

    Quote:
    Acid-free paper lasts phenomenally well, if kept in reasonable conditions.


    Yeah. And then what? Scan them again on disc after 100 years from now? Might as well scan them now.

    Beside the land they're sitting on would not be there forever. Changed ownership? No longer a lease contract from the new owner? Where are you going to put these 40 footer containers?
    4122 posts | registered
  38. Quote:
    Yeah. And then what? Scan them again on disc after 100 years from now? Might as well scan them now.

    No. Reprint them in 500-1000 years.

    Might as well scan them now? And store the scans on what?
    85478 posts | registered
  39. Any information on how they select the works to preserve? They obviously don't store anything published, so they've got to have some kind of selection strategy.

    I'm also wondering why a warehouse in California. This seems kinda dumb to me. As mentioned by others the facility will be prone to natural disaster. They should look for a more stable location. I'm thinking something alike to what Bill Gates uses with his Corbis company: they store their images at the Iron Mountain facility, which is basically a cold cave.
    They use shipping containers, so maybe they already thought about moving one day.

    While I know that acid free paper is good for 500-1000 years, have they thought about alternative non digital media? I've heard that many (government) long term archive projects use microfilm. Under the right conditions it's supposed to last as long as paper, but uses less space.
    194 posts | registered

You must to comment.

You May Also Like

    Need to register for a new account?

    If you don't have an account yet it's free and easy.

    Register
http://arstechnica.com/tech-policy/2011/06/internet-archive-backs-up-digital-books-on-paper/?comments=1&post=21753176
arstechnica.com
crawl=CC-MAIN-2013-20
136874
application/http; msgtype=response
sha1:FNUWGZHXVX2E7YZCVFDPWCHTEVHHVTSG
<urn:uuid:a1bbd2f5-8a99-426c-9084-0069a9a7bf9d>
2013-06-19T17:15:41Z
50.31.151.33
sha1:UZC3SBHRCOPMZF2TVYMAZ4Z67DT47GSI
<urn:uuid:15bb5295-b1d7-41c1-a306-f156057e453f>
http://arstechnica.com/tech-policy/2011/06/internet-archive-backs-up-digital-books-on-paper/?comments=1&post=21753176
response
<urn:uuid:cbb2ac08-e4b9-4e92-a3dc-2dcc170a772f>
robots: classic
hostname: ip-10-60-113-184.ec2.internal
software: Nutch 1.6 (CC)/CC WarcExport 1.0
isPartOf: CC-MAIN-2013-20
operator: CommonCrawl Admin
description: Wide crawl of the web with URLs provided by Blekko for Spring 2013
publisher: CommonCrawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf