Successfully reported this slideshow.

Saa Session 502 Born Digital Archives in Collecting Repositories

2,209 views

Published on

Digital archivists from the Universities of Hull (UK), Stanford, and Yale currently are collaborating on an Andrew W. Mellon Foundation-funded project. Born-Digital Collections: An Inter-Institutional Model for Stewardship (AIMS) will produce a common framework for managing born digital archives. Each digital archivist presents a short case study to cover areas of workflow for electronic records: collection development, accessioning, arrangement and description, and discovery and access.

Published in: Technology, Education
  • Be the first to comment

Saa Session 502 Born Digital Archives in Collecting Repositories

  1. 1. Born-Digital Archives inCollecting Repositories:<br />Turning Challenges into Byte-Size Opportunities<br />Gretchen Gueguen, Mark A. Matienzo, Simon Wilson, and Peter Chan<br />Session 502, 27 August 2011<br />Society of American Archivists Annual Meeting<br />
  2. 2. AIMS Project<br />"Born-Digital Collections: An Inter-Institutional Model for Stewardship“<br />Two year project to create a framework for stewardship of born-digital archival records in collecting repositories<br />Funded by the Andrew W. Mellon Foundation<br />
  3. 3. Partners<br />
  4. 4. Grant Goals<br />Processing of Hybrid Collections<br />Software Development <br />Community Development<br />Unconference (May 2011, Charlottesville, VA)<br />UK Symposium (June 2011, London, England)<br />Workshop (August 2011, Chicago, IL)<br />White Paper and Project Report<br />
  5. 5. Framework Development<br />A framework for collecting and delivering the born-digital materials that are quickly beginning to constitute the collections of contemporary scholarly, literary, and political figures and organizations.<br />
  6. 6. AIMS Framework<br />Discovery and Access<br />Accessioning<br />
  7. 7. Collection Development<br />Gretchen Gueguen<br />University of Virginia<br />
  8. 8. What is Collection Development?<br />Actions and policies of institutions to bring in material for end users (both current and future); includes prioritizing, developing relationships with creators, assessments, negotiating agreements and preparing for accessioning.<br />Within the AIMS framework<br />Viable, practical method to capture/process born-digital material from hybrid collections requires sound work at the beginning (i.e. policies, practices, agreements with donors, etc.) to set up later work<br />
  9. 9. Elements of Collection Development<br />Prerequisites<br />Establish relationship with donor<br />Analyze Feasibility<br />Negotiate Agreements<br />Prepare for Accessioning<br />
  10. 10. Prerequisites…<br />Neil Beagrie, "Plenty of Room at the Bottom? Personal Digital Libraries and Collections," D-Lib Magazine (June 2005)<br />Blagofaire. http://xkcd.com/239/<br />
  11. 11. Donor Relationship…<br />
  12. 12. Enhanced Curation<br />
  13. 13. Analyzing Feasibility…<br />
  14. 14. Negotiate Agreements…<br />All rights reserved by Chevrolet UK<br />
  15. 15. Prepare for Accessioning...<br />Scope and extent determined?<br />Coordination with acquisition of analog material?<br />Method and time <br />determined?<br />Pre-acquisition appraisal performed?<br />Enhanced curationcarried out?<br />Test capture if needed?<br />Development of new methodologies undertaken as needed/possible?<br />
  16. 16. Accessioning<br />Mark A. Matienzo, Yale University<br />
  17. 17. What is Accessioning?<br />Archival institution takes physical and legal custody of a group of records from a donor and documents the transfer in a register or other representation of the institution’s holdings<br />Within AIMS Framework<br />Processes which establish physical, administrative and intellectual control over transferred records; assessment and documentation of future needs; documentation of actions taken; beginning of safe storage and maintenance<br />
  18. 18. Elements of Accessioning<br />Prerequisites<br />Transfer records and gain administrative control<br />Physical control and stabilization<br />Intellectual control and documentation to support further processes<br />Maintain accessioned records<br />
  19. 19. Case Study:Re-Accessioning at Yale<br />Collaborative capacity building across two repositories<br />Manuscripts and Archives<br />Beinecke Rare Book and Manuscript Library<br />Addressing previously received accessions of containing electronic records on media<br />Still in testing phase, but working towards implementing in production<br />
  20. 20. Types of Records and Media<br />Wide variety of records creators<br />Literary authors<br />University faculty<br />University offices<br />Architectural firms<br />Common types of media<br />Floppy disks: 5.25” and 3.5”<br />Optical media: CDROM, CD-R, DVD-R, etc.<br />Zip disks<br />USB flash drives<br />
  21. 21. Goals of Re-Accessioning<br />Identify, document, and register media<br />Mitigate risk of media deterioration and obsolescence<br />Extract basic metadata from filesystems on media and files contained on filesystems<br />
  22. 22. Re-Accessioning Workflow<br />
  23. 23. Disk Imaging<br />Using “forensic” (bit-level) imaging process<br />Ensure data on media is not manipulated using write-protection<br />Uses software to acquire images<br />Includes hash-based verification process<br />
  24. 24.
  25. 25. Media Log<br />Using SharePoint list<br />Contains unique identifier of media<br />Records physical/logical characteristics of media<br />Documents success, failure, or status of various processes and additional notes<br />
  26. 26. Media Log<br />
  27. 27. Media Log<br />
  28. 28. Metadata Extraction<br />Can be repurposed for descriptive, administrative, and technical metadata<br />Uses command-line tools (Sleuthkit, fiwalk)<br />Outputs XML document<br />
  29. 29. Packaging and Transfer<br />Using BagIt packages/Bagger application<br />Packages contain disk images, extracted metadata, imaging logs, and high-level accession information<br />Transfer to storage is verified by comparison against manifest<br />
  30. 30.
  31. 31. Arrangement & Description<br />Simon WilsonHull University Archives<br />
  32. 32. Purpose of Arrangement & Description <br />The general objectives for Arrangement & Description are:<br /> - to preserve context<br /> - to establish intellectual control of the material<br /> - to provide a means of discovery <br />SAA definition, emphasis on minimizing the amount of handling<br />Within the AIMS framework<br />Processes which establish intellectual control of the material including implementation of policies and agreements with donors etc. to enable subsequent discovery and access<br />
  33. 33. Elements of Arrangement and Description <br />Prerequisites <br />Plan for processing <br />     - gather supporting information; files captured from media <br /> (accessioning); convert files (for viewing); appraisal strategy; <br /> assess arrangement options; consider preservation issues <br />Processing <br />   - implement arrangement strategy; add descriptive metadata and wider context (eg Collection Level Description); copyright & other legal considerations  <br />4. Prepare for Discovery & Access- remove restricted access to b-d material during processing<br />
  34. 34. Case Study - Stephen Gallagher<br />Background:2005: 42 boxes paper archives <br />2010: born-digital material: 14,320 files (13.6GB) transferred to us via external hard drive and a box of Amstrad disks<br />Create integrated catalogue to accommodate paper, born-digital and future accruals<br />
  35. 35. Case Study - Stephen Gallagher<br />Approach: <br />- current work higher priority in filing system<br />- considered each work a distinct ‘project’ <br />- structure reflect his way of working & the <br /> archival principles of control that creator, <br /> archivist & user can all understand<br />Series level was most logical solution- all related files placed in the series - reasonable return for our effort <br />
  36. 36. Case Study - Stephen Gallagher<br />300 files created using FinalDraft screenwriter software<br /><ul><li> view file (as created) to identify </li></ul>appropriate format for long term<br />preservation<br />Other issues: <br /><ul><li> copyright/third-party content 
  37. 37. commercial implications: access </li></ul>via repository = publication? - re-purposing of work from one (unsuccessful) project to another <br />
  38. 38. Challenges faced<br />Each collection is unique, approach will vary: <br /><ul><li> integrate born-digital material with existing material/arrangement? 
  39. 39. one-off collection (eg project) or likely to be subsequent accruals?
  40. 40. collection type; differs for personal papers & organisational records 
  41. 41. same personnel work on paper and born-digital components?  
  42. 42. can we appraise without knowing the contents?</li></ul> similar to paper material that is in a different language?<br />
  43. 43. Challenges faced<br />Volume of material :<br />  - depositor perception that 'storage is cheap‘ - does this mean <br /> we shouldn’t appraise the material we receive?<br />  - wide range of file types encountered  <br />  - not practical to describe each and every file<br /> - risk management - if you don’t check every file for sensitive information<br />  <br />    - we need to automate as much of the processing as possible<br />
  44. 44. Hypatia<br />Digital archivists' identified a gap in current tools – used experiences to define the requirements for a new tool <br />Key features identified: <br /><ul><li>need an intuitive (for archivists) graphical interface
  45. 45. drag'n'drop to create the intellectual arrangement
  46. 46. ability to return to original order of the material
  47. 47. view some file types, add descriptive metadata etc
  48. 48. high level of granularity when applying rights & permissions</li></ul>Technical (acquired at accessioning) and descriptive metadata - Discovery & Access process<br />
  49. 49. Discovery and Access<br />Peter Chan<br />Stanford University<br />
  50. 50. What is Discovery & Access<br />Discovery and Accessrefers to the systems and workflows that make processed or unprocessedmaterial and the metadata that support it available to users. <br />
  51. 51. Goals of D&A<br /><ul><li>To make material available to user communities by ensuring that they can:
  52. 52. find out about material
  53. 53. understand whether it is available for consultation and if so, how
  54. 54. access material.
  55. 55. To apply appropriate access restrictions in order to protect private and sensitive information as well as intellectual property.
  56. 56. To provide access to material in a format and/or environment that presents the original’s significant properties.</li></li></ul><li>Case Study - Stephen Jay Gould Papers<br />Analog component: 550 linear feet of papers (789 boxes, 119 cartons, 30 flat boxes, and 14 map folders.<br />File size and number: 59.7 MB and 2,567 files.<br />Media formats: 98 3 ½” floppy diskettes; 61 5.25” floppy diskettes; 4 sets of punch cards*; 3 computer tapes <br />File Types: Computer Programs; Data sets; Documents; Spreadsheets<br />File Formats: ASCII Text; WordPerfect 4.2, 5.0, 5.1, 6.0, 6.1; Microsoft Word 2.0, 6.0, 97, 2000; Microsoft RTF; Microsoft Excel 4.0; Lotus 1-2-3 2.0, etc.<br />* During processing of the “analog” papers in 2011, another 21 sets of punch cards and more floppy diskettes were found.<br />
  57. 57. D&A – EAD<br />
  58. 58. D&A – Facet Browsing<br />
  59. 59. D&A – Full text search<br />
  60. 60. D&A – See Contents on Web<br />
  61. 61. D&A – Tag & Annotation by Invited Persons / Public<br />Annotation:<br />
  62. 62. Impacts fromCollection Development<br />File formats: no restriction<br />Computer medium: no restriction (punch card, open reel tape, 5.25 inch floppy, 3.5 inch floppy), <br />File type: no restriction (computer program, data set, document, spreadsheet), <br />Agreement: permission to post contents online.<br />
  63. 63. Impacts fromAccessioning<br />Built 5.25 inch floppy capture station<br />Ask Computer History Museum to read punch cards<br />Open reel tapes – still outstanding<br />
  64. 64. Impacts fromProcessing<br />AccessData FTK was used to search files with restricted information, annotate files with appropriate descriptive metadata (book title, articles, etc.), and rights metadata (access restriction), generate technical metadata for the delivery platform to act upon.<br />Transit Solution was used to transform files to html format for display in web.<br />A XSLT program was written to transform the XSL-FO output from FTK to XML content document. A Ruby program was written to ingest the XML content document, original files, and the display derivatives to Fedora.<br />
  65. 65. FTK – Bookmark and Label<br />
  66. 66. FTK – Full Text, Pattern Search & Fuzzy Hash <br />
  67. 67. Emulation – Design Files<br />
  68. 68. Network Diagram for 50,000 Creeley Emails<br />
  69. 69. MUSE: Sentiment Analysis for Emails<br />
  70. 70. MUSE: See Individual Email<br />
  71. 71. Want to know more?<br />http://born-digital-archives.blogspot.com<br />http://born-digital-archives.blogspot.com<br />Gretchen Gueguen Mark Matienzo<br />gmg2n@virginia.edumark.matienzo@yale.edu<br />Simon Wilson Peter Chan<br />s.wilson@hull.ac.ukpchan3@stanford.edu<br />

×
https://www.slideshare.net/AIMS_Archives/saa-session-502-born-digital-archives-in-collecting-repositories
www.slideshare.net
crawl=CC-MAIN-2017-51
264045
application/http; msgtype=response
sha1:EZIA4FMEX3JUGTL77Q5TSMZP7OADTXVK
<urn:uuid:50d8cc92-cca1-4b42-8c18-8abc5ebd78da>
2017-12-17T01:25:08Z
108.174.10.19
text/html
sha1:X6FJQB5NDDRLWZZF7LCZPYR7PNLZQBQL
<urn:uuid:69bbb659-e78a-48f7-b5a4-eba82daf076f>
https://www.slideshare.net/AIMS_Archives/saa-session-502-born-digital-archives-in-collecting-repositories
length
response
<urn:uuid:27463527-5132-4e79-8c9f-a7e72d9f5b5a>
robots: classic
hostname: ip-10-45-104-224.ec2.internal
software: Nutch 1.6 (CC)
isPartOf: CC-MAIN-2017-51
operator: Common Crawl Admin
description: Wide crawl of the web for December 2017
publisher: Common Crawl
format: WARC File Format 1.0
conformsTo: http://bibnum.bnf.fr/WARC/WARC_ISO_28500_version1_latestdraft.pdf