LISTSERV mailing list manager LISTSERV 16.5

Help for CODE4LIB Archives


CODE4LIB Archives

CODE4LIB Archives


CODE4LIB@LISTS.CLIR.ORG


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

CODE4LIB Home

CODE4LIB Home

CODE4LIB  June 2017

CODE4LIB June 2017

Subject:

Re: SIP/AIP Content Guidelines

From:

Andrew Weidner <[log in to unmask]>

Reply-To:

Code for Libraries <[log in to unmask]>

Date:

Fri, 2 Jun 2017 13:10:53 -0500

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (170 lines)

Thank you Lydia and Grant. I'm impressed with the thorough documentation
that you all have produced for your preservation services.

I've posted a similar inquiry to the Archivematica forum that explains in a
bit more detail how this 1 object : 1 SIP/AIP preservation model arose out
of some throughput problems we encountered with transfers to archival
storage, especially with AV resources that seem to require 1 object per SIP
because of the temporary storage needs for processing such large files.
We've also noticed some murky indexing issues related to file sizes and
number of files in a SIP that lead us to believe the only consistent rule
for automated SIP creation that accommodates all of our content types is 1
object = 1 SIP/AIP.

We could manually create each SIP/AIP with larger aggregations of content
based on what we think Archivematica can handle. That seems like a lot of
work up front with the potential for introducing human error. In a workflow
that splits AIP and DIP creation into separate streams at the start,
manually aggregating content also hampers our ability to assign persistent
identifiers for preservation packages that we can link to from other
systems.

I'm curious how Scholars Portal plans to handle AIP creation in the Books
workflow, where the AIP and the DIP are created separately, and how you're
creating the metadata linkage back to the access platform. That seems to be
similar to what we're working on.

Andy


On Fri, Jun 2, 2017 at 12:00 PM, Grant Hurley <[log in to unmask]>
wrote:

> Hi Randy,
>
> This is a good question! Thanks for pointing out to Scholars Portal, Kari.
> I think that the hospitable concept of the SIP/AIP/DIP sort of precludes a
> perfect or ideal version in real life – the decision always in relation to
> the content and the preserving organization’s context, resources and needs.
> I agree with Kari that for many contexts, SIPs/AIPs are best aggregated on
> some functional basis. For us, the consideration is definitely related to
> how we provide access to DIPs and want to manage our metadata.
>
> We’re still very much in the pilot phase regarding using Archivematica for
> processing larger numbers of files. There’s definitely lots of interest in
> this out there - consulting the Archivematica forum<https://groups.google.
> com/forum/?fromgroups#!forum/archivematica> is a good place to start. I’d
> also point to institutions such as Simon Fraser University and the
> University of British Columbia (and there are definitely others out there
> too I’m forgetting!) who have developed great automated workflows for the
> ingest of theses and other materials through Archivematica.
>
> For context: Scholars Portal<http://scholarsportal.info/> hosts a number
> of content platforms consisting of shared collections of published research
> materials and data (geospatial, micro-data etc.) that we load from multiple
> sources to serve to the members of the Ontario Council of University
> Libraries and the general public. Our preservation services are most fully
> developed for our journals platform<https://journals.scholarsportal.info/>
> and the books project is in progress. All of our documentation for the
> preservation of journals is online here: https://spotdocs.
> scholarsportal.info/display/OAIS/Home and you might find some of the
> definitions of SIP/AIP/DIP and details on fixity checking relevant. An
> important thing to note for us on this scale is that the DIP is generated
> on demand from the stored AIP. This is also why there’s a 1 article – 1
> SIP/AIP model. For our books platform, we’re looking at a different
> workflow where the AIPs and DIPs are created separately, so the
> Archivematica workflow is concentrated on creating AIPs only, but whose
> metadata links back to the access platform. It’s also notable that in both
> of these cases, the SIP/AIP is not just the article or book – it can also
> include supplementary files and materials such as images – so its not as
> atomistic as it initially appears.
>
> Hope this helps!
>
> Best,
>
> Grant
>
> Grant Hurley | MA, MAS, MLIS
> Digital Preservation Librarian
> Scholars Portal, Ontario Council of University Libraries
> 416-978-5648 | [log in to unmask]<mailto:[log in to unmask]>
> | www.scholarsportal.info<http://www.scholarsportal.info> |
> www.granthurley.ca<http://www.granthurley.ca>
>
>
> On 2017-06-02, 10:25 AM, "Code for Libraries on behalf of Andrew Weidner" <
> [log in to unmask]<mailto:[log in to unmask]> on behalf of
> [log in to unmask]<mailto:[log in to unmask]>> wrote:
>
> Thanks for the advice, Kari. I'll take a look at those resources.
>
> It's the management burden of having thousands upon thousands of AIPs
> that will become the bottleneck / digital management problem in the future.
>
> This is the pushback that I often hear from digital preservationists when
> talking about the single object AIP approach, and I'd like to learn more
> about these concerns. Any clarification of the long term management burden
> posed by single object AIPs would be most helpful.
>
> Andy
>
> On Fri, Jun 2, 2017 at 9:17 AM, Kari R Smith <[log in to unmask]<mailto:
> [log in to unmask]>> wrote:
>
> Hi Randy,
> I suggest that you post this question to digipres list (ALA list) where
> many digital archivists and digital preservation folks will see you message
> and can respond from that perspective.  Also, don't forget the concept of
> the AIC (archival information collection) which is an aggregate of AIPs.
>
> I would recommend connecting with Scholar's Portal in Canada.  They have
> an Archivematica workflow that is very item level (article = 1 SIP) and
> have experience and many lessons learned about the issues regarding
> performance, management, and scale of having one item in a SIP/AIP.  Grant
> and Kate from SP recently did a presentation on their approach:
> http://charlotteinitiative.uncc.edu/sites/default/files/
> users/2477/presentations/davis-hurley-charlotteslides.pdf
>
> Also keep in mind that the purpose of the AIP is for long-term
> preservation that accumulates both metadata and changes to file formats
> over time.  The DIP (Dissemination information package) is the xIP for
> which you may want to have a 1 xIP = 1 item relationship.  The DIP is
> created from that AIP (in the ideal workflow) so you can have 100 DIPs
> generated from a single AIP (that contains the 100 image files, etc.)
>
> It's the management burden of having thousands upon thousands of AIPs that
> will become the bottleneck / digital management problem in the future.
> Aggregate solutions for digital files, even and especially for digitized
> material are more the norm than individual xIPs.
>
> Good luck,
> Kari
>
> Kari R. Smith
> Digital Archivist and Program Head for Born-digital Archives
> Institute Archives and Special Collections
> Massachusetts Institute of Technology Libraries, Cambridge, Massachusetts
> 617.253.5690   smithkr at mit.edu   http://libraries.mit.edu/archives/
> @karirene69
>
> -----Original Message-----
> From: Code for Libraries [mailto:[log in to unmask]] On Behalf Of
> Andrew Weidner
> Sent: Friday, June 02, 2017 9:57 AM
> To: [log in to unmask]<mailto:[log in to unmask]>
> Subject: [CODE4LIB] SIP/AIP Content Guidelines
>
> Hi all,
>
> Can anyone point me to guidelines or best practices documentation around
> creating SIPs for transfer to archival storage? What does an ideal AIP look
> like for digitized cultural heritage materials?
>
> I'd like to set up a pipeline that sends single object (e.g. one
> photograph, one book) SIPs from our digitization workflow to Archivematica
> for automated transfer to archival storage. Here's a brief slide deck
> outlining the approach I'm envisioning:
>
> https://docs.google.com/presentation/d/19F5seismyBdhgIWk7Kt0jmJjqis--
> FCpOwNr3v6Iu-w/edit?usp=sharing
>
> I welcome any thoughts that you all may have on this, especially about
> pitfalls to avoid.
>
> Thanks,
>
> Andy
>
>
>

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005
December 2004
November 2004
October 2004
September 2004
August 2004
July 2004
June 2004
May 2004
April 2004
March 2004
February 2004
January 2004
December 2003
November 2003

ATOM RSS1 RSS2



LISTS.CLIR.ORG

CataList Email List Search Powered by the LISTSERV Email List Manager
https://lists.clir.org/cgi-bin/wa?A2=ind1706&L=CODE4LIB&P=4281
lists.clir.org
crawl=CC-MAIN-2021-39
64953
application/http; msgtype=response
sha1:C3V2LB3J7SGSE23GDZUUPKFL2XWHR6AN
<urn:uuid:b8a1a24a-bb93-4cd9-9430-7302f6e70428>
2021-09-28T14:34:17Z
66.175.211.245
text/html
sha1:WC3PSJVV5LSUVRDEEW2FWG4S5C73OEO2
<urn:uuid:f1f80a65-715a-48bd-8679-c61706baa1e6>
https://lists.clir.org/cgi-bin/wa?A2=ind1706&L=CODE4LIB&P=4281
response
<urn:uuid:7a0ceb22-7add-4e16-baf6-cbb1826bcd7c>
isPartOf: CC-MAIN-2021-39
publisher: Common Crawl
description: Wide crawl of the web for September 2021
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-102
software: Apache Nutch 1.18 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.2-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/