eRecords Conference 2013: Processing, Storage, and Search of E-records

erecordsslogo-300x261This is the eighth post in a multi-part recap of the 2013 e-Records Conference. Some presentation materials from the e-Records Conference are available on the e-Records 2013 website.

By Nanette Pfiester, Program Planning and Research Specialist

Exhibitors meet practitioners at the 2013 e-Records Conference.

Exhibitors meet practitioners at the 2013 e-Records Conference.

Edward Mann is a Senior Storage Architect with ViON Corporation. Mr. Mann began his career as a systems engineer for Storage Technology Corporation (STK) in Colorado. In 1995 he helped to implement one of the first large scale storage area networks at NASA Kennedy. He is a continuing resource to ViON, presenting at varied venues including the Super Computing Conference.

ViON Corporation (www.vion.com) is a small, veteran-owned corporation, established in 1980, and headquartered in Herndon, VA. They specialize in designing, delivering and maintaining storage and server solutions to enterprise-wide data centers throughout the federal government and public sector & commercial marketplaces.

Mr. Mann explained critical aspects of dealing with electronic records. As his session description promised, he described:

The vastly increased demand for e-records has created a need for an entire dedicated IT infrastructure, beyond the mere scanning of documents. This work involves the movement of data between hierarchal islands of storage. However, even though archived a document must be easily retrievable and move seamlessly between tiers of storage, be they fast disk or tape.

In particular, Mr. Mann shared information about advances in storage options and information management.

View Mr. Mann’s slide deck at: https://www.tsl.texas.gov/slrm/training/conferences/erecords2013.html.

Concepts and vocabulary

This session included some technical terms and acronyms that may have been a little daunting. However, these concepts about managing electronic records could bridge electronic records conversations between IT (information technology) and RM (records management) departments.

It is important that IT, RM, and archivists share an agreed vocabulary and work from shared definitions of terms like archive and archives as they are somewhat different in each discipline. You must establish a consistent vocabulary or glossary to best work with other staff and vendors. Otherwise, projects may not have the desired results. Several glossary links are included at the end of this article.

Business drivers

Mr. Mann touched on four business drivers that underlie efforts to better manage burgeoning e-records stores. Techopedia says that a business driver is a resource, process or condition that is vital for the continued success and growth of a business. Most organizations have large volumes of e-records (like digitized documents, e-mail, unstructured data, etc.) in disparate formats and separate stores. Some combination of these drivers may justify your projects to better process, store, and search all e-records.

  • Legal drivers recognize that you have legal obligations to maintain and provide accurate records. For instance, you could become involved in litigation requiring e-discovery. Also, your ability to comply with legal requirements in areas such as accounting, health records, student records, privacy, transparency and more.
  • Search drivers recognize that you must be able to search and locate all relevant records when needed. Plus, locating records for litigation, compliance, audits, etc. may affect the long-term viability of the organization.
  • Efficiency drivers recognize that it is often difficult to locate and provide records from different locations, format, and storage media. How many hours could be saved with some efficient means to locate records regardless of their physical or virtual location?
  • Storage drivers recognize that it is costly to base storage decisions solely on cost. Although a 1-terabyte drive may be cheap, searching through millions of unmanaged documents is expensive. Better solutions are available today. Also, long-term storage may facilitate records use to refer to the organization’s past actions, to plan for the future, to prove compliance, to prevent record changes, and more.

Importance related to time

Mr. Mann showed a chart equating the importance of data over time and the media used for data at different times. In the slide, multi-directional arrows show the three interconnected information stores that must be retained for proper amounts of time.

  • Your most recent data is most likely of high importance. This includes what you are working on right now. On-line storage is immediately available on your current network, but is the most costly.
  • Your slightly older data may be a little less important as it may be referenced less frequently. Near-line storage is less costly but available pretty quickly.
  • Your oldest data is what gets Archived (note this is the IT definition of “archive”, see the vocabulary section at the end). Off-line storage may be the cheapest storage, may be less frequently accessed, and may take a little longer to access. This information store contains unchanging records. Backup volume, time and cost are reduced by processes like eliminating duplicate records. Newer storage solutions make the Archive more readily available than retrieving tapes from offsite storage to mount and search.

Search and Metadata

Ideally, Mr. Mann suggested you would want to find records through a single search across all three types of information stores. He described an ideal environment that should be integrated to allow for intelligent migration, tiering, searching, and reducing backup. These three information stores should be virtually linked in such a way that they are easily searched and accessed.

Mr. Mann stressed the need for excellent metadata to aid in searching records across all three types of information stores. Metadata is the data (information) connected with records. Just a few metadata elements might include subject, creator, create or update date, etc. He put forth the idea that the archived records are not a single store, but that virtual archives keep different types and sources of data separate with different permissions granted to users under tenants and namespaces. User accounts may authorize specific users’ permission in these storage areas to perform activities including browse, read, write, change owner, delete, purge, privileged delete, or search.

Successful projects

Mr. Mann described a process to improve project success that has been around since World War II. The ADDIE model suggests an iterative sequence to Analyze, Design, Develop, Implement, and Evaluate (ADDIE). I’ll cover just two of these steps here. The Analysis step is your needs assessment. That is when you identify “I’m here and want to get there.”  The Implementation step must be done so that you do not implement the wrong thing. If more projects used this algorithm, more projects could be successful.

Conclusion

The rapid growth of e-records volumes makes their processing, storage, and search increasingly complex. Mr. Mann’s presentation suggested new technological advances and compelling business reasons to explore how you can take control of your information using these techniques.

Some glossaries are:

Leave a Reply

Your email address will not be published. Required fields are marked *


This site uses Akismet to reduce spam. Learn how your comment data is processed.
https://www.tsl.texas.gov/slrm/blog/2013/12/e-records-conference-2013-processing-storage-and-search-of-e-records/
www.tsl.texas.gov
CC-MAIN-2024-38
66804
application/http; msgtype=response
sha1:6F25JMWSOS3W55SYMLMJOZD7R5ECYKAK
<urn:uuid:48a5f59a-cbfb-4542-817e-92283746a6d5>
2024-09-16T08:53:45Z
204.65.32.169
text/html
sha1:DJZY25EV2EPKUYS32S7WUXVP3DSZ5OSO
<urn:uuid:cd2fc101-8ee2-48cb-9c5b-034b5597538c>
https://www.tsl.texas.gov/slrm/blog/2013/12/e-records-conference-2013-processing-storage-and-search-of-e-records/
response
<urn:uuid:89cf1180-2f94-4af7-8a5b-1c160afe2c64>
isPartOf: CC-MAIN-2024-38
publisher: Common Crawl
description: Wide crawl of the web for September 2024
operator: Common Crawl Admin (info@commoncrawl.org)
hostname: ip-10-67-67-163
software: Apache Nutch 1.20 (modified, https://github.com/commoncrawl/nutch/)
robots: checked via crawler-commons 1.5-SNAPSHOT (https://github.com/crawler-commons/crawler-commons)
format: WARC File Format 1.1
conformsTo: https://iipc.github.io/warc-specifications/specifications/warc-format/warc-1.1/