getting bits off disks: using open source tools to stabilize and prepare born-digital materials for...

72
Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials for long-term preservation Sam Meister University of Montana Best Practices Exchange 2013 November 13, 2013

Upload: samalanmeister

Post on 16-Jan-2015

496 views

Category:

Education


2 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Getting Bits off Disks:Using open source tools to stabilize and

prepare born-digital materials for long-term preservation

Sam MeisterUniversity of Montana

Best Practices Exchange 2013 November 13, 2013

Page 2: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Born-Digital Workflow

Acquisition Accession

Arrangement&

Description

Discovery&

Access

Page 3: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Acquisition Accession

Arrangement&

Description

Discovery&

Access

Page 4: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Acquisition Process

Donor Survey

Feasibility Assessment

Transfer Agreement

Page 5: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Donor Survey

Creation

Context

Organization

Privacy & Security

Storage

Technical

Transfer Options

Page 6: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Donor survey

Current

Page 7: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Future

DrupalWeb Form XML / CSV

Page 8: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Feasibility Assessment

Do we have resources to feasibly acquire, preserve, and provide access to the digital materials?

Page 9: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Transfer

Physical Media Network

ARCHIVES

Page 10: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Current

Page 11: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Future

DrupalWeb Form XML / CSV

Page 12: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Acquisition Accession

Arrangement&

Description

Discovery&

Access

Page 13: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Accession Process

Disk Image Media

Initial Analysis

Produce AIP

Page 14: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Data Transfer

3.5 Floppy Drive5.25. Floppy Drive

Zip DriveCD / DVD Drive

USB Write-BlockerSATA / IDE Write-Blocker

Hardware

FTK ImagerGuymager

FC5205

Software

Page 15: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Disk Imaging

“A single file or storage device containing the complete contents and structure representing a data storage medium or device, such as a

hard drive, tape drive, floppy disk, CD/DVD/BD, or USB flash drive”

Page 16: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Disk ImagingBorn Digital Workstation 1.0

Page 17: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Disk ImagingBorn Digital Workstation 2.0

Page 18: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 19: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 20: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 21: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 22: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Disk Imaging

Get Media

Assign Identifier

PhotographMedia

Record Characteristics

Write-Protect Media

Create Image

Export Files

Virus Scan

Page 23: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 24: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 25: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

FC5205Disk Image and Browse

Page 26: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 27: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 28: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 29: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 30: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 31: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 32: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

FTK Imager

Page 33: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 34: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 35: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 36: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 37: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 38: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Issue:

Unknown / Unrecognized Filesystems

Page 39: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Options:

Kryoflux

Page 40: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Initial Analysis

Extract Metadata

Identify Restricted

Info

Identify Duplicates

GenerateReports

Page 41: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Initial Analysis

Hardware

BitCuratorfiwalk

Bulk Extractor

Software

Page 42: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

“an effort to build, test, and analyze systems and software for incorporating digital forensics methods

into the workflows of a variety of collecting institutions”

Page 43: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

BitCurator:

fiwalk

Page 44: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 45: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 46: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

BitCurator:

bulk_extractor

Page 47: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 48: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 49: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 50: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 51: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

BitCurator:

Reports

Page 52: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 53: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 54: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

AIP = Archival Information Package

Produce AIP

Page 55: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Produce AIP

Hardware

Archivematica

Software

Page 56: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

“a free and open-source digital preservation system that is designed to maintain standards-based,

long-term access to collections of digital objects”

Page 57: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 58: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 59: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 60: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 61: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 62: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation
Page 63: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Produce AIP

Archivematica

Using version 0.10 on dedicated workstation

(testing as virtual server)

Current

Install version 1.0 on server with multiple client

nodes (workstations)

Future

Page 64: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Acquisition Accession

Arrangement&

Description

Discovery&

Access

Page 65: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

A & D

Prepare

Develop Processing Plan

Implement Processing Plan

Page 66: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

A & D

• Integrate Born Digital materials into existing A&D process / tools (mix of Excel, Word, XMetal XML editor)

Current

• Determine tools needed for reviewing content (data visualization)

• Integrate Born Digital materials into collection management system

Future

Page 67: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Born-Digital Workflow

Acquisition Accession

Arrangement&

Description

Discovery&

Access

Page 68: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

• Embrace iterative approach (use what you have and get what you need when you need it)

• Capture as much metadata as possible (descriptive, structural, administrative)

• Start with workflow requirements (what needs to be done) then test tools (what things will get it done)

• Build flexibility into system (may not always be ideal scenarios)

Lessons Learned

Page 69: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Open Source - Issues

• May require specific IT environment (Linux)

• Tools likely to change quickly

• User interfaces / experience may be simple

• Will need ongoing support from IT / Systems staff

Page 70: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Open Source - Benefits

• Limited initial resources needed to install and test

• Provides opportunity to engage systems / IT in new areas

• Designed and developed in collaboration with archival community

• Direct communication channels to contribute to / modify development roadmap

• Quickly build initial standards-compliant workflow

Page 71: Getting Bits off Disks: Using open source tools to stabilize and prepare born-digital materials  for long-term preservation

Resources

FC5205 Disk Image http://www.deviceside.com/fc5025.html

Kryofluxhttp://www.kryoflux.com/

BitCuratorhttp://www.bitcurator.net/

Archivematicahttps://www.archivematica.org/wiki/Main_Page