preserving digital television

THE PRESERVATION

OF DIGITAL TELEVISION:

Challenges, Requirements, and Strategies

Kara Van Malssen New York University

kara.vanmalssen@nyu.edu

LA PRESERVACION DE LA TELEVISION DIGITAL: Retos, Requerimientos, y EstrategiasSeminario de Conservación de Obras y Documentos Sobre Soprotes Electrónicos

Ciudad de México, 31 de julio 2009

Digital television:1. Born Digital2. Digitized from an analog / physical source

Photo by chrisdlugosz via Flickr http://www.flickr.com/photos/chrisdlugosz/3403751594/

Why is preserving digital television especially CHALLENGING

very large filesvery large filesvery large files

?

Analog formats cannot wait for the cost of uncompressed video storage to drop!

no standard format for

production or archiving

many LARGE files

Static Media Viewed with the naked eyeUtilized without additional componentsVery long useful life if stored properly

Photo from the Library of Congress via Flickr http://www.flickr.com/photos/library_of_congress/3738806589/

Dynamic MediaDependent on machines to view and utilizeMedia are fragileIndustry changes results in format obsolescence

Photo by Martin Deutsch via Flickr http://www.flickr.com/photos/teflon/2352818254/

Digital MediaNot tangibleEven more dependenciesMore frequent obsolescence of various componentsPlayback requirements not obviousMore content than ever before

Preservation of physical media...Primarily required good storage and disaster protection

Photo by vodstrup via Flickr http://www.flickr.com/photos/vodstrup/1486839907/

Preservation of digital mediaMuch more than just good storage and disaster protection

Photo by Scoobay via Flickr http://www.flickr.com/photos/scoobay/3163954667/

Preservation of audiovisual media is an ongoing process

There is no starting and stopping point

No audiovisual format will last forever

We can’t save everything

RISK FACTORS

Photo by kino-eye via Flickr http://www.flickr.com/photos/kino-eye/39529915/

Bit Rot

Photo by damclean via Flickr http://www.flickr.com/photos/damclean/2143598772/

Software Obsolescence+ Upgrades

Hardware Obsolescence

Processor Obsolescence

Photo by huangjiahui via Flickr http://www.flickr.com/photos/huangjiahui/3179858408/

Storage Media Obsolescence

Photo by Ian-S via Flickr http://www.flickr.com/photos/ian-s/2785762687/

Storage Media Failure

Photo by massdistraction via Flickr http://www.flickr.com/photos/sharynmorrow/3718174646

“Despite storing CD-Rs recorded in ideal conditions, tech site TechARP unboxed

300 CDs recorded between 7 to 9 years ago, and found that

they have a failure rate approaching 10 percent for

the first 173 discs--the restoration is still on-going.”Paul Mah, “The Problem of Bit Rot Revisited.”

FierceCIO: Tech Watch, 21 July 2009http://www.fiercecio.com/techwatch/story/problem-cd-bit-rot-revisited/2009-07-21?

utm_medium=nl&utm_source=internal

Lack of METADATA“If a piece of program material is not

correctly placed and identified on a digital system, it might as well not be there - no one will be able to find it or even know it exists.”

Cox, Tadic, Mulder. Descriptive Metadata for Television. Focal Press, 2006. p63.

If we didn’t know that footage of a Harvard Law

student speaking at a protest in 1990

existed at WGBH, would our lives be much different?

But because we do know of its existence

in the archive, and because it can be

ACCESSED, our collective history

is enriched

Lack / loss of organizational support

and resources

Photo by Roby (C) via Flickr http://www.flickr.com/photos/roby72/2401722298/

Photo by wallyg via Flickr http://www.flickr.com/photos/wallyg/2488178506/

REQUIREMENTS

1. Bit PreservationSustaining the 0s and 1s, or ensuring that the video, audio, and ancillary files remain intact over time with no loss or corruption of bits

Photo by adrenalin via Flickr http://www.flickr.com/photos/adrenalin/4250667/

2. Accessibility and Usability of ContentEnsuring that video, audio, and ancillary files can be found, retrieved, interpreted, played back, and delivered to the appropriate users.

Photo by Sunshine Junior via Flickr http://www.flickr.com/photos/sunshinetoday168/1323387457

3. Organizational InfrastructureAn entity (repository) explicitly responsible for keeping the content alive and accessible.

Photo by stefan1024 via Flickr http://www.flickr.com/photos/stefan1024/3682770758

STRATEGIES

RedundancyReplicated, geographically distributed storage

Online StorageIn house

Nearline tape storage In house

Offline tape storage Off site

1. Bit Preservation

Regularly audit files, repair corrupt files with backups

1. Bit Preservation

Photo by Niklas via Flickr http://www.flickr.com/photos/pivic/2829376039/

1. Bit Preservation

Refresh storage media periodically

Photo by TaranRampersad via Flickr http://www.flickr.com/photos/knowprose/111119035/

Where is the content?What is the content?

Who made it?Can I use it?

How can I display it properly?How can I preserve it?

Where did it come from?

Identification & OrganizationDescriptive MetadataDescriptive MetadataRights MetadataTechnical & Structural MetadataPreservation MetadataSource Metadata

Create, update, manage, and maintain good metadata throughout the

life cycle of the digital object

2. Accessibility and Usability of Content

Photo by Sunshine Junior via Flickr http://www.flickr.com/photos/sunshinetoday168/1332297470/

Preservation Version / MasterAccess Versions / Derivatives

Support current and future uses by

preserving the highest quality version to allow for many derivatives

2. Accessibility and Usability of Content

Proprietary or open source?Dependent on specific hardware and/or software?High market saturation?Good documentation / support community?

2. Accessibility and Usability of ContentPhoto by stefan1024 via Flickr http://www.flickr.com/photos/stefan1024/3701790091

Create files using open, standardized, non-proprietary file formats and codecs, high resolution or uncompressed, and avoid transcoding during production

Photo by tochis via Flickr http://www.flickr.com/photos/tochis/3431207670/

Migrate to new file formatwhen necessary

2. Accessibility and Usability of Content

3. Viable Organizational infrastructure

PRESERVATION

Establish a

And ensure sufficient funding, and staffing

Photo by comicbase via Flickr http://www.flickr.com/photos/comicbase/3338358430/

GENERAL REFLECTIONS

Photo by seriykotik1970 via Flickr http://www.flickr.com/photos/seriykotik/195406053/

Refer to the ISO standard OAIS (Open Archival Information System) Reference Model

PRODUCER

CONSUMER

MANAGEMENT

ADMINISTRATION

ACCESSINGEST

PRESERVATION PLANNING

SIP

AIP

DIP

DATAMANAGEMENT

ARCHIVAL STORAGE

AIP

DESCRIPTIVE

INFO

DESCRIPTIVE

INFO

queries

result setsorders

Establish Retention PoliciesWe can’t save everything

by Simon Evans

Establish file and folder naming conventions & workflow procedures

Implement changes during transition to digital production and distribution workflow

Create digital content with the long term in mindNot just the immediate broadcast or distribution needs

Photo by gbaku via Flickr http://www.flickr.com/photos/gbaku/2513320483/

Digital Asset Management

System derivatives

pre-production

production

derivatives

post-production

broadcast

web

legal

titlelocation

names & roles

programming

rightscontract details

formatlogs

IDs

final titleair date(s)

summary

broadcast dates

EDL

summariestranscripts

stills

library / archive

ITbackup dates

checksum

preservationstandardize terms

technical information

locationerrors

user tags

Technology watch

Photo by jacobian via Flickr http://www.flickr.com/photos/jacobian/2385490476/

Investigate cooperative solutions

Photo by quapan via Flickr http://www.flickr.com/photos/hinkelstone/2435823037/

Don’t know what they have

Have incomplete or no catalogs for much of their content

Lack equipment to playback much of their legacy content

Do not know the © status of much of their content

Cannot realize the potential of their collections!

Today, many television stations...

The digital era brings great opportunity to improve discovery, re-use, and cost savings in production,

plus wider access and interaction with audiences

Lets not risk the loss of our collective heritage, or waste the opportunity to take advantage

of the digital transition.

THANK YOU!kara.vanmalssen@nyu.edu