besser--lazerow 12/10/02 1 the challenge of media preservation: digital works and time-based media...

Besser--Lazerow 12/10/02 1

The Challenge of Media Preservation:Digital Works and Time-Based Media

Howard Besser

NYU Archiving and Preservation Program and Library Senior Scientist

http://www.gseis.ucla.edu/~howard

http://www.tisch.nyu.edu/preservation/


The Challenge of Media Preservation:Digital Works and Time-Based Media

_ How these works differ from conventional works

_ Preservation problems they pose_ Necessary paradigm shifts_ Importance of managed environments,

metadata, ancillary materials


Conventional Works

_ Manuscripts, books, paintings, sculpture_ We have a good sense of what the original object

is_ Objective is to make object itself endure

(temperature/humidity control, chemicals/pigments/fibers/adhesives, …)

_ Goal is to keep object as close as possible to original state (though occasionally contraversy arises over whether to let aging show)


Electronic Media

_ Video, audio, digital, new media_ Often difficult to determine what the original object is_ Difficult to make the original object endure (magnetic

particle deterioration, warping, etc.)_ Even if we could make the original object endure, we

wouldn’t have the infrastructure to view it in the future_ Need to develop a paradigm shift from preserving the

original object to preserving info content_ Need to pay more attention to maintaining authenticity and

replicating user experience


Preserving electronic media:What’s the problem & What can we do

about it?_ The Problem: Maintaining Accessibility to a Work over time_ New Products and New Expectations necessitate New Paradigms_ The Easy Part of the Problem: Physical storage devices &

hardware players_ The Hard Part of the Problem: File Formats_ Problems with Many Types of Digital Works Today_ Special Characteristics of Electronic Works_ What is the Work?_ Some approaches to solutions


Serious Digital Longevity Problems

What we know from prior widespread digital file formats

Non-text works separating from their metadata

Inaccessibility of software needed to view a non-text work

Inability to even decode the file format of a non-text work


The Problem:Maintaining Accessibility to a

Work over time

_ Preservation techniques for Physical artifacts do NOT address the problem of preserving electronic works

_ Need to shift from preserving Physical Artifact to preserving disembodied content


Electronic Art is not like canvas paintings

_ In the future these may include– Moving image materials

– Multimedia

– Interactive programs (including hypertext novels & games)

– Computer generated art

_ These works share some common characteristics with other “strange” works like– Performance Art

– Conceptual Art

– Site-specific installations

– Experiential Art


Thinking of the Future (1/2)

_ Screens will be different resolutions and different aspect ratios

_ CRTs won’t exist_ A decade or 2 from now, today’s user

interfaces will look like arrow-key navigation looks like today


Thinking of the Future (2/2)

_ Today’s streaming media are small windows, slow speeds

_ As bandwidth increases, viewers will expect higher quality streams

_ Creators may need to consider how they’ll be able to deliver higher-bandwidth streams– Delivery Derivatives vs. Masters encoded w/standards

– May also want to re-edit the piece to take advantage of changes in technology, viewer expectations, society-


The Easy Part of the Problem: Physical storage devices &

hardware players


Analog - EIAJ 30


Analog - U-Matic


Analog - VHS & Betacam


8mm video


Digital -128M Optical


Digital - Optical R/W


Digital - DAT


Digital - Syquest


Digital - Zip 100


Digital - Writeable CD-ROM (650M)


Edison


Early Wax


The Hard Part of the Problem:File Formats

_ even with text like Wordstar & Word


Problems with Many Types of Digital Works Today-

Disappearing Information The Viewing Problem The Scrambling Problem The Inter-relation Problem The Custodial Problem The Translation Problem


The Viewing Problem

Digital Info requires a whole infrastructure to view it

Each piece of that infrastructure is changing at an incredibly rapid rate

How can we ever hope to deal with all the permutations and combinations


The Scrambling Problem

Dangers from: Compression to ease storage & delivery Container Architecture to enhance digital

commerce


The Inter-relation Problem

-Info is increasingly inter-related to other info

-How do we make our own Info persist when it points to and integrates with Info owned by others?

-What is the boundary of a set of information (or even of a digital object)?


The Custodial Problem

In the past, much of survival was due to redundancy

How do we decide what to save? Who should save it?

Mellon-funded E-Journal Archives How should they save it?-


The Custodial Problem:How to save information?

Methods for later accessRefreshingMigrationEmulation

Issues of authenticity and evidence


The Translation Problem

Content translated into new delivery devices changes meaning– -A photo vs. a painting– -If Info is produced originally in digital form in

one encoded format, will it be the same when translated into another format?

– Behaviors


Serious Longevity Problems

What we know from prior widespread electronic file formats

Previous formats required little ongoing intervention (remote storage facilities, Iron Mtn); digital formats require intense ongoing management


Conceptual Approaches to Digital Preservation

_ Refreshing always necessary due to volatility of physical strata– Impact on evidential value

_ Migration -- advantages & disadvantages_ Emulation -- advantages & disadvantages

_ And will need a long-term managed environment-


Managed Environment

_ More than temperature & humidity control_ Periodic monitoring of the works_ Periodic monitoring of the technical

environment for viewing the works (software, systems, hardware)

_ Trusted repositories-


OCLC/RLGDigital Repository Attributes

_ Administrative responsibility_ Organizational viability_ Financial sustainability_ Technological suitability_ System security_ Procedural accountability


OCLC/RLGSelected Recommendations

_ Policies, Certification processes, Risk management, Persistent ID, Migration/Emulation experiments

_ Stakeholders meet to decide how to describe what is in a dig repository

_ Examine special properties of particular classes of digital objects

_ Technical standards for exchange and interoperability btwn repositories

_ Develop projects and case studies_ Copyright issues


Preservation Repositories:Open Archival Info System Model

High-level reference model describing submission, organization and management, and continuing access

Conceptual framework for different organizations to share discussions with a common language

Producers, consumers, management, actual repository SIP, DIP, AIP AIP consists of data objects plus representation info

(Content, Preservation Description, Packaging, Descriptive)

Originally developed for Space Science community


Preservation Repositories:Open Archival Info System Model

Producer

Management

Consumer


Older Longevity Projectshttp://sunsite.berkeley.edu/Longevity/

CPA Task Force Getty “Time & Bits” Conference & Follow-ups- Preservation experiments in US and Europe

NEDLIB, CURL, Michigan

Internet Archive Long Now


Other Digital Preservation Activities-

LC Natl Dig Info Infrastructure & Preservation InterPARES Emulation Projects E-Journal Archiving ERPANET Persistent Naming


LC’s National Digital Information Infrastructure and

Preservation Program_ Authorized Dec 2000_ LC, Dept of Commerce, NARA, White House

Office of Sci & Tech Policy_ with help from CLIR, NLM, NAL, OCLC, RLG_ Ongoing collab process_ Commissioned papers on preserving: the Web,

periodicals, digital sound, E-Books, Digital TV, Digital Video


InterPARES International Research on Permanent Authentication

Records in Electronic Systems

_ Ongoing international archival world project examining how to make electronically-generated records last over time

_ Developing the theoretical and methodological knowledge needed, then will formulate model policies, strategies, and standards

_ Next year will be extended to include images and rich media


Emulation Projects

_ CAMiLEON (Michigan/Leeds)_ NEDLIB


E-Journal Archiving

_ Issues– License, don’t own; may not be even able to obtain right to make archival

copy

– Increasingly no paper back-up at all

– Usually we don’t have the important redundancy factor

_ Mellon funded projects (2001)– Yale, Harvard, Penn working w/individual publishers

– Cornell, NYPL--specific disciplines

– MIT exploring characteristics that change (dynamic)\

– Stanford--archiving software tools


Electronic Resource Preservation and Access NETwork (ERPANET)

_ Best practices and skills development for digital preservation of cultural heritage and scientific objects

_ 3 year project launched Nov 2001; 1.2 million Euros


NEA 2001 grant to BAVCfor $150,000

_ “To support development and dissemination of a DVD that contains a curriculum for the preservation of electronic art. The DVD will feature a preservation overview; discussions with conservators, artists, curators and technicians; a curriculum to train professionals in the field and project case studies to conserve electronic art.”


Special Characteristics of Electronic Works

_ What Really is the Work?-_ Disappearing software_ Enormous number of elements can, at times, be very important to

preserve (randomness, interactivity, pacing, color, format, original artifact, elements used to construct the artifact)

_ Pieces and Boundaries_ Recontextualization (Postmodernism)--which rendition to save?_ Dynamic & Lack of Fixity (evolving works)_ Interactivity_ Historical context_ Difficulty of authentication over time


What’s special about Cult Heritage Materials?

_ Images & rich media_ Inter-relationships btwn parts_ For Contemporary Art: What is the Work?-


What Really is the Work?-


LeWitt: Wall Drawing 340


Installing LeWitt


LeWitt Install Directions


ECI - Imagespace (early 80s)


ECI - Hole in Space (both)


ECI - 84-locations


ECI - 84-Community Memory


ECI - 84-kids


ECI - 84-MOCA


ECI - 84-Annotating Video


ECI - Avatars & Humans


Video Technology to Make the Head Spin (NYT 3/2/00)


Special Characteristics of Electronic Works (again)

_ What Really is the Work?-_ Disappearing software_ Enormous number of elements can, at times, be very important to

preserve (randomness, interactivity, pacing, color, format, original artifact, elements used to construct the artifact)

_ Pieces and Boundaries_ Recontextualization (Postmodernism)--which rendition to save?_ Dynamic & Lack of Fixity (evolving works)_ Interactivity_ Historical context_ Difficulty of authentication over time


More Technical Issues

_ Complexity of formats (storage & compression)_ Synchronicity between media/streams_ Persistent Ids-_ Website mgmt-


Persistent IDs--the Problem

_ Need to separate work ID from work location

_ URNs probably won’t be ready until 2003_ Becomes a business process issue when one

organization maintains the resource and another organization references it (ie. licensed from vendors or managed by separate administrative structures)


More Persistent IDs--the Approach for today

_ PURLs_ Handles_ HTTP redirects

_ And worry about costs now and conversion costs when URNs become feasible


Website ManagementMore issues with referencing IDs

_ References for mirror sites_ References for back-up sites when main site

is down or bottle-necked_ References for off-site copies and archival

copies


Pieces of the General Solution (1/2)

-We need to insist upon clearly readable standardized ways for digital objects to self-identify their formats

-We should discourage scrambling -We need to better understand information

inter-relates to other Info, and what constitutes “boundaries” of Info objects


Pieces of the General Solution (2/2)

-People and organizations wishing to make information persist need guidelines of how to go about doing it

-We need to better understand how translating from one storage or display format to another affects the meaning of a work

-We need to save the “behaviors” of a digital object, not just it’s “contents”


Metadata & Standards can be the first line of defense

Can tell you– where the file is (if you can’t find the file)– where more info about the file is (if you have the

file but most other metadata has become separated)

– what the file format is– what the compression scheme is– what application program and version is needed

for the file


From the technological point of view:

Standards offer the best hope of overcoming Impediments

_ Easier to maintain a single set of standards over long periods of time

_ Puts your institution in the same large boat with lots of other institutions who will face obsolescence and migration problems periodically throughout the future


What can we do specific to electronic media?

_ Works themselves may no longer even exist; in many cases, what we can save amounts to forensic evidence

_ Enormous number of elements can, at times, be very important to preserve (pacing, original artifact, elements used to construct the artifact)

_ Too complex to save every one of these aspects for every type of material_ Importance of saving pieces, representations, and documentation_ Involve creators & curators to capture intentions_ Importance of Standards_ Familiarize ourselves with recent conservation developments

(Guggenheim’s Variable Media, Who Knows?, TechArcheology, Tate, IMAP)-


Standards for encodingartists intentions

(group efforts w/i Cult Heritage community)

_ Artists Interviews Project, Netherlands Institute for Cultural Heritage 1998-1999, Modern Art: Who Cares (http://www.icn.nl/english/6.4.2.html)

_ TechArcheology: A Symposium on Installation Preservation (SFMOMA)_ More recent SFMOMA/Tate collaborations_ IMAP_ Guggenheim’s Variable Media

_ Need to be expanded to include curators’ & viewers’ reaction to the work


Authors Intentions: Kendall Example


Things that can be done

_ Save documentation about the work and its context

_ Save interviews with viewers about the experience_ Construct repositories that save software, works,

hardware, and engage in ongoing emulation_ Encode creators’ intentions_ Adhere to non-proprietary software/standards as

much as possible-


Tensions around Standards

Follow Standards (No Quicktime, Realmedia, DHTML, Flash, …)

_ innovative, new functionsinnovative, new functions


Structural Metadata Standards for Encoding Multimedia-

_ SMIL_ MPEG 4


Synchronized Multimedia Integration Language (SMIL)

_ For repurposing and reuse in different ways_ Use XML to reference various pieces in different

ways_ Supported by Realmedia but not Microsoft or

Macromedia


MPEG 4

_ Object-oriented_ Very low level of granularity (even objects vs

backgrounds)_ Scaleable bandwidth use_ Binary Format for Scenes (BIFS) borrows

concepts from VRML


Identification/Provenance(Still Images)-

The number of variant forms of a work can be enormous Image Families A digital image frequently has many layers of parentage Information about the parentage that can indicate the

quality and veracity of the image (Dublin Core "Source" and "Relation")

how to deal with different versions derived from the same scan or different encoding schemes

Vocabulary Standards to express this


The number of variant forms of a work can be enormous

different views of the same object different scans of the same photo different resolutions different compression schemes different compression ratios different file storage formats different details of the same image ...

Image Families


Identification/Provenance

how to deal with different versions (browse, hi-res, medium res) derived from the same scan or different encoding schemes (TIFF, PICT, JFIF)

Vocabulary Standards to express this– VRA Surrogate Categories– CIMI's "Image Elements”


Incorporate parts of Functional Requirements for

Bibliographic Records (FRBR)_ work_ expression_ manifestion_ item

_ (and push into “change history” section of Technical Image Metadata)


NISO/DLF Technical Image Metadata Workshop--4/99

(Z39.87-2002 draft)

create metadata needed to manage images in digital repositories over long periods of time (full life-cycle mgmt)

document image provenance & history ensure that the images will be rendered

accurately on any output device


Reference Modelsfor

Digital Libraries:Actors and Roles

DELOS/NSF Working Group

http://www.delos-nsf.actorswg.cdlib.org/


NSF/DELOS Actors/Roles Project

_ Classes of Actors, including– Persons

– Organizations

– automata

_ Roles & implications– Production

– Dissemination

– Management

– use


Multimedia & Collaborative Authorship imply

_ Not only:– Authors

– Editors

– Publishers

_ But also creators of– Text

– Illustrations

– Composers

– Musicians...


New products & Shifts in user expectations

_ VCR led to user expectation of time-shifting, which in turn created market for video-on-demand and cinema multiplexes with staggered times

_ WWW will further increase expectation of immediacy, as well as:– viewing massive amounts of related materials

– viewing material in fragmented ways

_ DVDs are even further increasing need for ancillary materials, as well as for interactivity and viewing fragments


Key Paradigm Shifts for Libraries/Museums/Archives

(also implications for creators)

_ Maintain long-term managed environment (life-cycle if possible)

_ From archiving completed whole works to asset mgmt of both component parts of works and ancillary materials related to the work

_ From preserving a physical artifact to saving a digital work that has no tangible embodiment


Saving Component Parts & Ancillary Materials

_ Edit decision list_ Out-takes_ Special effects_ Initial casting calls_ Sketches of sets_ Interviews_ Scripts...

_ If the Archivists don’t act to save these, their role will be marginalized. Need to proactively seek out material that may be routinely tossed out.

_ These materials may become important for either re-issuing the piece in a later version, or for future curators trying to understand the artist’s intention enough to re-display the piece when all the software used is no longer available.


Not just saving the Work,but also saving Conservation

info about the Work


Condition ReportUK National Trust


Annotated ImageMississippi State, Heather Gray


Polarizws ImageMississippi State, Heather Gray


X-RayInvestigating the Renaissance--Portrait of a Man, Fogg Museum


UnderdrawingInvestigating the Renaissance--Last Judgment, Fogg Museum


How Best to save these?

_ Use Standards wherever possible_ Be aggressive about asset mgmt -- saving

component parts and ancillary materials_ Both creator and Archive should develop an

institution-wide plan for saving electronic works– Refreshing and either migration or emulation– Standard encoding schemes– What is the work? And prioritize what needs to be saved– Save ancillary materials and records


NYU Moving Image Archiving Program (MIAP)

_ 2 year MA Program_ train future professionals to manage preservation-level

collections of film, video, new media, and other types of digital works

_ Provide prospective collection managers and archivists with an international, comprehensive education in the theories, methods, and practices of moving image archiving and preservation

_ Curriculum will cover all aspects of moving image archiving


MIAP Program Training

_ Film History/Historiography and Film Style_ Conservation, Preservation, Storage, and Management_ Legal Issues and Copyright_ Laboratory Techniques_ Moving Image Cataloging_ Curatorial Work and Museum Studies_ Programming_ New Media and other Digital Technologies_ Access to Archival Holdings


NYU MIAP Courses_ Introduction to Moving Image Archiving and Preservation_ Film History/Historiography_ Film Form and Film Sense_ Television History and Culture_ History and Culture of Museums, Archives, and other Repositories_ Conservation & Preservation of Moving Image Material--Principles_ Collection Management_ Access to Moving image Collections_ Copyright, Legal Issues, and Policy_ Handling New Media_ The Archive, the Collection, the Museum_ Curating, Programming, Exhibiting, and Repurposing/Recontextualizing Moving Image Material_ Film Restoration_ Video Restoration_ Digital Preservation and Restoration_ Elective or Independent Study_ Advanced Preservation Studies Workshop_ Thesis or Portfolio


Howard Besser

NYU Archiving and Preservation Program

and Library Senior Scientist http://sunsite.berkeley.edu/Longevity/ http://www.longnow.com/10klibrary/TimeBitsDisc/ http://www.oclc.org/digitalpreservation/presmeta_wp.pdf http://is.gseis.ucla.edu/us-interpares/ http://www.archive.org/ http://www.firstmonday.dk/issues/issue7_6/besser/index.html http://www.gseis.ucla.edu/~howard/Metadata/UC-May00/ http://www.niso.org/commitau.html http://www.ecafe.com/ http://www.gseis.ucla.edu/~howard http://www.tisch.nyu.edu/preservation/

The Challenge of Media Preservation:Digital Works & Time-based Media

besser--lazerow 12/10/02 1 the challenge of media preservation: digital works and time-based media...

Documents