the crucial role of file formats in building and preserving digital media cultures

Marco Fioretti ([email protected]) 2010/09/09 ESF Paying Attention Conferencehttp://mfioretti.com Linkopinghttp://stop.zona-m..net Some Rights Reserved

The crucial role of file formats in building and preserving Digital Media Cultures

and in the practical impact of such cultures on society

by Marco Fiorettihttp://mfioretti.net

http://stop.zona-m.net

2Marco Fioretti ([email protected]) 2010/09/09 ESF Paying Attention Conferencehttp://mfioretti.com Linkopinghttp://stop.zona-m..net Some Rights Reserved

Marco Fioretti

Freelance writer, activist and teacher about open digital standards, Free Software, digital technologies and the their relations and impact on education, ethics, civil rights and environmental issues

●Author of the Family Guide to Digital Freedom (http://digifreedom.net)●Member of:

● OpenDocument Fellowship● Digistan.org● Eleutheros.it

Author background


● In this digital world, how do we preserve the possibility of

attention from the future?

● Are we making it possible for future generations (or even to

ourselves 10 years from now) to pay attention to our digital

works?

● How do we keep or exchange attention across social classes and

across opposite sides of the digital divide?

Speaking of generational responsibilities...


● Digital media cultures are rich and powerful, but also terribly fragile

● Digital text or multimedia may be much more durable than house, but very often isn't. Why?

● especially considering that it's infinitely copiable without degradation?

● Is this related to mechanical / electrical obsolescence, e.g. 5" floppies or data tapes from the 80's unreadable because there are no more drives for those objects around? NO!

Characteristics of digital cultures


● Software creates habits and relationships, that is culture

● We don't make up information just to use cool software. We use software because we need to manage information. Software is the intermediary between people, their own digital documents and all kinds of services

● Technology (especially digital technology) is legislation

● What does this have to do with formats???

● Question: why do we change so many desktop computers every few years, even if they are still perfectly working?

Social impact of digital technologies

Basic concepts and definitions

Q: How do we create, access and preserve information?

A: Thanks to three very different things:

Physical Support: the material object containing the information

Data Format: the rules by which the information is encoded on the support

User Interface: the tools used to write and read the data according to the format

almost always, Support, Format and Interface can (and should) be independent from each other


Support, Format, Interface: analog electronic example

Support Format

Support and format are mixed here: Photograms can only be impressed on a specific type of tape, in a way not usable with other cameras and projectors

Interface

Camera and Projector that are useless with any other tape

NOTE: this is the very popular Super 8mm home movie format, released on the market in 1965 by Eastman Kodak, not widely used since the 1980's


Support, Format, Interface: non electronic example

Interface

+

Any manual writing instrument (don't forget quill and charcoal!)and your eye!

Format

Hyeroglyphs (which could also be written on paper, papyrus, wood...) and the meanings associated to each gliph

Support

The Rosetta Stone,

II Century BC


Support, Format, Interface: digital, finally!

Support

Hard drives, floppies, CDROMs, DVDs, Compact Flash drives... usable with different hardware

They all contain the same bits that can represent wildly different types of information: text, images, audio...

Format

CHARACTER ENCODING:

the meaning associated to each bit sequence:

EX: “01000001” means “A”

FILE FORMAT:how each piece of data can and

must be stored and marked:

<style:properties style:columnwidth="1.785cm"/>...

<table:tablecell><text:p>600000</text:p></table:tablecell>

+

Interface

Any software program aware of the file format, regardless of :

the hardware it runs on: x86 or Apple computer, cell phone, DVD player, remote server...

Its license of use


Why is digital information good?

If all conceivable kinds of information (from texts to music, images and 3D models) can be represented as a series of bits

We only need:

ONE class of generic storage devices: bit containers which can change shape and technology without particular problems and are very cheap

ONE (ok, very large...) class of telecom networks, ie bit transporters

And all these data can be preserved or distributed with much less money, time and effort than before!

55 73 65 20 4f 70 65 6e 44 6f 63 75 6d 65 6e 74 21


Why is digital information bad?

If all conceivable kinds of information (from texts to music, images and 3D models) can be represented as bits sequences stored in bits containers, we have (at least) two big problems:

Bit containers are much more fragile than non-digital media: parchment lasts millennia, hard drives a few years

This problem has a relatively easy solution (make many copies of information, refresh them frequently) and is outside the scope of this seminar

The second problem is that, even when the container works perfectly, the sequences of bits are absolutely useless if they are locked and we lose the key and cannot buy one:

Wor

k

Cultu

rePr

ivat

e

lifePu

blic

data55 73 65 20 4f 70 65 6e 44 6f 63 75 6d 65 6e 74 21


Format wars: Mars, 1976

July 20th, 1976:the Viking Lander is the First

Spacecraft to Operate on the Surface of Mars

2003"All the programmers had died or left

NASA”

"It was hopeless to try to go back to the original tapes”

“With the data in an unknown format, [it was necessary] to track down printouts and hire students to retype everything”

(www.cbsnews.com/stories/2003/01/21/tech/main537308.shtml)


The UK National Archives, which holds 900 years of written material, has

more than 580 terabytes of data in older file formats that are no longer

commercially available.

Chief Executive of The National Archives , Natalie Ceeney, said society faced

the possibility of "losing years of critical knowledge"

“some digital documents held by the National Archives had already been lost

forever because the programs which could read them no longer existed.”

“the issue of older file formats was a bigger problem than reading outdated

forms of media, such as floppy discs of various sizes and punch cards”.

"We are starting to find an awful lot of cases of what has been lost. What we

have got to make sure is that it doesn't get any worse."

Format wars 2007: time bomb at the UK National Archives

“The root cause of the problem is the range of proprietorial file formats that proliferated during the early digital revolution”.

Source: http://news.bbc.co.uk/2/hi/technology/6265976.stm

Test: how many other public archives are in the same situation?



● Flipnote Studio is an animation editor for Nintendo DSi game consoles

● Animations are stored in a proprietary format called PPM that has no official specification

● They can be uploaded directly only to a Nintendo Portal, not to other websites like YouTube,

Who owns the cartoons that YOU create?

● Because Nintendo has no plans to support posting YOUR Flipnotes to other websites

(thanks to Fabrizio for this information!)

Pic

ture

© 2

01

0 F

ab

rizio


“Quitime ™ and a TIFF (uncompressed)

decompressor are needed to see this

picture”

File formats issues at #patn10?

Pic

ture

© 2

01

0 F

ab

rizio

Why?


Is this “sitsim” (situated simulation)

usable on smartphones cheaper

than the iPhone, or on traditional

computers?

Interactive multimedia? Even worse...

Pic

ture

© 2

01

0 F

ab

rizio

●Above all:

● Will it still be both usable and editable when (when, not “if”)

Apple and the producers of all the software used to make it

won't be around anymore?


● Software programs are pens, file formats are the

alphabets of the digital society

● Technically speaking, file formats can almost always be completely

separated from the specific software program(s) used to manage them

● If only truly open file formats are used, it doesn't matter if some pens

are very expensive and/or patented

● Would you tolerate pens that only allow to you write your notes and

letters with the font and language that the pen maker likes?

What are file formats?


● Keeping digital data and documents in proprietary formats can be

worst than not having them at all:● it destroys or diverts (the possibility of) attention

● it creates even more digital divides

● Carelessness about formats transforms culture and (possibilities) of

paying attention

● People can only see, remember, use and understand, that is pay

attention, if digital documents and communications are in open

formats

Conclusions: pay attention to file formats


● Public document and services are increasingly managed through purely digital

data that can be published online at very low cost

● Open Data/Open Government may transform current states in active, really

participated democracy

● but only if all data are in plain sight in the right way everybody may quickly

offer public services, create wealth or "watch the watchmen"

● because public data presented in the wrong file format are almost useless since

they can't be QUICKLY processed or correlated

● E.G: a budget in PDF format instead of spreadsheet is much harder to validate

Oh, and what about (raw) data?


● Education to the role of file formats is necessary

● Only use open formats whenever it's possible

● Only use the right formats for each task

● (teach to) Go beyond "printed-pageness" and visual look of documents

(why waste more time on font or margins than on content, for text that

should be a Wiki page usable even on cell phones?)

● Teach to analyze and process data of public interests

● Oh, and of course: please, please, please...

Conclusions (2)


● Refuse to use proprietary file formats in Universities!!!!

● It creates discrimination between students that can afford the

hardware needed by closed formats

● All your work risks becoming unaccessible in a few years

● None of the problems above can be solved by software piracy

● It can be very bad PR for your University:

www.linuxjournal.com/article/8739


● This is necessary for both older and younger

people!

● "Web native" only means "born when the

WWW was already a mass-phaenomenon".

● It doesn't guarantee at all that a “Web native”

is also automatically Web savvy!

Conclusions (4)


● Absolutely Non Technical seminar on the

economic, social and cultural impact of file

formats:http://mfioretti.com/2009/02/file-formats-can-favor-or-hamper-innovation-active-citizenship-and-really-free-markets/

● Contact info: http://mfioretti.com

[email protected]

● Questions?

● Thanks!

Resources, feedback, greetings...

http://mfioretti.com/

mailto:[email protected]