New Metaphors for Multimedia Authoring Environments ChristineStrothotte CarolaZwick InstituteofIndustrialDesign,Magdeburg,InteractionDesignGroup

cstrothotte@mac.com carolazwick@mac.com

Abstract

We present a new basic metaphors and concepts for aut-horing environments by distinguishing between content and form. By distinguishing between planning the scope of action (content) and the act of storytelling (form) we provide multimedia authors with means to plan com-plex, but still consistent stories and to develop new and more sophisticated formats how to tell their stories. These new metaphors also offer new ways for partici-pation to the audience by creating interactive formats of narration.

1. Introduction

We all got used to absorbing information throughmultimedia.Addressing all senses of the audience inordertocommunicateinformationisusedbroadly.Theaudiencenowadaysiseducatedindecipheringnarrativeand rhetorical concepts but also visual and auditive fi-gures.

These concepts and figures are constantly refined and thus theperceptionskillsof theaudienceaswell.Soreadingmultimediacontentseemstobeanongoingeducationalprocess.

Ontheothersidewithcreatingmultimediacontentthecaseseemstobeutterlydifferent;wehadtonoticethattheirisaveryassymetricalilliteracybetweenrea-dingandwritingmultimedia.

Whilecontentontheinternetshowsabroadinterestinsharingandpublishinginformation,thedisplayfor-mat for amateurs isoftenderived fromarchaicmediatypes likephoto albumsordiarysoften executed in aclumsyfashion.

Tools for creating more sophisticated multimediacontentstrugglebetweentwoobjectives:providingtheuserwithunlimitedopportunitiestoexpressthemselves

butatthesametimekeepingthecomplexityoftheuserinterfacegraspable.

2. Tools shape the content

Every environment influences the result of the work by endorsing specific solutions and every tool generates it‘s own aesthetic artifacts. Not only experts can tellthedifferencebetweenaPowerPointpresentationandaKeynote.

Butmorethanthevisualquality,theinternalstruc-ture of an application and the chosen metaphor define a multimediaproduct.Thechoiceoftheglobalmetaphorinfluences deeply the inherent structure of the outco-mingproduct,becausethemetaphorguidestheprodu-cer and implies a specific kind of result and the way it canbereceivedbytheconsumers.

2.1. Global Metaphors

Inordertosupporttheunderstandingandtheusagemultimedia authoring environments guide the user byintroducing a global metaphor. This metaphor ideallyembeds all interface elements and functions into onelogicalcontext.

Twothethemostpopulartools(MacromediaDirec-torandMacromediaFlash)are timebasedmetaphors.Derived from the idea of a film set the author (director) is creating action on a stage by defining snapshots in time(frames).

Whiletimebasedmetaphorsenableuserstoquicklyanimategraphicalelementsonthescreen, theyarein-consistentoratleasthardtograspwhenparallelpathsofaction are emerging or user input is defining the action.

Inadditionbyprovidingavarietyoffunctionsthatare disparate or only loosely connected to the globalmetaphortheseauthoringenvironmentsareutterlycom-plextouse.

2.2. Distinction between Content and Form

For us, the distinction between „what“, the con-tentand„how“theformwithinmultimediaproductionmeanstoseparatetheprocessofplanningactions(timeand location=content) on the one hand and the art oftellingstories(=form)on theotherhand.Wewillcallthe first scope „arena of action“ and the second „story-telling“.

2.1. Content and Form in frame based authoring environments

With framebased authoring environments all cha-ractersandelementsof thestoryarestored ina libra-ry.Theauthor thendecideswhichvisualandauditiveelements should be present at a certain point in timeby positioning these elements at a specific location on the stage in a specific frame of the score (time). With this metaphor the planning of storytelling takes placeintheauthorsmind.Theinterfacedoesnotdistinguishbetweentheactionshappeningwithinthestoryandthedecisionwhatandhowtheauthorsintendtorevealthistotheiraudienceatacertainpointintime.

Themainpointofdistinguishingbetweenformandcontentintheabovesenseisthatmorethanonestoriescanbetoldonthebasisofthesamematerial.Acompari-sonfromeverydaylifemayhighlightthepoint:Imagine10peopleattendingadinnerandareaskedafterwardsabouttheevening:youwillgetatleast10differentsto-riesalthoughall10attendedthesamedinnerandspendtheeveningprobablyinthesameroomtogether.

Tomakeitevenmoreclear:Imagine10peopleareinvited toa rockfestival. If theyareaskedafterwardsyouwillgetevenmoredifferentstories,becausetherewereprobablyparallelmusicsessionsgoingonat thesametime.

Orrememberthemovie„Babel“whichstartswithaparallelpathofdifferentstories–fourinall–thatarealllinkedtoanotherstoryandsoindirectlyareallcon-nected.As the movie progresses, the audience slowlyseeshowthedifferentscenariosareconnected.

Sothesynchronicityofeventsisamajordriverfortriggering and creating compelling stories. This syn-chronicityisnotrepresentedorsupportedbytraditionalmultimediaauthoringenvironments.

Distinguishingbetween these twophases inmulti-mediaauthoringissupportingthecreativityoftheaut-hor: He or she is able to first plan and set the scope of action(expositionorprotasis)beforefocussingontheconceptofcommunicatingthestorytotheaudience.

Mostapplicationswhichsupport theproductionofmultimedia,havelibrary-likecollectionsofdatawhichserveaspoolsofsinglemediaelements.Butthesepoolsusuallystoreonlysingledatalikepictures,movies,textsetc.

Whatcomesatleastclosetotheideaofhavingthe„scopeofaction“ontheonehandandthepossibilityof„storytelling“ separately, is to define different scenes in MacromediaFlashordifferentmoviesinMacromediaDirector.

Different storytelling is possible only by jumpingbetweenthesinglescenesormoviesindifferentways.It is symptomatic that defining the path through the dif-ferentscenesormoviescanonlydonebyprogrammingcode,whilemostofthefunctionalityinbothauthoringenvironmentsisaccessablebydirectmanipulation.

With frame based authoring environments main-tainingconsistencywithin theoverallstory,e.g. if theviewingangleischanged,isintheresponsibiltyoftheauthor´s memory and not supported by the authoringtool.

2.2. XML and HTML

TheIntroductionofXML(ExtensibleMarkupLan-guage)in1998asaW3CRecommendationisawonder-fulillustrationtoourconcept.WhileHTMLwasdesi-gnedtodisplaydataandtofocusonhowdatalooks,wasXMLdesigned to describedata and to focusonwhatdatais[w3schools].

WhileXMLdescribesthedata,aDTD(DocumentType Description) or an XML Scheme, defines how the data will look like. In our terms: XML describes thecontent, DTD describes the form; see [Castro 2000].As a result, one XML file can have several, complete-lydifferentvisualappearancesandadifferentrangeofinformation.Atimetablehastobevisualizedinacom-pletelydifferentmannerifyouwanttostudyitona17“PowerMacoronyouriPhone.

Distinguishing between content and form is veryusefulandopensupafullrangeofnewoptionsinmul-timediastorytelling.

3. Telling stories differently - telling different stories

Withtheproject„writingmultimedia“weexploredother metaphors for creating multimedia content.Thegoalwastodevelopalternativestoframebasedmeta-phorsthusprovidingalternativetoolstosupportthecre-atingofotherformatsandartifactsinmultimedia.

3.1. Inspiration: medieval panel paintings and fictional maps

Inthelatemedieval,panelpaintingsortableauswereverycommontotellcomplexstories.ThefollowingFi-gure1isatriptychbyLucasCranachTheElder.Itquo-tesfromtheOldTestament(1Kings17),whereElijah

andprophetsofBaalhadashowdowntoseewhowouldsend fire to light their alters: God lit the fire for Elijah, andtheprophetsofBaalwerekilled.ButitdepictsalsothethreeelectorsofSaxony,aswellastheartisthim-self.Thebiblicaltaledescribesaperiodofatleastthreeyears.

Fictional MapsThemapinFigure2representsthemystictimeand

location„middleearth“inthenovelLordoftheRings.By creating a detailed map of a fictional place the story is supported by „fictional facts“ and thus gains credibi-lity.Wewonderwhethertheauthoralsousedthismaptocreateplausiblerelationshipsbetweentimeandloca-tion,e.g.travelroutesforhisnovel.

Figure 1: Elijah and the Priests of Baal Lucas Cranach the Elder, 1545, Staatliche

Kunstsammlungen Dresden, © Gemäldegalerie Alte Meister

Figure 2: Map „middle earth“ in the novel „Lord of the Rings“ [Tolkien 1979]

4. Creating a narrative world

Theconsequenceof thediscussionaboutdistingu-ishingbetwencontentandformisanewenvironmentwhich allow authors to first create a narrative world and decide about their actors/elements and actions, beforetheyneedtodecidehowtoshowthisworldtotheirau-dience.

In thisenvironmentall threadsof thestorycanbeplannedandanimatedinparallelpaths.Inasecondstepthe authors use one or more cameras to film parts of the actiontakingplaceintheir„virtualfairyland“.Todistin-guishbetweenthesetwostepsofcreationsupportdiffe-rentwaysofstorytelling.Forexampletellingcomplexstoriesismadeeasierbycreatingsplitscreenanimationstoshowthedramaturgyoftheirstoryevolving.

By distinguishing between the story and the wayit will be presented to the audience help the authorsto fine tune their means of storytelling thus also refi-netheirmultimediaauthoringskills.Thispracticeisinsyncwiththebasicprincipleofpersonalcomputing„tri-al and error“: they are able to easily film their story in manydifferentvariationsinordertodecidewhichwaymightbethemostcompellingwayfortheircontentandintention.

Duringtheprojectinsummer2007,threeconceptsfollowingthisbasicideaofdistinguishingbetwencon-tent and formweredeveloped.Theprototypical reali-zationincludeinteractivesimulationsaswellasanima-tionsexplainingthewayofusage.Theseuserscenariosdifferentiate clearly between creating the content anddirecttheform.

Allthreeofthemfocusondifferenttypesofmediaand will be presented throughout the following sub-chapters.

4.1. Different stories out of one Text

Thisconceptmakesstoriesoutoftheoldestmediatypeweuseoncomputers;outoftext.Becausewetakethedistinctionbetweencontentandformreallyserious,we propose two steps: first, generating or producing the textwithinthisenvironmentandsecond,animateit.

Defining the content can be done by importing exis-tingtextsorwritinganewstory.Thecharacteristicfea-ture of the medium, its linearity will be intensified by displayingthewholetextinoneline,irrespectiveofthelengthofthetext.

Toanimatethetext,anobviousmetaphorwasused.Whilemovingavirtualvideocamera,theusercanse-lect the detail, can define the size of the text by using thevirtualzoom,cancontrolthespeedorevenimitateahandycam.

Figure3 isa shortversionofauser scenario.Theprototypewillgiveevenmoreinsights. Figure 3: Recording the text

(done by Merian Fueller)

Figure 4: Establishing the scope of action in time and location

Figure 5 explains how to use the library to add morelocationsand(moreimportant)howtheactorsappearinthescopeofaction

Figure 5: Actors appear

With defining the time span for the acts, the first phase, creating the content, is finished and the second phase,creatingtheformisreadyforaction.

The author can start to „film“ his or her story. Since thesettingismadeearliertheauthorcanrecordasmanytakesasnecessaryuntiltheresultistohisorhercom-pletesatisfaction.

Within the environment storytelling should takeplaceasarealtimerecording:bymovingthe„focusingscreen“onlythisparticularsectionofthewholearran-gementofobjectsandactionsiscaptured.

Besidethepossibilitytoredoasingletakeasmanytimesaspossible,theauthorscanalsoproducedifferenttakes of the same scene to compose a more complexmultiangularnarratione.g.bycombiningthesetakesasplit screen technique for the final montage.

4.2. The „Rashomon Effect“: creating multiangular stories

In the Japanese film directed by Akira Kurosawa [Goodwin 1993] from 1950, a woodcutter, a priest, abandit,thesamurai‘swifeandsamuraihimselfallme-morizeacrimecompletetlydifferent.Thisstoryandthistechniqueofstorytellingrevealstherelativityoftruth.

The following concept (developed by Ulrike Kin-derman)enablesaproducerofamultimediaproducttocreatethescopeofaction(content)independentlyofthemediaperformance(form).

Whenplanningthescopeofactiontheauthorisabletopaint thewholepicture includingall scenesandallpersonsinvolvedwithasmanydetailsaspersonalima-ginable.Byplanningthistheauthorisabletoestablishacrediblesettingorframeworkheorshecanrelyon.

The following twopictures inFigure4 showhowthescopeofactioniscreatedbyorganizingdifferentlo-cationshorizontallyanddifferenttimespansvertically.Thefutureisindicatedbyanincreasingbluetransparentlayertowardstheupperegdeofthewindow,thuscrea-tingtheimpressionofdepth(now:near=atthebottomandthefuture:faraway=atthetopoftheworkspace).

Figure 6: Storytelling in Action

ThelastFigureofthisconcepts(seeFigure7)showstwo different movies telling different perspectives ofonescopeofaction.

Figure 7: Different stories from different angles

4.3. The interactive comic book

The next concept (developed by Janina Südekum)focussesonaparticulargenreofstories,oncomics.Inshowingthisconceptwewillfocusontheviewofthespectator.

Figure8demonstrateshowtobuildacomicstripoutof a stack of single images by „filming“.

Figure 8: The first images form a linear comic strip

Figure 9: The Growth of the comic strip

Figure9,showsasinglelineofimages,itisastilllinearstorythatisanimatedbyslidingfromrighttoleftover the screen thus supporting conventional readinghabits.During thestorytelling, theauthordecides toopenupparallelpathsofactions.Thestorydividesintotwosto-rylines,followingeachofthetwocharactersseparately,likeinFigure10.

Figure 10: Choices for the consumer open up

ThefollowingtwopicturesinFigure11demonstratetheviewsofthespectatorafterthedecisionwhichpathheorshewantstofollow.

Figure 11: Different views depending on the choice of the consumer

5. ConclusionsSopushingthisprincipleevenfurtherwouldresult

inmakingthestoryinteractive:byofferingtheaudiencethechoicewhichthreadofthestorytheywanttofollowtheindividualperceptionandthesubjectiveviewontoaparticulareventcanbecaptured.Bydeliberatelyswit-ching between these views the audience can broadentheirview.Thistechniqueenablestheauthortoincreasethesuspenseofastorybyleavingittotheaudiencetoputthepiecesofthepuzzletogether.

Particulartoolsforsupportingcollaborationbetweenpeopleondifferentlocationscanbeaveryusefuladdi-tion.Theconceptofdistinguishingbetweencontentandformpredestinateitfordistributingdifferenttasksintheprocess of defining the scope of action as well as in the processofstorytelling.

One last crosslink:This basic concept follows theveryold,Greekartofmemory.SimonidesofCeosde-velopedthemethodofplacingororderingthingsinamemory‚palace‘whichallowsforbetterrecall.Thesememorypalaces,andthemnemotechnician‘sapproachremainedpopularuntil themiddleages.AlsoThomasAquinasrecommendedit(see[Yates1966]).Itcomestothe basic line that first you create an information space, thecontent,andasasecondstepyouthinkabouthowtotellthestory.Thisanalogyisahighdemandforfurtherresearch.

6. Acknowledgments

We would like to thank all students of the project„writingmultimedia“ likeMerianFueller,UlrikeKin-dermann, Janina Südekum,Tran LeQuyen, ConstanzeLangerandValentinLaube.

AndwewillgiveourthankstoMichaelRüger,ourlong-time partner in finding new challenges. He inspi-reduswithhisworkontheSophieprojecttothinkaboutnewwaysofproducingmultimedia.

7. References

[Castro2000]ElizabethCastro:XMLfortheWorldWideWeb(VisualQuickStartGuide),PeachPitPress,2000

[FaithandPower2004]Catalogue„FaithandPower.SaxonyintheEuropeoftheReformationEra“,

[Wurmanetal.2000]RichardSaulWurman,DavidSume,Lo-ringLeifer:InformationAnxiety2,Que,2000

[w3schools]http://www.w3schools.com/xml

[Tolkien 1979] Tolkien J.R: „Herr der Ringe“, Klett-CottaVerlag,1979

[Goodwin1993]JamesGoodwin:AkiraKurosawaandInter-textualCinema,TheJohnsHopkinsUniversityPress,1993

[Yates1966]FrancesA.Yates:TheArtofMemory,Routledge&KeganPaul,London,1966