virtual campfire/inmv storytelling on the iphone

Lehrstuhl Informatik 5(Informationssysteme)

Prof. Dr. M. JarkeI5-CC-0210-1

UMIC Workshop on Future Mobile

Applications

Yiwei Cao and Xi Chen

Information Systems & DatabasesRWTH Aachen University

February 26, 2010 Aachen, Germany

Virtual Campfire iNMV - Storytelling on the iPhone

UMIC Workshop on Future Mobile Applications




Applications AgendaUMIC: Ultra High-speed Mobile Information and CommunicationVirtual Campfire: community-aware context-aware semantics-aware multimedia storytelling – Mobile multimedia management– Virtual Campfire architecture and concept– Conclusions and outlook

Virtual Campfire iNMV




Applications UMIC Research ClusterA research cluster established under DFG Excellence Initiative (2006 – 2011)Interdisciplinary research cooperation in four research areas (over 15 institutions)– Mobile applications and services - Future mobile Web services

and applications – Wireless transport platforms– RF subsystem and SoC design– Performance and reliability, security and privacy, energy efficiency,

and software quality




Applications

Challenges for Mobile Multimedia Management

Communities produce and consume media at different places, times, etc.Semantic mapping of context information

– GPS data -> Tribunes, Curves, etc. (Places)– Time data -> Lap, Start, Finish, Box stop, etc. (Events)– Racing car data –> Ferrari, McLaren-Mercedes, BMW-Sauber Community, etc. (Agents)




Applications

Awareness of Community, of Context and of Semantics

To improve mobile multimedia management through bridging multimedia semantics and multimedia contextProfessional community support for multimedia management

MPEG‐7

Dublin Core

TV‐Anytime

EXIF

...

Ontology A(Commsonomy)

RDF

RDFS

OWL

Ontology B

SPARQL

Domain Information

Multimedia resources

Precise multimedia query results

Community of Practice (CoP)

Context




Applications Virtual Campfire Architecture

Data SourcesContext-Aware Services

Community Engine (LAS)

WWW

Story Template Services

Comsonomy Services

UserManager

Map Services

StorytellingServices

Object Manager

(Mobile) Interfaces

SessionManager

SNA Services

Multimedia Semantics Manager

MultimediaAnnotation

MultimediaMPEG-7 Services

MultimediaAdaptation

MultimediaPlayer

Metadata

Databases

Connectors: HTTP, SOAP

MultimediaRepository

MediaCreation

MediaSearch

MediaTagging

Semantic Browsing

Mashups

Multimedia Storytelling

MultimediaInput

Data Access

MultimediaRepositoryMultimediaRepository

Invoking services Data flows




Applications Virtual Campfire Concept

Data repository: cultural heritage, Web 2.0 user generated media etc.

Multimedia management: tagging, semantic annotation

Multimedia storytelling: collaboration, stereotyping

Contextualization

Mobile provisioning

Mobile media creation, sharing, and storytelling

Mobile information capture

Semantization




Applications ConclusionsMobile multimedia management is a complex process: cross-platform and cross-communityVirtual Campfire for mobile multimedia management

Interoperability enhancement with multimedia metadata standards [KSC*05b]Community-aware static multimedia adaptation based on community clustering algorithms [KSCa06]Context-aware dynamic multimedia adaptation based on semantic enrichment of multimedia Information [CKHJ08]Bridging multimedia semantics and multimedia context to reduce data uncertainty [CKKh09] Web 2.0 based storytelling on Community of Practice [CKMa08]Multimedia storytelling for cultural heritage domains [KCJa09]

The storytelling workshop: Workshop on Storytelling and Educational Games (STEG)




Applications

Mobile storytelling for better multimedia creation and sharing

Virtual Campfire Goes On ...

Sensing the world via mobile devicesEntertaining

Location/navigation

Social networkingPersonal information management /assistant

Mobile community support and success measure

LBS Storytelling for cultural heritage

Storytelling for mobile gaming

Situated bilingual vocabulary learning

Mobile context-aware Chinese poetry learning games

Template-based adaptive storytelling

Event based mobile gaming?

http://www.merinews.com/article/top-10-google-android-applications-in-2009/15792647.shtmlhttp://androidcommunity.com/top-10-uk-android-market-apps-revealed-by-t-mobile-20090217/

http://www.merinews.com/article/top-10-google-android-applications-in-2009/15792647.shtml

http://www.merinews.com/article/top-10-google-android-applications-in-2009/15792647.shtml

http://androidcommunity.com/top-10-uk-android-market-apps-revealed-by-t-mobile-20090217/

http://androidcommunity.com/top-10-uk-android-market-apps-revealed-by-t-mobile-20090217/




Applications ReferencesR. Klamma, Y. Cao, M. Jarke: Storytelling on the Web 2.0 as a New Means of Creating Arts, Borko Furht (eds.): Handbook of Multimedia for Digital Entertainment and Arts, Springer, 2009, pp. 623-650Y. Cao, A. Hannemann, R. Klamma, D. Renzel: A Community Success Model for Gaming Communities, Journal of Multimedia, Vol. 4, No. 2 (2009), pp. 87-93, April 2009, Academy PublisherD. Renzel, R. Klamma, Y. Cao, D. Kovachev: Virtual Campfire - Collaborative Multimedia Semantization with Mobile Social Software, Proc. of the 10th International Workshop of the Multimedia Metadata Community on Semantic Multimedia Database Technologies (SeMuDaTe'09), CEUR Workshop Proceedings, Vol. 539, Graz, Austria, December, 2009Y. Cao, M. Jarke, R. Klamma, O. Mendoza, S. Srirama: Mobile Access to MPEG-7 Based Multimedia Services, Proc. of the Tenth International Conference on Mobile Data Management, May 18-20, 2009, Taipei, Taiwan, pp. 102-111Y. Cao, R. Klamma, M. Hou, M. Jarke: Follow Me, Follow You - Spatiotemporal Community Context Modeling and Adaptation for Mobile Information Systems , Proc. of the 9th International Conference on Mobile Data Management, April 27-30, 2008, Beijing, China, pp. 108-115




Applications iNMV - Agenda

iNMV FeaturesDeveloping EnvironmentImplementationOur experiencesInstallation




Applications

iNMV FeaturesMultimedia

Media Search– Page based results browser– Scrollable original image preview– Metadata panel which shows tags,

description and MPEG-7 semantic descriptions

Media Creation– Get photos from camera or library– Annotate Media– Generate MPEG-7 compatible semantic

descriptionsKeywords Auto-completion




Applications

iNMV FeaturesStorytelling

Features– Story browser– Story player with video capability– Play, Pause, Fast Forward, and Rewind– Metadata panel

Story repository– YouTell saves stories in a repository– iNMV refreshes the story list and downloads

stories directlyStory format

– SMIL (Synchronized Multimedia Integration Language) files generated by YouTell client

– For composing interactive multimedia presentations




Applications iNMV Developing EnvironmentsiPhone 3G

– OS Version 3.1.3XCode

– Version 3.1.2– API & Conceptual Documents

iPhone SDK with Simulator– Version 3.2 beta

Language– Objective-C– C

Instrument– Monitor application performance– Measure memory usage




Applications Cocoa Design Patterns

Delegation– To notify an event is occurring– Run custom codes inside a complex object

Model View Controller– Model: The underlying data engine– View: Define the user interface– Controller: Bridge between model and view

Target-Action– One object sends messages to another– E.g. a button sends an action to the controller




Applications UI Design Principles

Displaying area are limited on the iPhone– Focus on user’s data– One thing at a time– Use Tab Bar to separate different windows– Use Navigation Bar for streamlined operations




Applications Memory Management

Conventional Cocoa memory management– Reference counting system

– Track your own references– The runtime does the memory freeing job

– Allocation/Release/Retainment– “Alloc, copy, new” operations return a retained object




Applications Memory Management Strategies

Garbage Collection is not supportedWhenever alloc an object, always remember to release itLazy Initialization to save memoryUse separated XIB files for performanceSet a variable to nil after you release it (especially for NSTimers)Do not use over 20 MB of memoryRespond to low memory warnings




Applications iNMV Communicates to VCCommunication with HTTP Connector

– Send HTTP Requests to the Connector– Requested service and method are determined by the HTTP URL– Parameters are wrapped in an XML document which is

encapsulated in the HTTP Header

The underlying XML processor– XML Encoder/Decoder– NSXML* classes are missing on the iPhone– Use Goolge‘s Objective-C wrapper around libxml2




Applications Lessons LearnedTime-consuming operations freeze the UI– They can be dispatched to other threads

Download large amount of data also causes non-responsiveness– Use asynchronized download procedure

When UI is waiting for some data processes– Show a progress bar or an activity indicator

Mobile multimedia adaptation– Delivering suitable contents to the mobile phone– Expansive operations should be taken care of by the server




Applications iNMV InstallationFor workshop participants with iPhone: installation possible at Demo sessionWe’ll setup the a profile for you during the demo, following things are needed:

– Your iPhone– Your iPhone’s Identifier(ID)

– A valid account (please register at http://vermeer.informatik.rwth-aachen.de:9080/LASRegistration/index.jsp)

– Community: UMIC

http://vermeer.informatik.rwth-aachen.de:9080/LASRegistration/index.jsp

http://vermeer.informatik.rwth-aachen.de:9080/LASRegistration/index.jsp

virtual campfire/inmv storytelling on the iphone

Technology

future mobile applicationsr

handbook of multimedia

mobile media creation

conclusions umic workshop

references umic workshop

agenda umic workshop

based storytelling

awareness of community