Suitability of Searching and Representing Multimedia Learning Resources in 3D Virtual Gaming Environment

Presented byAbu Saleh Md. Mahfujur Rahman

SupervisorProfessor Dr. Abdulmotaleb El Saddik

Multimedia Communications Research Laboratory (MCRLab)University of Ottawahttp://www.mcrlab.uottawa.ca

Outline

Motivation and Related Study Proposed System 3D Gaming Environment Implementation Result Analysis Conclusion and Future Work

Motivation and Related Study

Information visualization is the use of computer supported, interactive, visual representations of abstract data to amplify cognition.

The traditional systems: Poor way of Information visualization Less intuitive navigation space Lack of entertainment in learning

Problem statement: Suitability of visualizing learning resources from a set of available search repositories using metaphors in a structured [13] way, which improves the perception of the learners as well as offers entertainment with education.

Motivation and Related Study (cont.) Our approach:

3D virtual environment: attractive large display space as well as natural and cognitive aspects.

Using gaming metaphor: provide entertainment in navigating the environment

Organized representation of data: allows the users to directly interact with the data, get insight, and come up with new hypotheses.

a) Car metaphor

b) Spaceship metaphor

c) Library metaphor

Proposed System

Data Model LO in Repositories LO in P2P LO in Web

Graphical Model Data

Representation Model

VE Model 3D GUI Control

Figure 2. Schematic diagram of the proposed system

System Architecture

Data Model

LOM

Rep

osito

ries

LO in

P2P

VE

Mod

el

Dat

a Rep

. M

odel

Graphical Model

3D G

UI

Con

trol

Sear

chin

g

Pers

onal

izat

ion

3D Gaming Environment

Clu

ster

ing

Nav

igat

ion

Inte

ract

ion

Ran

dom

izat

ion

layer 1 layer 2

Web

Sea

rch

Res

ults

Proposed System (cont.)

Data Model: Learning Objects Communities using different elements and

vocabularies create difficulties to share and reuse resources

Learning Object: "any entity, digital or non-digital, that may be used for learning, education or training” (IEEE LTSC)

LOM is defined by IEEE LTSC for identifying, searching, and retrieving LOs

Data Model: Access Paradigm LOR contains LOs or

their references along with LOM

All elements are optional

Can be difficult to know which elements to use

Figure 3. Learning Object access paradigm

Proposed System (cont.)

LOs Repository

Shared Metadata Interface

WWW

LOM Repository

RESULTS

pdfjpeg HTML

mov

Proposed System (cont.)

Graphical Model Data Representation

Model

Ideogram: content type specification

Learning Object title

[ title ]

a) Polygonal billboard skeleton b) Rendered traffic model

Figure 4. 3D Traffic Sign Template for

representation of LOs (e.g. traffic sign for html)

TABLE IModel template representations of Learning Object

Model SelectionCollision Detection

Number of PolygonSelection Precision

Memory Requirement

Resultant Frame Rate/sec.

Billboard Easy Less (2 triangles) Imprecise Less Optimal

3D polygonal Difficult More (100+ triangles) Precise More Average

Polygonal billboard Easy Moderate (15 triangles) Precise Less Optimal

Proposed System (cont.)

Graphical Model VE Model

Avatar Terrain Reference Models

3D Gaming Environment

Searching Personalization Clustering and 3D Mapping Navigation Interaction Randomization

Yahoo, Google Web-services

P2PDirectory

Remote Repository

Query Interface

Search Result Clustering

Semantic Query Translation

Mapping to 3D Spatial Layout

laye

r 3

laye

r 2

laye

r 1

3D Virtual Traffic Arch.

Figure 5. Layered search model

3D Gaming Environment (cont.)

Clustering and 3D Mapping Display relevant and grouped search results in

the 3D road layout so that navigation and information selection become easy.

Based on the searched keywords the prototype applies heuristic methods to create relevance metrics to construct groups from the search results.

3D Gaming Environment (cont.) Clustering and 3D Mapping

Algorithm SEARCH_RESULT_CLUSTERING Progressively look-up the

search result to find relevant keywords based on the initial search keyword

Add new keywords to the keyword set and apply equation 2 to calculate distances between each pair to form keyword array matrix

Quick-Sort the matrix values based on the distances in ascending order

Apply Minimum Spanning Tree (MST) algorithm over the matrix

End SEARCH_RESULT_CLUSTERING

a) MST tree b) Road architecture to the MST tree

T

CT1

CT3

CT2

CT4

CT5

T

CT1 CT2

CT5CT4CT3

Figure 6. Semantic keyword metrics MST tree and corresponding road architecture

3D Gaming Environment (cont.)

Personalization Enables users to personalize their searching,

navigational and learning experience within the 3D world.

Maintains individual user profile that includes user’s demographic information, age limit, area of expertise, topics of interest, search history, among others.

Individual profile info are taken into consideration while searching LOs and delivering the targeted 3D virtual environment to the user.

Allows to select the 3D shape of the learning objects and also the virtual avatar

A user is able to save any visited object and put it in his/her favorite list.

3D Gaming Environment (cont.)

Navigation Collision detection is used to

help the user stay within the search layout

Appropriate ideograms, landmark, and reference models are provided.

A preview of the 3D landscape is used as a navigational map in order to show the navigating user his/her point of presence in the 3D environment.

Furthermore, a direction indicator that always points to a reference position prevents the user from getting lost in the 3D world.

Direction indicator

Point of Presence of the Avatar Car

Traffic Group Layout

Figure 7. Navigational layout of the 3D metaphor

3D Gaming Environment (cont.)

Interaction User interacts with the

displayed search results for Viewing the associated

metadata description Playing/previewing the LO Saving the search history

in the local repository for later review

The user interaction can be performed through Keyboard Mouse Joystick Microphone or any

combination of these devices.

Virtual Environment Graphical Components

Event Processing Engine

Avatar / Car LO 3D Models

Global Event Listener

Joystick Mouse Keyboard

Global Event Interface APIs

User Interaction Devices / Processes

Microphone

Virtual Environment Graphical Components

GUI Controls

Figure 8. High level user interaction model

Implementation

Software Architecture

Client Side Components Server Side Components

3D Graphics Engine

Search Interface

Metadata Extractor

User Mananger

3D Template Repository

LOM-XML Parser

LOM Clustering Module Remote

Repository

Web-Search: Yahoo/Google

Local SQL LORs

Profile Repository

P2P DirectoryList

Figure 9. Software architecture of EYE&WHY prototype

Implementation

Sample Interface: dynamic traffic layout

Figure 10. A typical 3D search interface

Group interaction menu. Comprising options viz:a) Search related b) Save allc) Share alld) Make parent

Group sign

Dynamic road for each group of search results

Dynamic navigation layout

Implementation (cont)

Sample interface: static traffic layout

Figure 11. A typical 3D search interface

Static grid based (3x3) road layout

Mouse based navigational layout

Group signs are located (based on the algorithm) on the start of the roads

Implementation (cont)

Searching/sharing learning objects

Figure 11. A typical 3D search interface

Advanced search options

Sharing LO paramers

Implementation (cont)

Searching/sharing learning objects

Figure 11. A typical 3D search interface

Implementation (cont)

Searching/sharing learning objects

Figure 11. A typical 3D search interface

Search Result Clustering:A) KeywordB) User tagC) Term frequency

Remote Repository

Web-Search: Yahoo/Google

P2P DirectoryList

[ title ]

[ title ]

(a)

(b)

(c)

(e)

(d)

Multi-modal Interactions

Visualization metaphors Information organization Information repository

Test Result Analysis

Quantitative Measurements

Figure 11. Mapped search results vs. frame rate/sec

Figure 12. Mapped search results vs. total triangle count

Test Result Analysis (cont)

Quantitative Measurements

Figure 13. Search time comparisons of different search

services

Figure 14. Search result grouping and mapping time in

3D VE

Test Result Analysis (cont)

Qualitative Measurements

Figure 15. User response percentage on the evaluation of the 3D metaphor

TABLE II User satisfaction on the overall evaluation

Ease of Use Information Clustering Visual Quality Navigation Search Response

Mean 4.489 3.834 3.79 4.184 4.497

Std. Dev. 0.5 0.9 1.0 0.6 0.3

Test Result Analysis (cont)

Qualitative Measurements

Figure 16. User satisfaction based on 3D application/gaming experience

Figure 17. Comparison between responses of users from two different

age groups.

Conclusion and future-work

We presented a new way of visualizing search results in a 3D virtual gaming environment and measured the suitability of such approach. Gaming by default has its own attractions

to the user especially to the younger users as it provides intuitiveness in the search and exploration operations that are performed while navigating the 3D world.

We evaluated our prototype system in terms of quantitative and qualitative measurements aspects.

The spatial landscape of a 3D environment, in order to represent more information at a time, is proved to be useful.

Use of enhanced user control, e.g. HMD, Data Gloves, Haptics, etc.

Finding new metaphors to enhance user experience

Applying rules and heuristics to add scoring system to achieve the game like experience

Collaboration between users in a single searching environment

Conclusion and future-work (cont.)

1. E. Duval and w. Hodgins, "a lom research agenda," proc. 12th int’l conf. World wide web (www2003), may 2003.

2. IEEE learning object metadata, final draft standard, jul. 15, 2002 (ieee1484.12.1); http://ltsc.Ieee.Org/wg12/files/lom_1484_12_1_v1_final_draft.Pdf, accessed sept. 05, 2005.

3. Edusourcecanada, canadian network of learning object repositories; http://www.Edusource.Ca/english/objects_eng.Html, accessed oct. 5, 2005.

4. Ariadne - http://www.Ariadne-eu.Org, accessed jan 17, 2005.5. W. Cellary, w. Wiza, and k. Walczak, "visualizing web search results in 3d,"

computer, vol. 37, no. 5, pp. 87-89, may 2004.6. W. Wiza, k. Walczak, and w. Cellary, "periscope - a system for adaptive 3d

visualization of search results," proc. 9th int’l conf. 3d web technology (web3d'04), pp. 29-40, apr. 2004.

7. J. Klerkx, e. Duval, and m. Meire, "using information visualization for accessing learning object repositories," proc. 8th int'l conf. Information visualization (iv'04), pp. 465-470, july 2004.

8. Lightweight java game library, http://lwjgl.Org/, accessed sept. 20, 2005.9. P. Chiu et al., “Mediametro: browsing multimedia document collections with a 3d

city metaphor,” proc. 13th annual acm international conference on multimedia, hilton, singapore, pp. 213–214, nov. 2005.

10. C. Russo dos santos, p. Gros. "Multiple views in 3d metaphoric information visualization," 6th international conference on information visualization (iv'02), pp. 468-473, 2002.

11. Wordnet - a lexical database for the english language, http://wordnet.Princeton.Edu/, accessed jan. 27, 2006.

12. Jxta – open protocols for peer to peer network, http://www.Jxta.Org/, accessed jan 20, 2006.

13. Mukherjea, S., Foley, J., and Hudson, S., (1995):“Visualizing Complex Hypermedia Networks through Multiple Hierarchical Views”. Proceedings of ACM SIGCHI '95

References

متشکرم