1

AE@SG Media Grid Initiative

By

A/Prof. Bu-Sung Lee, Francis

Nanyang Technological University

2

Content

BackgroundArchitectureApplications

Maya(with plug-in)PovRayMental Ray

Resource Matching

3

Objectives

Illustration of an ICT application encompassing diverse components in applications, portal & middleware

Application has potential to benefit digital media industry

Complements promotion of Grid Computing in digital media industry

4

Background

Project duration: Nov’2004 till Nov 2006

Initiative of Infocomm Development Authority(iDA) Singapore Hewlett Packard(HP)

Participants School of Information Systems, Singapore

Management University School of Computer Engineering, Nanyang

Technological University Institute of High Performance, A*STAR

5

Background(2)

6

Portal

Media Grid Architecture

iHPC Meta-scheduler

ESCSubmit job

SOAP ESC

ESC

Platform LSF Meta-scheduler

NGOLSF Grid

Submit job

WWW Browser

Maya

submit

iMac

ssh

http

ssh

animator

Toolkits- Data collection- Job monitoring- Results preview- Fault-redundancy (Missing job resubmission)- Input data preprocessing/uncompressing

Applications:- Pixie- Povray- Modeler- MentalRay

Resource Matchmaker

Mental Ray/others

ESC

7

Use scenarioData

Media Portal

Charging model

QM

MAYA/3DS,etc.

Infoservice

Jobservice

othersvcs

Virtualization/Middleware

Layer

data

data

data

data

data

8

Work Packages

WP1 - Point of access for users. Provide real-time collaborative environment & Media Workbench

WP2 - Gridification of animation & image rendering applications

WP3 - Generic middleware for job execution & management

WP3 - Expanding the scope of HP – Quartermaster tool from UDC to Grid for resource virtualization & allocation

WP4 - Globus standards. No development undertaken WP5 - Aims to model usage patterns using NGO/GOG

data & work on market based resource allocations models/biz models

9

Multiple Interfaces for Job Submission

10

Step 1 – Create Model in Maya

11

Step 2 – Job Submission from Maya to the Grid

Each frame is associated with one Grid job.

12

Step 3 – Output Data Collection from Portal

The package contains all resultant TIF files.

13

Speedup Sample

The time to render a RIB file is about 40 seconds.

351 RIB files need about 4 hours for rendering in a single computer.

Clusters hpc-pdpm has 20 CPUs in 10 machines. Cluster melon has 20 CPUs in 10 machines.

By using LSF to submit the jobs to hpc-pdpm and melon, now rending 351 RIB files only need about 14 minutes including the time for data stage in/out. The speedup is about 17.

14

Job Submission from Portal to LSF

The uploaded package includes all rib files, textures and shaders.

15

Check Status for the Most Recently Submitted Job

Progress

16

Portlet of Checking Status for the Rendering Jobs Submitted from Maya and from Portal

17

2. POV-Ray Interactive Rendering

GoalsUse Grid to render complex photo-realistic

objects when a single computer is not sufficient.

Deliver the application through the rich media portal directly to the users, without installing additional software

Perform on-the fly visualization of the rendering process to give the user more control.

18

Architecture overview

Web interface

GRID

Cluster A

Nodes

Cluster B

Nodes

Portal

Proxies

Rendering is performed in parallel for every web user

19

POV-Ray Interactive Rendering

20

POV-Ray Interactive Rendering

21

POV-Ray Interactive Rendering

Rendering in progress

22

POV-Ray Interactive Rendering

23

3. Interactive Shape Modeling App.

Goals Use Grid to work with 3D shapes when a single

computer computational power is not sufficient. Deliver the application through the rich media portal

directly to the users, without installing additional software

Methods Java3D web-based interface to receive user

commands and visualize the model Real-time communication link between computing

nodes and the interface

24

Architecture overview

Web interface

GRID

Cluster A

Nodes

Cluster B

Nodes

PortalWeb clients

Proxies

Rendering is performed in parallel for every web user

25

Task distribution

1. Object is split into several parts

2. The parts are distributed among the processing nodes

3. The results are sent back and combined

4. Users are able to visualize the modelIn their browsers interactively

26

Interactive Modeling App.

27

Interactive Modeling App

28

Interactive Modeling App

29

4. Mental Ray

30

Mental Ray Arch.

Melon020Mentalray client

Melon001Mentalray server

Melon002Mentalray server

Melon003Mentalray server

Melon004Mentalray server

31

Web-Service Enabled Resource Management

Understanding User Requirements

Formal Representation of Grid Resources

page 3August 31, 2003 Grid definition

32

Web-Service Enabled Resource Management

Discovery and Matching of Grid Resources

•Problems Countered in Integration •Information Service

33

Conclusion

Maya Application plug-in and portal submission

AchievementsMental Ray submission through portal.

Integration from Application to

Information management.Integration with LSF

34

The challenge

Thanks!